AI Hallucination Research › Briefings

Briefings Blog

The running blog from the RLB Specialist Panel delves into real-world scenarios where the compliance, legal, or AI lab team interacts with frontier AI models under specific regulations. The blogs are anonymised to remove client-specific details and include insights from the RLB team analysing the hallucinations experienced in AI models while working on these cases. For example, when a model returns a confident answer that contradicts the regulator's primary text, such as a fabricated staff letter, a wrong appendix, or an inverted scope, these issues are discussed here. Each blog explains one set of findings and what it would have meant for the team that would have acted on it, sans this research initiative. This blog is frequently updated, a few times a day.

263 briefings in the archive · Subscribe via Atom: /briefings/feed.xml (this blog) · /feed.xml (all RegLegBrief publications)
Audience colours: AI Labs Practitioner (profession) Sector × Department
Audience
Jur.
Regulator
Profession
Sector
Dept
Range
Sort
Per page
Showing 5 of 263 · page 49 of 53
Sunday, 30 August 2026
Practitioner: Professional Engineers INT OECD

Professional Engineers: AI summaries of Recommendation of the Council on Digital Technologies and the Environment may understate professional obligations

For Professional Engineers working with Recommendation of the Council on Digital Technologies and the Environment (2025 Revision): where Specialist-Panel-verified divergences between frontier AI summaries and the...

Professional Engineers advising clients on digital infrastructure environmental impact and data-centre energy reporting are increasingly using AI to draft technical annexes referencing national data-centre energy intensity figures, prepare environmental impact assessment baseline statistics for digital-infrastructure projects, populate grid-operator consultation submissions with OECD-cited benchmark data, and verify regulator-issued statistics against primary publication chains.

The OECD's 2025 Revision of the Recommendation on Digital Technologies and the Environment carries a named, citable statistic on Ireland's data-centre share of metered electricity, drawn from Ireland's Central Statistics Office, that professional engineers will reach for when contextualising client engagements on data-centre offtake, sustainability reporting, and digital-infrastructure assurance work. That statistic is exactly the kind of figure the RLB Specialist Panel tested two frontier AI subjects against.

The RLB Specialist Panel issued a Specialist Panel application-style question on the share of Ireland's 2021 metered electricity that data centres accounted for, per the figure cited in the OECD Digital Economy Outlook 2024 chapter referenced by the 2025 Recommendation, sourced from Ireland's CSO (2023). Two frontier AI models tested by the RLB Specialist Panel returned the figure as 14 per cent and extended the answer with a four-point time series running from 5 per cent in 2015 through 21 per cent in 2023. The regulator's verbatim text records 11 per cent in 2021, with no multi-year trajectory.

The failure class is Fabricated Fact: a confidently delivered, citably attributed statistic that does not match the source document, compounded by a fabricated time series that does not appear anywhere in the OECD or CSO published record.

For professional engineers, this is operationally consequential because the wrong figure is not a vague paraphrase. It is delivered with a real source chain, CSO 2023 via OECD Digital Economy Outlook 2024, that survives standard reference-check review. A Professional Engineer who uses this AI response as a research shortcut will embed a wrong baseline statistic, 14 per cent rather than the verbatim 11 per cent, into a technical annex, an environmental impact assessment, or a policy submission, attributed to a real and reputable source chain (CSO 2023 via OECD Digital Economy Outlook 2024).

The fabricated time series (5 per cent rising to 21 per cent across 2015 to 2023) compounds the risk: it reads as contextual corroboration and would not be detected without independently verifying each year against the primary document. In a formal process, planning approval, grid operator consultation, or regulatory submission, a misattributed statistic of this kind is the type of error that surfaces under technical cross-examination and reflects on the engineer's verification practice, not merely their choice of tool.

The audit's finding on this question is published with an immutable RLB Citation ID. The relevant entry is RLB-H-INT-OECD-OECD-DIGITAL-TECHNOLOGIES-ENVIRONMENT-2025-Q006-Sonnet46. The full audit is published at the OECD Digital Technologies and the Environment Recommendation (2025 Revision) hub on RegLegBrief.com.

Practitioner: Accountants (CA/PA) INT OECD

Accountants (CA/PA): AI summaries of Recommendation of the Council on Digital Technologies and the Environment may understate professional obligations

For Accountants (CA/PA) working with Recommendation of the Council on Digital Technologies and the Environment (2025 Revision): where Specialist-Panel-verified divergences between frontier AI summaries and the...

Accountants (CA/PA) advising clients on digital infrastructure environmental impact and data-centre energy reporting are increasingly using AI to validate ESG-disclosure benchmark figures against regulator-cited statistics, draft client sustainability-opinion sections referencing OECD-cited national data, prepare due-diligence memos on carbon-accounting baselines for data-centre offtake engagements, and populate audit working papers with OECD-cited benchmark statistics.

The OECD's 2025 Revision of the Recommendation on Digital Technologies and the Environment carries a named, citable statistic on Ireland's data-centre share of metered electricity, drawn from Ireland's Central Statistics Office, that accountants will reach for when contextualising client engagements on data-centre offtake, sustainability reporting, and digital-infrastructure assurance work. That statistic is exactly the kind of figure the RLB Specialist Panel tested two frontier AI subjects against.

The RLB Specialist Panel issued a Specialist Panel application-style question on the share of Ireland's 2021 metered electricity that data centres accounted for, per the figure cited in the OECD Digital Economy Outlook 2024 chapter referenced by the 2025 Recommendation, sourced from Ireland's CSO (2023). Two frontier AI models tested by the RLB Specialist Panel returned the figure as 14 per cent and extended the answer with a four-point time series running from 5 per cent in 2015 through 21 per cent in 2023. The regulator's verbatim text records 11 per cent in 2021, with no multi-year trajectory.

The failure class is Fabricated Fact: a confidently delivered, citably attributed statistic that does not match the source document, compounded by a fabricated time series that does not appear anywhere in the OECD or CSO published record.

For accountants, this is operationally consequential because the wrong figure is not a vague paraphrase. It is delivered with a real source chain, CSO 2023 via OECD Digital Economy Outlook 2024, that survives standard reference-check review. A CA or PA who accepts the AI-generated figure at face value and includes it in a client deliverable, a sustainability opinion, a due-diligence memo, or a board briefing, will have signed off on a materially incorrect statistic attributed to a named official source. The client loses the ability to rely on that deliverable as an accurate benchmark.

If the figure is used to contextualise a disclosure in a regulated filing or an ESG-linked transaction document, correcting the record after publication or submission is costly and reputationally damaging. The fabricated time series compounds the risk: it provides apparent trend evidence that may influence investment or risk-assessment conclusions drawn by the client or a counterparty reviewing the document.

The audit's finding on this question is published with an immutable RLB Citation ID. The relevant entry is RLB-H-INT-OECD-OECD-DIGITAL-TECHNOLOGIES-ENVIRONMENT-2025-Q006-Sonnet46. The full audit is published at the OECD Digital Technologies and the Environment Recommendation (2025 Revision) hub on RegLegBrief.com.

Saturday, 29 August 2026
Sector: Investment Banking and Dept: Governance & Company Secretarial INT BIS-CPMI

Investment Banking Governance & Company Secretarial teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO Initial Margin Disclosure (2026 consult)

For Investment Banking Governance & Company Secretarial teams working with CPMI-IOSCO Consultation on Updated Guidance and Public Disclosures to Implement Initial Margin Proposals: Specialist-Panel-verified findings...

Governance and company secretarial teams at internationally active investment banks subject to the CPMI-IOSCO Initial Margin Disclosure Consultation are increasingly using AI to scope board-level briefings on CCP counterparty governance, draft policy updates on margin model oversight standards, generate audit committee papers on the May 2026 consultative document (d232), prepare board resolution language for adoption of revised CCP disclosure assessment frameworks, brief non-executive directors on the consultation's implications for the firm's CCP exposure profile, and produce the cross-jurisdictional governance mapping that records how the consultation's expected obligations translate across the firm's home and host regulator footprint.

The work product anchors the entity's documented governance position; once it enters the board pack and the minute, it is on the formal governance record.

Two frontier AI models tested by the RLB Specialist Panel on the consultation's text on CCP override framework disclosure produced a detailed three-part enumeration that the consultation does not contain, and converted a "should" expectation into a "must" mandatory requirement. The failure class is Source-Credit Fabrication: a structured enumeration of regulator-issued requirements that the regulator did not set, supported by a secondary commentary URL rather than the primary BIS d232 cover note. The structure of the closed list, more than the words, is what makes the misstatement survive a quick pre-circulation review.

For a Governance and Company Secretarial team, the operational consequence is that any board-level briefing on CCP counterparty governance, any policy update on margin model oversight standards, or any framework for assessing the adequacy of CCP disclosures that incorporates the AI output anchors the firm's governance position to a regulatory standard that was never set. Under CPMI-IOSCO's oversight framework and the jurisdiction-level prudential requirements that implement it, a regulator examining the firm's CCP counterparty risk governance could find that the firm's assessment criteria are unsupported by the actual regulatory text.

The enforcement and remediation exposure is difficult to contain once the fabricated standard is embedded in formal governance records: a corrected document is not enough, the entity needs a corrected governance record that explains why the original was incorrect.

The finding is from a Specialist Panel application-style question, framed the way a governance analyst or assistant company secretary would type it into an AI assistant when scoping the next board briefing on CCP counterparty governance, with the request scoped narrowly to the override framework disclosure area. The Panel bound the model output against the verbatim consultation text on the override framework, held as primary substrate. Citation: RLB-H-INT-BIS-CPMI-IOSCO-INITIAL-MARGIN-DISCLOSURE-CONSULT-2026-Q005-Sonnet46.

Sector: Law Firms and Dept: Legal INT BIS-CPMI

Law Firms Legal teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO Initial Margin Disclosure (2026 consult)

For Law Firms Legal teams working with CPMI-IOSCO Consultation on Updated Guidance and Public Disclosures to Implement Initial Margin Proposals: Specialist-Panel-verified findings on where AI summaries diverge from...

Legal teams at international law firms advising CCPs, clearing members, and prime brokers on the CPMI-IOSCO Initial Margin Disclosure Consultation are increasingly using AI to draft client advisory notes on the proposed disclosure obligations, generate comment-letter submissions on the May 2026 consultative document (d232), prepare regulatory mapping memoranda for cross-jurisdictional clients, validate threshold language against the released text, and scope the implementation gap that a CCP client will need to close between its current public disclosure programme and the expectations the consultation contemplates.

The work product is partner-signed: the advisory note, the comment letter, and the mapping note are all deliverables on which the firm takes a documented position on what the regulator requires.

Two frontier AI models tested by the RLB Specialist Panel on the consultation's text on CCP override framework disclosure produced a confidently framed mandatory standard where the consultation states an expectation, and added a three-part disclosure specification that the consultation does not contain. The failure class is Source-Credit Fabrication: a structured enumeration of regulator-issued requirements with no basis in the source document, supported by a secondary commentary URL rather than the primary BIS text.

The structure of the enumeration is the part of the failure that survives a quick review; a closed three-part list reads as if it were drawn from a settled standard.

For a Legal team at an international law firm, the misframing has direct professional indemnity exposure. A client advisory note that asserts mandatory CCP disclosure of three specific categories, when none of those categories appears in the consultation, embeds a fabricated regulatory standard inside a deliverable the client will rely on for its disclosure programme and its dialogue with its lead regulator.

The exposure crystallises when the client structures or defers its disclosure programme in reliance on the mandatory characterisation, and subsequently faces supervisory challenge or compliance cost that would not have arisen had the advice correctly framed the obligation as a strong expectation. A comment-letter submission filed on the public record against the consultation will be read by the Secretariat and by other commenters, and the misstatement will be visible.

The finding is from a Specialist Panel application-style question, framed the way a senior associate or counsel would type it into an AI assistant when preparing a client advisory note for a CCP, a clearing member, or a prime broker on the scope of the consultation. The Panel bound the model output against the verbatim consultation text held as primary substrate. Citation: RLB-H-INT-BIS-CPMI-IOSCO-INITIAL-MARGIN-DISCLOSURE-CONSULT-2026-Q005-Sonnet46.

Sector: Investment Banking and Dept: Risk INT BIS-CPMI

Investment Banking Risk teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO Initial Margin Disclosure (2026 consult)

For Investment Banking Risk teams working with CPMI-IOSCO Consultation on Updated Guidance and Public Disclosures to Implement Initial Margin Proposals: Specialist-Panel-verified findings on where AI summaries...

Risk teams at internationally active investment banks holding CCP counterparty exposures under the CPMI-IOSCO Initial Margin Disclosure Consultation are increasingly using AI to scope CCP due diligence assessments, draft margin model governance policy updates, generate credit risk and collateral management committee briefing notes, prepare risk appetite documentation that references the May 2026 consultative document (d232), re-baseline CCP credit limits when a CCP's disclosure programme changes, and brief the chief risk officer on how the consultation's expected obligations translate into the firm's CCP exposure management framework.

The work product feeds directly into CCP counterparty limits, collateral haircut policy, and the firm's documented view of which CCPs are inside or outside its risk appetite.

Two frontier AI models tested by the RLB Specialist Panel on the consultation's text on CCP override framework disclosure produced a confident three-part disclosure specification that the consultation does not contain, and converted a "should" expectation into a "must" mandatory requirement. The failure class is Source-Credit Fabrication: a structured enumeration of regulator-issued requirements with no basis in the source document, supported by a secondary commentary URL rather than the primary BIS d232 cover note. The structure of the closed list, multiplied across the firm's CCP counterparty universe, generates a large number of credit and collateral decisions.

For a CCP risk team, the operational consequence is that any CCP due diligence assessment built on the AI output will hold counterparties to a phantom mandatory standard. CCPs that disclose general information on their override framework without enumerating the three fabricated categories will be flagged as deficient, distorting credit risk limits, collateral management policy, and board-level risk appetite documentation. The internal audit and second-line risk review will find that the policy rationale cites obligations not in the source text, and the firm's risk governance file will show a documented gap between its assessment criteria and the actual regulator-issued expectation.

A CCP whose disclosure happens to enumerate categories close to the fabricated three will appear over-compliant, masking actual gaps the risk team should have flagged.

The finding is from a Specialist Panel application-style question, framed the way a risk analyst would type it into an AI assistant when scoping the next CCP due diligence refresh for the credit risk and collateral management committee, with the request scoped to the override framework disclosure area specifically. The Panel bound the model output against the verbatim consultation text on the override framework, held as primary substrate. Citation: RLB-H-INT-BIS-CPMI-IOSCO-INITIAL-MARGIN-DISCLOSURE-CONSULT-2026-Q005-Sonnet46.

↑ Back to top