AI Hallucination Research › Briefings

Briefings Blog

The running blog from the RLB Specialist Panel delves into real-world scenarios where the compliance, legal, or AI lab team interacts with frontier AI models under specific regulations. The blogs are anonymised to remove client-specific details and include insights from the RLB team analysing the hallucinations experienced in AI models while working on these cases. For example, when a model returns a confident answer that contradicts the regulator's primary text, such as a fabricated staff letter, a wrong appendix, or an inverted scope, these issues are discussed here. Each blog explains one set of findings and what it would have meant for the team that would have acted on it, sans this research initiative. This blog is frequently updated, a few times a day.

263 briefings in the archive · Subscribe via Atom: /briefings/feed.xml (this blog) · /feed.xml (all RegLegBrief publications)
Audience colours: AI Labs Practitioner (profession) Sector × Department
Audience
Jur.
Regulator
Profession
Sector
Dept
Range
Sort
Per page
Showing 5 of 263 · page 50 of 53
Saturday, 29 August 2026
Sector: Investment Banking and Dept: Compliance INT BIS-CPMI

Investment Banking Compliance teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO Initial Margin Disclosure (2026 consult)

For Investment Banking Compliance teams working with CPMI-IOSCO Consultation on Updated Guidance and Public Disclosures to Implement Initial Margin Proposals: Specialist-Panel-verified findings on where AI summaries...

Compliance teams at internationally active investment banks operating under the CPMI-IOSCO Initial Margin Disclosure Consultation are increasingly using AI to verify CCP counterparty disclosure adequacy, generate margin-policy bulletins for the front office and clearing operations desks, draft regulatory mapping notes on the May 2026 consultative document (d232), update onboarding checklists for new CCP relationships, prepare supervisory submissions on the firm's CCP risk governance, and brief the second-line compliance assurance function on the consultation's implications for the firm's CCP exposure management.

The work product depends on a correct reading of whether the CPMI-IOSCO Secretariat has set a binding requirement or a strong expectation, and on whether the listed disclosure categories the AI returns actually appear in the source.

Two frontier AI models tested by the RLB Specialist Panel on the consultation's text on CCP override framework disclosure produced a confident misstatement of the obligation standard and added three disclosure categories that do not appear anywhere in the consultative document. The failure class is Source-Credit Fabrication: a structured enumeration of regulator-issued requirements that the regulator did not issue, with the obligation reframed from "should" to "must" and a secondary commentary URL given as the source. The drift sits in a single sentence about a single disclosure category but propagates wherever the CCP assessment template is used.

For a compliance officer drafting a CCP counterparty risk assessment, a board-level margin policy update, or a supervisory submission to the lead regulator, the misframing has direct enforcement consequences. A CCP assessment that holds counterparties to the fabricated three-part standard will flag compliant CCPs as deficient and trigger remediation correspondence with no regulatory basis. A supervisory submission that asserts the mandatory characterisation embeds, in the firm's official record with its regulator, a misstatement of the consultation's binding character.

Once the submission is on file, it is difficult to unwind: the firm must file a corrected submission and explain why the original was incorrect, and the file shows an entity that did not understand its own consultation-stage obligations.

The finding is from a Specialist Panel application-style question, framed the way a compliance analyst would type it into an AI assistant when scoping the next CCP counterparty disclosure assessment refresh for the prime brokerage or clearing operations desk, with the request scoped narrowly to the override framework disclosure area. The Panel bound the model output against the verbatim consultation text on the override framework, held as primary substrate. Citation: RLB-H-INT-BIS-CPMI-IOSCO-INITIAL-MARGIN-DISCLOSURE-CONSULT-2026-Q005-Sonnet46.

Practitioner: Company Secretaries INT BIS-CPMI

Company Secretaries: AI summaries of CPMI-IOSCO Initial Margin Disclosure (2026 consult) may understate professional obligations

For Company Secretaries working with CPMI-IOSCO Consultation on Updated Guidance and Public Disclosures to Implement Initial Margin Proposals: where Specialist-Panel-verified divergences between frontier AI summaries...

Company secretaries supporting boards of central counterparties, clearing members, and internationally active investment banks subject to the CPMI-IOSCO Initial Margin Disclosure Consultation are increasingly using AI to draft board paper summaries of the proposed CCP override framework disclosure obligations, generate audit committee briefing notes on the May 2026 consultative document (d232), prepare board resolution language for adoption of revised disclosure frameworks, validate disclosure scope statements before they enter the board pack, and produce the briefing memos that brief non-executive directors on the consultation's implications for the entity's CCP counterparty governance posture.

The work product is high-leverage: the board paper is the entity's documented understanding of the obligation, and the minute is the record of the board's adoption of that understanding.

Two frontier AI models tested by the RLB Specialist Panel on the consultation's text on CCP override framework disclosure produced a confidently framed three-part disclosure specification that the consultation does not contain, and converted a "should" expectation into a "must" mandatory requirement. The failure class is Source-Credit Fabrication: the model returned a structured enumeration of disclosure categories with the confidence of a settled obligation, citing a secondary commentary URL rather than the primary BIS text. The structure of a closed three-part list, not just the words, conveys a settledness that the consultative document does not carry.

For a company secretary, the operational consequence is that any board paper drafted with that AI output will record an obligation standard the regulator did not set, with line-item disclosure categories the regulator did not specify. Board resolutions adopted on the basis of that drafting commit the entity to a disclosure framework structured against a fabricated standard. The board minute records the adoption. The audit trail of the company secretary's review and the board's sign-off becomes evidence in any later regulatory examination that the entity built its disclosure programme to a specification not derived from the source document.

Where the company secretary supports multiple group entities, the same paper template will tend to recur, propagating the misstatement across the group's governance records before any reviewer compares the paper against the BIS source.

The finding is from a Specialist Panel application-style question, framed the way a board paper drafter would type it into an AI assistant when preparing a CCP counterparty governance update for an audit committee, with the request scoped to the override framework disclosure area specifically. The Panel bound the model output against the verbatim consultation text held as primary substrate for the question. Citation: RLB-H-INT-BIS-CPMI-IOSCO-INITIAL-MARGIN-DISCLOSURE-CONSULT-2026-Q005-Sonnet46.

Sector: Statutory Boards & Agencies and Dept: Finance INT IMF

Statutory Boards & Agencies Finance teams: documentation and reporting gaps possible from AI reading of IMF Charges & Surcharge Reform (2024)

For Statutory Boards & Agencies Finance teams working with Review of Charges and the Surcharge Policy, Reform Proposals (October 2024): Specialist-Panel-verified findings on where AI summaries diverge from the...

Finance teams at statutory boards and agencies with sovereign credit, country-risk, or multilateral-engagement remit are increasingly using AI to update country-risk tiering notes following the the IMF October 2024 Surcharge Reform, generate management information packs that quantify surcharge relief at the cohort level, and validate the IMF Board's published 20-to-13 projection before incorporating it into briefings for principals.

The RLB Specialist Panel put a set of practitioner-grade questions on the IMF October 2024 Surcharge Reform to two frontier AI models with web search active. Each question is prepared by the Panel based on the workflows that finance teams at statutory boards & agencies firms actually use AI for under this reform, covering the pre-reform baseline of surcharge-paying members, the post-reform cohort projection through fiscal year 2026, and the immediate distributional impact of the 1 November 2024 effective date.

The Panel then binds every AI response to verbatim regulator-issued source text held as primary substrate, comparing the AI output line-by-line against the IMF Executive Board's published record. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced.

On the IMF October 2024 Surcharge Reform, the AI subjects returned the same wrong cohort figure in the form of Numeric Drift, in the form of Inference Drift on one model and Outdated Retrieval on the other for finance teams at statutory boards & agencies firms.

For finance teams at statutory boards & agencies firms working with the the IMF October 2024 Surcharge Reform, the cohort figure feeds directly into internal management information packs, portfolio impact notes, investment committee briefings, and board-level papers. A document that absorbs an AI-supplied 19-to-11 figure misstates the reform's scope by one country at each end of the projection. The per-country relief count inherits the error and presents as 8 rather than 9.

Where the AI output is supported by a confident citation of an IMF press release that does not actually support the figure attributed to it, the document carries an appearance of verification it does not have. The firm-side exposure is reputational and governance-driven: a board member, rating agency, or co-investor reading the document and checking the figure against IMF.org finds the discrepancy in seconds, and the firm's primary-source verification practice becomes the next question.

The published Specialist Panel findings, with model attribution, carry the following citation identifiers, each hyperlinked to the bound regulator-issued source text on the the IMF October 2024 Surcharge Reform regulation hub. The audit register surfaces these findings for finance teams at statutory boards & agencies firms so that any AI-assisted figure entering a deliverable on the surcharge cohort, the FY2026 projection, or the per-country relief count can be re-validated against the IMF Executive Board record before the document is issued:

Friday, 28 August 2026
Sector: Sovereign Wealth & Investment and Dept: Treasury INT IMF

Sovereign Wealth & Investment Treasury teams: documentation and reporting gaps possible from AI reading of IMF Charges & Surcharge Reform (2024)

For Sovereign Wealth & Investment Treasury teams working with Review of Charges and the Surcharge Policy, Reform Proposals (October 2024): Specialist-Panel-verified findings on where AI summaries diverge from the...

Treasury teams at sovereign wealth and investment firms tracking IMF surcharge-paying borrowers are increasingly using AI to update debt-service trajectory models, generate internal credit assessment refreshes following the the IMF October 2024 Surcharge Reform, and validate the per-country cohort against the IMF Board's published projection before figures enter board-level materials.

The RLB Specialist Panel put a set of practitioner-grade questions on the IMF October 2024 Surcharge Reform to two frontier AI models with web search active. Each question is prepared by the Panel based on the workflows that treasury teams at sovereign wealth & investment firms actually use AI for under this reform, covering the pre-reform baseline of surcharge-paying members, the post-reform cohort projection through fiscal year 2026, and the immediate distributional impact of the 1 November 2024 effective date.

The Panel then binds every AI response to verbatim regulator-issued source text held as primary substrate, comparing the AI output line-by-line against the IMF Executive Board's published record. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced.

On the IMF October 2024 Surcharge Reform, the AI subjects returned the same wrong cohort figure in the form of Numeric Drift, in the form of Inference Drift on one model and Outdated Retrieval on the other for treasury teams at sovereign wealth & investment firms.

For treasury teams at sovereign wealth & investment firms tracking IMF surcharge-paying borrowers, the cohort figure feeds directly into debt-service trajectory models, internal credit assessments, and exposure refreshes for sovereign-credit committees. A trajectory model anchored to a 19-country pre-reform cohort and an 11-country post-reform cohort mis-identifies one borrower's surcharge status across the projection window.

Where the model feeds into per-country debt-service trajectories or portfolio-level burden-sharing revenue estimates, that single-country mis-classification compounds: every downstream analysis built on the wrong cohort path inherits the error, and reconciliation against the IMF Board's published projection requires a manual line-by-line check rather than a model refresh.

The published Specialist Panel findings, with model attribution, carry the following citation identifiers, each hyperlinked to the bound regulator-issued source text on the the IMF October 2024 Surcharge Reform regulation hub. The audit register surfaces these findings for treasury teams at sovereign wealth & investment firms so that any AI-assisted figure entering a deliverable on the surcharge cohort, the FY2026 projection, or the per-country relief count can be re-validated against the IMF Executive Board record before the document is issued:

Sector: Sovereign Wealth & Investment and Dept: Finance INT IMF

Sovereign Wealth & Investment Finance teams: documentation and reporting gaps possible from AI reading of IMF Charges & Surcharge Reform (2024)

For Sovereign Wealth & Investment Finance teams working with Review of Charges and the Surcharge Policy, Reform Proposals (October 2024): Specialist-Panel-verified findings on where AI summaries diverge from the...

Finance teams at sovereign wealth and investment firms with IMF-program-country exposure are increasingly using AI to update portfolio impact notes on the the IMF October 2024 Surcharge Reform, generate sovereign-exposure summaries for investment committees, and validate the pre-reform and post-reform cohort counts against the IMF Executive Board's published record before documents are circulated internally or to co-investors.

The RLB Specialist Panel put a set of practitioner-grade questions on the IMF October 2024 Surcharge Reform to two frontier AI models with web search active. Each question is prepared by the Panel based on the workflows that finance teams at sovereign wealth & investment firms actually use AI for under this reform, covering the pre-reform baseline of surcharge-paying members, the post-reform cohort projection through fiscal year 2026, and the immediate distributional impact of the 1 November 2024 effective date.

The Panel then binds every AI response to verbatim regulator-issued source text held as primary substrate, comparing the AI output line-by-line against the IMF Executive Board's published record. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced.

On the IMF October 2024 Surcharge Reform, the AI subjects returned the same wrong cohort figure in the form of Numeric Drift, in the form of Inference Drift on one model and Outdated Retrieval on the other for finance teams at sovereign wealth & investment firms.

For finance teams at sovereign wealth & investment firms working with the the IMF October 2024 Surcharge Reform, the cohort figure feeds directly into internal management information packs, portfolio impact notes, investment committee briefings, and board-level papers. A document that absorbs an AI-supplied 19-to-11 figure misstates the reform's scope by one country at each end of the projection. The per-country relief count inherits the error and presents as 8 rather than 9.

Where the AI output is supported by a confident citation of an IMF press release that does not actually support the figure attributed to it, the document carries an appearance of verification it does not have. The firm-side exposure is reputational and governance-driven: a board member, rating agency, or co-investor reading the document and checking the figure against IMF.org finds the discrepancy in seconds, and the firm's primary-source verification practice becomes the next question.

The published Specialist Panel findings, with model attribution, carry the following citation identifiers, each hyperlinked to the bound regulator-issued source text on the the IMF October 2024 Surcharge Reform regulation hub. The audit register surfaces these findings for finance teams at sovereign wealth & investment firms so that any AI-assisted figure entering a deliverable on the surcharge cohort, the FY2026 projection, or the per-country relief count can be re-validated against the IMF Executive Board record before the document is issued:

↑ Back to top