AI Hallucination Research › Briefings

Briefings Blog

The running blog from the RLB Specialist Panel delves into real-world scenarios where the compliance, legal, or AI lab team interacts with frontier AI models under specific regulations. The blogs are anonymised to remove client-specific details and include insights from the RLB team analysing the hallucinations experienced in AI models while working on these cases. For example, when a model returns a confident answer that contradicts the regulator's primary text, such as a fabricated staff letter, a wrong appendix, or an inverted scope, these issues are discussed here. Each blog explains one set of findings and what it would have meant for the team that would have acted on it, sans this research initiative. This blog is frequently updated, a few times a day.

263 briefings in the archive · Subscribe via Atom: /briefings/feed.xml (this blog) · /feed.xml (all RegLegBrief publications)
Audience colours: AI Labs Practitioner (profession) Sector × Department
Audience
Jur.
Regulator
Profession
Sector
Dept
Range
Sort
Per page
Showing 5 of 263 · page 32 of 53
Thursday, 02 July 2026
Sector: Law Firms and Dept: Legal INT BIS-CPMI

Law Firms Legal teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO Initial Margin Disclosure (2026 consult)

For Law Firms Legal teams working with CPMI-IOSCO Consultation on Updated Guidance and Public Disclosures to Implement Initial Margin Proposals: Specialist-Panel-verified findings on where AI summaries diverge from...

Legal teams at international law firms advising CCPs, clearing members, and prime brokers on the CPMI-IOSCO Initial Margin Disclosure Consultation are increasingly using AI to draft client advisory notes on the proposed disclosure obligations, generate comment-letter submissions on the May 2026 consultative document (d232), prepare regulatory mapping memoranda for cross-jurisdictional clients, validate threshold language against the released text, and scope the implementation gap that a CCP client will need to close between its current public disclosure programme and the expectations the consultation contemplates.

The work product is partner-signed: the advisory note, the comment letter, and the mapping note are all deliverables on which the firm takes a documented position on what the regulator requires.

Two frontier AI models tested by the RLB Specialist Panel on the consultation's text on CCP override framework disclosure produced a confidently framed mandatory standard where the consultation states an expectation, and added a three-part disclosure specification that the consultation does not contain. The failure class is Source-Credit Fabrication: a structured enumeration of regulator-issued requirements with no basis in the source document, supported by a secondary commentary URL rather than the primary BIS text.

The structure of the enumeration is the part of the failure that survives a quick review; a closed three-part list reads as if it were drawn from a settled standard.

For a Legal team at an international law firm, the misframing has direct professional indemnity exposure. A client advisory note that asserts mandatory CCP disclosure of three specific categories, when none of those categories appears in the consultation, embeds a fabricated regulatory standard inside a deliverable the client will rely on for its disclosure programme and its dialogue with its lead regulator.

The exposure crystallises when the client structures or defers its disclosure programme in reliance on the mandatory characterisation, and subsequently faces supervisory challenge or compliance cost that would not have arisen had the advice correctly framed the obligation as a strong expectation. A comment-letter submission filed on the public record against the consultation will be read by the Secretariat and by other commenters, and the misstatement will be visible.

The finding is from a Specialist Panel application-style question, framed the way a senior associate or counsel would type it into an AI assistant when preparing a client advisory note for a CCP, a clearing member, or a prime broker on the scope of the consultation. The Panel bound the model output against the verbatim consultation text held as primary substrate. Citation: RLB-H-INT-BIS-CPMI-IOSCO-INITIAL-MARGIN-DISCLOSURE-CONSULT-2026-Q005-Sonnet46.

Sector: Investment Banking and Dept: Risk INT BIS-CPMI

Investment Banking Risk teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO Initial Margin Disclosure (2026 consult)

For Investment Banking Risk teams working with CPMI-IOSCO Consultation on Updated Guidance and Public Disclosures to Implement Initial Margin Proposals: Specialist-Panel-verified findings on where AI summaries...

Risk teams at internationally active investment banks holding CCP counterparty exposures under the CPMI-IOSCO Initial Margin Disclosure Consultation are increasingly using AI to scope CCP due diligence assessments, draft margin model governance policy updates, generate credit risk and collateral management committee briefing notes, prepare risk appetite documentation that references the May 2026 consultative document (d232), re-baseline CCP credit limits when a CCP's disclosure programme changes, and brief the chief risk officer on how the consultation's expected obligations translate into the firm's CCP exposure management framework.

The work product feeds directly into CCP counterparty limits, collateral haircut policy, and the firm's documented view of which CCPs are inside or outside its risk appetite.

Two frontier AI models tested by the RLB Specialist Panel on the consultation's text on CCP override framework disclosure produced a confident three-part disclosure specification that the consultation does not contain, and converted a "should" expectation into a "must" mandatory requirement. The failure class is Source-Credit Fabrication: a structured enumeration of regulator-issued requirements with no basis in the source document, supported by a secondary commentary URL rather than the primary BIS d232 cover note. The structure of the closed list, multiplied across the firm's CCP counterparty universe, generates a large number of credit and collateral decisions.

For a CCP risk team, the operational consequence is that any CCP due diligence assessment built on the AI output will hold counterparties to a phantom mandatory standard. CCPs that disclose general information on their override framework without enumerating the three fabricated categories will be flagged as deficient, distorting credit risk limits, collateral management policy, and board-level risk appetite documentation. The internal audit and second-line risk review will find that the policy rationale cites obligations not in the source text, and the firm's risk governance file will show a documented gap between its assessment criteria and the actual regulator-issued expectation.

A CCP whose disclosure happens to enumerate categories close to the fabricated three will appear over-compliant, masking actual gaps the risk team should have flagged.

The finding is from a Specialist Panel application-style question, framed the way a risk analyst would type it into an AI assistant when scoping the next CCP due diligence refresh for the credit risk and collateral management committee, with the request scoped to the override framework disclosure area specifically. The Panel bound the model output against the verbatim consultation text on the override framework, held as primary substrate. Citation: RLB-H-INT-BIS-CPMI-IOSCO-INITIAL-MARGIN-DISCLOSURE-CONSULT-2026-Q005-Sonnet46.

Sector: Investment Banking and Dept: Compliance INT BIS-CPMI

Investment Banking Compliance teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO Initial Margin Disclosure (2026 consult)

For Investment Banking Compliance teams working with CPMI-IOSCO Consultation on Updated Guidance and Public Disclosures to Implement Initial Margin Proposals: Specialist-Panel-verified findings on where AI summaries...

Compliance teams at internationally active investment banks operating under the CPMI-IOSCO Initial Margin Disclosure Consultation are increasingly using AI to verify CCP counterparty disclosure adequacy, generate margin-policy bulletins for the front office and clearing operations desks, draft regulatory mapping notes on the May 2026 consultative document (d232), update onboarding checklists for new CCP relationships, prepare supervisory submissions on the firm's CCP risk governance, and brief the second-line compliance assurance function on the consultation's implications for the firm's CCP exposure management.

The work product depends on a correct reading of whether the CPMI-IOSCO Secretariat has set a binding requirement or a strong expectation, and on whether the listed disclosure categories the AI returns actually appear in the source.

Two frontier AI models tested by the RLB Specialist Panel on the consultation's text on CCP override framework disclosure produced a confident misstatement of the obligation standard and added three disclosure categories that do not appear anywhere in the consultative document. The failure class is Source-Credit Fabrication: a structured enumeration of regulator-issued requirements that the regulator did not issue, with the obligation reframed from "should" to "must" and a secondary commentary URL given as the source. The drift sits in a single sentence about a single disclosure category but propagates wherever the CCP assessment template is used.

For a compliance officer drafting a CCP counterparty risk assessment, a board-level margin policy update, or a supervisory submission to the lead regulator, the misframing has direct enforcement consequences. A CCP assessment that holds counterparties to the fabricated three-part standard will flag compliant CCPs as deficient and trigger remediation correspondence with no regulatory basis. A supervisory submission that asserts the mandatory characterisation embeds, in the firm's official record with its regulator, a misstatement of the consultation's binding character.

Once the submission is on file, it is difficult to unwind: the firm must file a corrected submission and explain why the original was incorrect, and the file shows an entity that did not understand its own consultation-stage obligations.

The finding is from a Specialist Panel application-style question, framed the way a compliance analyst would type it into an AI assistant when scoping the next CCP counterparty disclosure assessment refresh for the prime brokerage or clearing operations desk, with the request scoped narrowly to the override framework disclosure area. The Panel bound the model output against the verbatim consultation text on the override framework, held as primary substrate. Citation: RLB-H-INT-BIS-CPMI-IOSCO-INITIAL-MARGIN-DISCLOSURE-CONSULT-2026-Q005-Sonnet46.

Wednesday, 01 July 2026
Practitioner: Company Secretaries INT BIS-CPMI

Company Secretaries: AI summaries of CPMI-IOSCO Initial Margin Disclosure (2026 consult) may understate professional obligations

For Company Secretaries working with CPMI-IOSCO Consultation on Updated Guidance and Public Disclosures to Implement Initial Margin Proposals: where Specialist-Panel-verified divergences between frontier AI summaries...

Company secretaries supporting boards of central counterparties, clearing members, and internationally active investment banks subject to the CPMI-IOSCO Initial Margin Disclosure Consultation are increasingly using AI to draft board paper summaries of the proposed CCP override framework disclosure obligations, generate audit committee briefing notes on the May 2026 consultative document (d232), prepare board resolution language for adoption of revised disclosure frameworks, validate disclosure scope statements before they enter the board pack, and produce the briefing memos that brief non-executive directors on the consultation's implications for the entity's CCP counterparty governance posture.

The work product is high-leverage: the board paper is the entity's documented understanding of the obligation, and the minute is the record of the board's adoption of that understanding.

Two frontier AI models tested by the RLB Specialist Panel on the consultation's text on CCP override framework disclosure produced a confidently framed three-part disclosure specification that the consultation does not contain, and converted a "should" expectation into a "must" mandatory requirement. The failure class is Source-Credit Fabrication: the model returned a structured enumeration of disclosure categories with the confidence of a settled obligation, citing a secondary commentary URL rather than the primary BIS text. The structure of a closed three-part list, not just the words, conveys a settledness that the consultative document does not carry.

For a company secretary, the operational consequence is that any board paper drafted with that AI output will record an obligation standard the regulator did not set, with line-item disclosure categories the regulator did not specify. Board resolutions adopted on the basis of that drafting commit the entity to a disclosure framework structured against a fabricated standard. The board minute records the adoption. The audit trail of the company secretary's review and the board's sign-off becomes evidence in any later regulatory examination that the entity built its disclosure programme to a specification not derived from the source document.

Where the company secretary supports multiple group entities, the same paper template will tend to recur, propagating the misstatement across the group's governance records before any reviewer compares the paper against the BIS source.

The finding is from a Specialist Panel application-style question, framed the way a board paper drafter would type it into an AI assistant when preparing a CCP counterparty governance update for an audit committee, with the request scoped to the override framework disclosure area specifically. The Panel bound the model output against the verbatim consultation text held as primary substrate for the question. Citation: RLB-H-INT-BIS-CPMI-IOSCO-INITIAL-MARGIN-DISCLOSURE-CONSULT-2026-Q005-Sonnet46.

Sector: Statutory Boards & Agencies and Dept: Finance INT IMF

Statutory Boards & Agencies Finance teams: documentation and reporting gaps possible from AI reading of IMF Charges & Surcharge Reform (2024)

For Statutory Boards & Agencies Finance teams working with Review of Charges and the Surcharge Policy, Reform Proposals (October 2024): Specialist-Panel-verified findings on where AI summaries diverge from the...

Finance teams at statutory boards and agencies with sovereign credit, country-risk, or multilateral-engagement remit are increasingly using AI to update country-risk tiering notes following the the IMF October 2024 Surcharge Reform, generate management information packs that quantify surcharge relief at the cohort level, and validate the IMF Board's published 20-to-13 projection before incorporating it into briefings for principals.

The RLB Specialist Panel put a set of practitioner-grade questions on the IMF October 2024 Surcharge Reform to two frontier AI models with web search active. Each question is prepared by the Panel based on the workflows that finance teams at statutory boards & agencies firms actually use AI for under this reform, covering the pre-reform baseline of surcharge-paying members, the post-reform cohort projection through fiscal year 2026, and the immediate distributional impact of the 1 November 2024 effective date.

The Panel then binds every AI response to verbatim regulator-issued source text held as primary substrate, comparing the AI output line-by-line against the IMF Executive Board's published record. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced.

On the IMF October 2024 Surcharge Reform, the AI subjects returned the same wrong cohort figure in the form of Numeric Drift, in the form of Inference Drift on one model and Outdated Retrieval on the other for finance teams at statutory boards & agencies firms.

For finance teams at statutory boards & agencies firms working with the the IMF October 2024 Surcharge Reform, the cohort figure feeds directly into internal management information packs, portfolio impact notes, investment committee briefings, and board-level papers. A document that absorbs an AI-supplied 19-to-11 figure misstates the reform's scope by one country at each end of the projection. The per-country relief count inherits the error and presents as 8 rather than 9.

Where the AI output is supported by a confident citation of an IMF press release that does not actually support the figure attributed to it, the document carries an appearance of verification it does not have. The firm-side exposure is reputational and governance-driven: a board member, rating agency, or co-investor reading the document and checking the figure against IMF.org finds the discrepancy in seconds, and the firm's primary-source verification practice becomes the next question.

The published Specialist Panel findings, with model attribution, carry the following citation identifiers, each hyperlinked to the bound regulator-issued source text on the the IMF October 2024 Surcharge Reform regulation hub. The audit register surfaces these findings for finance teams at statutory boards & agencies firms so that any AI-assisted figure entering a deliverable on the surcharge cohort, the FY2026 projection, or the per-country relief count can be re-validated against the IMF Executive Board record before the document is issued:

↑ Back to top