AI Hallucination Research › Briefings

Briefings Blog

The running blog from the RLB Specialist Panel delves into real-world scenarios where the compliance, legal, or AI lab team interacts with frontier AI models under specific regulations. The blogs are anonymised to remove client-specific details and include insights from the RLB team analysing the hallucinations experienced in AI models while working on these cases. For example, when a model returns a confident answer that contradicts the regulator's primary text, such as a fabricated staff letter, a wrong appendix, or an inverted scope, these issues are discussed here. Each blog explains one set of findings and what it would have meant for the team that would have acted on it, sans this research initiative. This blog is frequently updated, a few times a day.

263 briefings in the archive · Subscribe via Atom: /briefings/feed.xml (this blog) · /feed.xml (all RegLegBrief publications)
Audience colours: AI Labs Practitioner (profession) Sector × Department
Audience
Jur.
Regulator
Profession
Sector
Dept
Range
Sort
Per page
Showing 5 of 263 · page 35 of 53
Monday, 29 June 2026
Sector: Payment Institutions and Dept: Compliance INT BIS-CPMI

Payment Institutions Compliance teams: documentation and reporting gaps possible from AI reading of PFMI (Principles for Financial Market Infrastructures)

For Payment Institutions Compliance teams working with Principles for Financial Market Infrastructures (PFMI): Specialist-Panel-verified findings on where AI summaries diverge from the regulator's text, and what that...

Compliance teams at Payment Institutions firms working on the CPMI-IOSCO Principles for Financial Market Infrastructures (PFMI, 2012) are increasingly relying on AI to scope third-party oversight programmes for CSP relationships under an FMI mandate, complete PFMI disclosure-framework responses and self-assessments, draft committee-mandate language for board submissions, and verify governance-arrangement claims against the regulator-issued Key Consideration text. The PFMI framework is the global standard for systemically important payment systems, central counterparties, and securities settlement infrastructures, and the document's structure makes it particularly amenable to AI summarisation: numbered Principles, numbered Key Considerations, and lettered annexes that the model can address by number.

That surface structure is also what makes the failure mode the RegLeg Brief Specialist Panel records here invisible at runtime: the document is regularly cited by Key Consideration number in board papers, disclosure-framework returns, and counterparty representations, which means a misattributed citation does not register as a substantive error in the draft, it registers as a competent regulatory paragraph that the reader will not check against the regulator's primary text unless something else prompts the verification.

Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confidently wrong reconstructions of the PFMI's governance and oversight architecture under Principle 2 (governance) and Annex F (oversight expectations for critical service providers). The Panel records two findings in the class the team labels "Source-Credit Fabrication and Supervisor-Scope Inversion", in which the models stated a substantively plausible governance position and pinned it to a named Key Consideration that the published PFMI text does not support. The finding identifiers are RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q011-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q022-Opus47.

For Compliance teams at Payment Institutions firms, the failure shape matters because the work product is disclosure-framework returns, self-assessment responses, CSP-oversight policies, third-party risk-management procedures, and committee-mandate submissions to the board, all of which travel under the firm's name to a board, supervisor, counterparty, or public reviewer who can locate the cited Key Consideration and check it against the regulator's primary text.

Compliance teams at payment institutions filing a disclosure-framework return or completing a self-assessment response are the population most exposed when AI output embeds a fabricated Key Consideration or inverts a supervisor's scope, because the document goes to a national authority and will be cross-checked against the regulator's primary text in any Level 2 or Level 3 monitoring review.

The Panel documents the finding identifiers RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q011-Sonnet46; RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q022-Opus47. The AI subjects under test were Claude Opus 4.7 and Claude Sonnet 4.6, each running with web search enabled, mirroring the workflow most practitioners run when they ask an assistant a Principle 2 or Annex F question. The verbatim regulator text is held as primary substrate (R2-REGULATION-d101a_PFMI_main_text.pdf). Each finding card sets out the exact strings the model produced, the verbatim regulator excerpt the model's output contradicts, and the failure-class label the RegLeg Brief Specialist Panel assigns.

The records are open-access; AI labs named in any finding have an unconditional right of reply, and the Specialist Panel will document any factual correction or contextual response alongside the original finding.

Sector: Investment Banking and Dept: Governance & Company Secretarial INT BIS-CPMI

Investment Banking Governance & Company Secretarial teams: documentation and reporting gaps possible from AI reading of PFMI (Principles for Financial Market Infrastructures)

For Investment Banking Governance & Company Secretarial teams working with Principles for Financial Market Infrastructures (PFMI): Specialist-Panel-verified findings on where AI summaries diverge from the regulator's...

Governance & Company Secretarial teams at Investment Banking firms working on the CPMI-IOSCO Principles for Financial Market Infrastructures (PFMI, 2012) are increasingly relying on AI to draft board charters and committee terms of reference under PFMI Principle 2, prepare board papers describing risk-management frameworks, generate committee-mandate templates for board subcommittees, and validate governance-policy language against the regulator-issued Key Considerations.

The PFMI framework is the global standard for systemically important payment systems, central counterparties, and securities settlement infrastructures, and the document's structure makes it particularly amenable to AI summarisation: numbered Principles, numbered Key Considerations, and lettered annexes that the model can address by number.

That surface structure is also what makes the failure mode the RegLeg Brief Specialist Panel records here invisible at runtime: the document is regularly cited by Key Consideration number in board papers, disclosure-framework returns, and counterparty representations, which means a misattributed citation does not register as a substantive error in the draft, it registers as a competent regulatory paragraph that the reader will not check against the regulator's primary text unless something else prompts the verification.

Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confidently wrong reconstructions of the PFMI's governance and oversight architecture under Principle 2 (governance) and Annex F (oversight expectations for critical service providers). The Panel records one finding in the class the team labels "Source-Credit Fabrication", in which the models stated a substantively plausible governance position and pinned it to a named Key Consideration that the published PFMI text does not support. The finding identifiers are RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q022-Opus47.

For Governance & Company Secretarial teams at Investment Banking firms, the failure shape matters because the work product is board charters, risk-committee terms of reference, governance policy manuals, and committee-mandate submissions to the FMI's board, all of which travel under the firm's name to a board, supervisor, counterparty, or public reviewer who can locate the cited Key Consideration and check it against the regulator's primary text.

Governance and Company Secretarial teams responsible for committee architecture at investment banks are the population most exposed when AI output imports a fabricated 'non-executive chair' mandate into a board charter, because the charter circulates to the board, the supervisor, and counterparty due-diligence reviewers, all of whom can locate the cited Key Consideration and check it.

The Panel documents the finding identifiers RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q022-Opus47. The AI subjects under test were Claude Opus 4.7, each running with web search enabled, mirroring the workflow most practitioners run when they ask an assistant a Principle 2 or Annex F question. The verbatim regulator text is held as primary substrate (R2-REGULATION-d101a_PFMI_main_text.pdf). Each finding card sets out the exact strings the model produced, the verbatim regulator excerpt the model's output contradicts, and the failure-class label the RegLeg Brief Specialist Panel assigns.

The records are open-access; AI labs named in any finding have an unconditional right of reply, and the Specialist Panel will document any factual correction or contextual response alongside the original finding.

Sector: Investment Banking and Dept: Compliance INT BIS-CPMI

Investment Banking Compliance teams: documentation and reporting gaps possible from AI reading of PFMI (Principles for Financial Market Infrastructures)

For Investment Banking Compliance teams working with Principles for Financial Market Infrastructures (PFMI): Specialist-Panel-verified findings on where AI summaries diverge from the regulator's text, and what that...

Compliance teams at Investment Banking firms working on the CPMI-IOSCO Principles for Financial Market Infrastructures (PFMI, 2012) are increasingly relying on AI to scope third-party oversight programmes for CSP relationships under an FMI mandate, complete PFMI disclosure-framework responses and self-assessments, draft committee-mandate language for board submissions, and verify governance-arrangement claims against the regulator-issued Key Consideration text. The PFMI framework is the global standard for systemically important payment systems, central counterparties, and securities settlement infrastructures, and the document's structure makes it particularly amenable to AI summarisation: numbered Principles, numbered Key Considerations, and lettered annexes that the model can address by number.

That surface structure is also what makes the failure mode the RegLeg Brief Specialist Panel records here invisible at runtime: the document is regularly cited by Key Consideration number in board papers, disclosure-framework returns, and counterparty representations, which means a misattributed citation does not register as a substantive error in the draft, it registers as a competent regulatory paragraph that the reader will not check against the regulator's primary text unless something else prompts the verification.

Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confidently wrong reconstructions of the PFMI's governance and oversight architecture under Principle 2 (governance) and Annex F (oversight expectations for critical service providers). The Panel records two findings in the class the team labels "Source-Credit Fabrication and Supervisor-Scope Inversion", in which the models stated a substantively plausible governance position and pinned it to a named Key Consideration that the published PFMI text does not support. The finding identifiers are RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q011-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q022-Opus47.

For Compliance teams at Investment Banking firms, the failure shape matters because the work product is disclosure-framework returns, self-assessment responses, CSP-oversight policies, third-party risk-management procedures, and committee-mandate submissions to the board, all of which travel under the firm's name to a board, supervisor, counterparty, or public reviewer who can locate the cited Key Consideration and check it against the regulator's primary text.

Compliance teams at investment banks signing off on a disclosure-framework return or a self-assessment response are the population most exposed when AI output embeds a fabricated Key Consideration or inverts a supervisor's scope, because the document goes to a national authority or the FSB-coordinated monitoring review and will be cross-checked against the regulator's primary text.

The Panel documents the finding identifiers RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q011-Sonnet46; RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q022-Opus47. The AI subjects under test were Claude Opus 4.7 and Claude Sonnet 4.6, each running with web search enabled, mirroring the workflow most practitioners run when they ask an assistant a Principle 2 or Annex F question. The verbatim regulator text is held as primary substrate (R2-REGULATION-d101a_PFMI_main_text.pdf). Each finding card sets out the exact strings the model produced, the verbatim regulator excerpt the model's output contradicts, and the failure-class label the RegLeg Brief Specialist Panel assigns.

The records are open-access; AI labs named in any finding have an unconditional right of reply, and the Specialist Panel will document any factual correction or contextual response alongside the original finding.

Sector: Payment Institutions and Dept: Legal INT BIS-CPMI

Payment Institutions Legal teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO Cyber Resilience for FMIs (2016)

For Payment Institutions Legal teams working with Guidance on Cyber Resilience for Financial Market Infrastructures (CPMI-IOSCO 2016): Specialist-Panel-verified findings on where AI summaries diverge from the...

Legal teams at payment institutions advising on FMI participation, cyber-incident notification, and cyber-supervisory citation referencing are increasingly relying on AI to draft FMI-participation legal memoranda, generate notification language for regulator filings, prepare counsel-to-board briefings, and validate citation references in contractual and regulatory deliverables. In practice, AI is used to draft FMI-participation legal memoranda, generate cyber-incident notification language for regulator filings, prepare counsel-to-board briefings on CPMI-IOSCO 2016 expectations, and validate cyber-supervisory citation references in contractual and regulatory deliverables.

That workflow places the regulator-issued text of the 2016 guidance, its 2018-2020 derivative standards, and its current operative status at the centre of every AI-generated deliverable for payment-institution legal teams.

Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confident, citable reconstructions of the CPMI-IOSCO 2016 Cyber Guidance (June 2016) that the regulator-issued primary text directly contradicts across nine findings spanning four failure classes: Source-Credit Fabrication (an asserted NIST Cybersecurity Framework citation that the 2016 guidance does not contain), Misattribution (the slogan 'secure the periphery, protect the core' located inside CPMI-IOSCO 2016 guidance or its 2018 wholesale-payments paper rather than the actual 2018 speech source), Anachronistic Cross-Reference (the 2016 guidance asserted as definitionally aligned with the November 2018 FSB Cyber Lexicon and the October 2020 FSB Effective Practices that postdate it), and Outdated Standing Claim (the 2016 guidance presented as the unchanged operative standard when CPMI-IOSCO has issued a May 2026 consultative document under active revision).

Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows payment-institution legal teams use AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For payment-institution legal teams, the failure pattern is operationally consequential. A legal memorandum that recites an explicit NIST CSF citation that the 2016 guidance does not contain misstates the regulatory foundation. A counsel-to-board briefing that records the 2016 guidance as the unchanged operative standard, when CPMI-IOSCO has issued a May 2026 consultative document, embeds a falsifiable status claim into a regulated deliverable.

The audit's nine findings are documented with immutable RLB Citation IDs. Representative entries include RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q019-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Opus47, and RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Sonnet46. The full audit is documented at the CPMI-IOSCO 2016 Cyber Resilience Guidance hub on RegLegBrief.com.

Sector: Management & Risk Consulting and Dept: Compliance INT BIS-CPMI

Management & Risk Consulting Compliance teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO Cyber Resilience for FMIs (2016)

For Management & Risk Consulting Compliance teams working with Guidance on Cyber Resilience for Financial Market Infrastructures (CPMI-IOSCO 2016): Specialist-Panel-verified findings on where AI summaries diverge...

Compliance-practice teams at management and risk consulting firms delivering FMI cyber-supervisory readiness assessments and compliance-programme design are increasingly relying on AI to draft readiness assessments, generate programme design papers, prepare cyber-compliance gap analyses, and produce supervisory-coverage briefings citing the CPMI-IOSCO 2016 framework. In practice, AI is used to draft FMI cyber-supervisory readiness assessments, generate compliance-programme design papers citing CPMI-IOSCO 2016 expectations, prepare client-deliverable cyber-compliance gap analyses, and produce cyber-supervisory-coverage briefings for FMI participants and FMI operators.

That workflow places the regulator-issued text of the 2016 guidance, its 2018-2020 derivative standards, and its current operative status at the centre of every AI-generated deliverable for consulting compliance-practice teams.

Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confident, citable reconstructions of the CPMI-IOSCO 2016 Cyber Guidance (June 2016) that the regulator-issued primary text directly contradicts across nine findings spanning four failure classes: Source-Credit Fabrication (an asserted NIST Cybersecurity Framework citation that the 2016 guidance does not contain), Misattribution (the slogan 'secure the periphery, protect the core' located inside CPMI-IOSCO 2016 guidance or its 2018 wholesale-payments paper rather than the actual 2018 speech source), Anachronistic Cross-Reference (the 2016 guidance asserted as definitionally aligned with the November 2018 FSB Cyber Lexicon and the October 2020 FSB Effective Practices that postdate it), and Outdated Standing Claim (the 2016 guidance presented as the unchanged operative standard when CPMI-IOSCO has issued a May 2026 consultative document under active revision).

Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows consulting compliance-practice teams use AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For consulting compliance-practice teams, the failure pattern is operationally consequential. A readiness-assessment document that records the 2016 guidance as containing an explicit NIST CSF citation documents the engagement criterion on a wrong reading of the source. A compliance-programme design paper that records the 2016 guidance and the FSB Cyber Lexicon as definitionally aligned collapses a two-year vocabulary gap and lands inside a billable client deliverable.

The audit's nine findings are documented with immutable RLB Citation IDs. Representative entries include RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q019-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Opus47, and RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Sonnet46. The full audit is documented at the CPMI-IOSCO 2016 Cyber Resilience Guidance hub on RegLegBrief.com.

↑ Back to top