AI Hallucination Research › Briefings

Briefings Blog

The running blog from the RLB Specialist Panel delves into real-world scenarios where the compliance, legal, or AI lab team interacts with frontier AI models under specific regulations. The blogs are anonymised to remove client-specific details and include insights from the RLB team analysing the hallucinations experienced in AI models while working on these cases. For example, when a model returns a confident answer that contradicts the regulator's primary text, such as a fabricated staff letter, a wrong appendix, or an inverted scope, these issues are discussed here. Each blog explains one set of findings and what it would have meant for the team that would have acted on it, sans this research initiative. This blog is frequently updated, a few times a day.

263 briefings in the archive · Subscribe via Atom: /briefings/feed.xml (this blog) · /feed.xml (all RegLegBrief publications)
Audience colours: AI Labs Practitioner (profession) Sector × Department
Audience
Jur.
Regulator
Profession
Sector
Dept
Range
Sort
Per page
Showing 5 of 263 · page 37 of 53
Sunday, 28 June 2026
Sector: Retail Banking and Dept: Technology & Data INT BIS-CPMI

Retail Banking Technology & Data teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO Cyber Resilience for FMIs (2016)

For Retail Banking Technology & Data teams working with Guidance on Cyber Resilience for Financial Market Infrastructures (CPMI-IOSCO 2016): Specialist-Panel-verified findings on where AI summaries diverge from the...

Technology and Data teams at retail banks designing FMI-gateway cyber controls and retail-payment cyber-resilience playbooks are increasingly relying on AI to design control documents, generate playbooks, draft architecture review papers, and prepare cyber-control mappings against the CPMI-IOSCO 2016 framework. In practice, AI is used to design retail-payment FMI-gateway cyber-control documents, generate cyber-resilience playbooks for retail-payment-system access, draft cyber-architecture review papers citing CPMI-IOSCO 2016 expectations, and prepare cyber-control mapping documents against the 2016 guidance categories.

That workflow places the regulator-issued text of the 2016 guidance, its 2018-2020 derivative standards, and its current operative status at the centre of every AI-generated deliverable for retail-banking technology and data teams.

Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confident, citable reconstructions of the CPMI-IOSCO 2016 Cyber Guidance (June 2016) that the regulator-issued primary text directly contradicts across nine findings spanning four failure classes: Source-Credit Fabrication (an asserted NIST Cybersecurity Framework citation that the 2016 guidance does not contain), Misattribution (the slogan 'secure the periphery, protect the core' located inside CPMI-IOSCO 2016 guidance or its 2018 wholesale-payments paper rather than the actual 2018 speech source), Anachronistic Cross-Reference (the 2016 guidance asserted as definitionally aligned with the November 2018 FSB Cyber Lexicon and the October 2020 FSB Effective Practices that postdate it), and Outdated Standing Claim (the 2016 guidance presented as the unchanged operative standard when CPMI-IOSCO has issued a May 2026 consultative document under active revision).

Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows retail-banking technology and data teams use AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For retail-banking technology and data teams, the failure pattern is operationally consequential. A cyber-control mapping that records an asserted NIST CSF citation in the 2016 guidance documents the mapping foundation on a wrong reading of the source. A cyber-resilience playbook that records the 2016 guidance as containing forensic-analysis-database operational depth points the engineering team at a specification level the 2016 text does not contain.

The audit's nine findings are documented with immutable RLB Citation IDs. Representative entries include RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q019-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Opus47, and RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Sonnet46. The full audit is documented at the CPMI-IOSCO 2016 Cyber Resilience Guidance hub on RegLegBrief.com.

Sector: Management & Risk Consulting and Dept: Operations INT BIS-CPMI

Management & Risk Consulting Operations teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO Cyber Resilience for FMIs (2016)

For Management & Risk Consulting Operations teams working with Guidance on Cyber Resilience for Financial Market Infrastructures (CPMI-IOSCO 2016): Specialist-Panel-verified findings on where AI summaries diverge...

Operations teams at management and risk consulting firms delivering FMI cyber-resilience target operating models and transformation roadmaps are increasingly relying on AI to draft target-operating-model documents, generate transformation roadmaps, prepare client-deliverable cyber-control gap-assessment artefacts, and produce programme design documents citing the CPMI-IOSCO 2016 framework. In practice, AI is used to draft FMI cyber-resilience target-operating-model documents, generate transformation roadmap papers citing CPMI-IOSCO 2016 expectations, prepare client-deliverable cyber-control gap-assessment artefacts, and produce cyber-resilience-programme design documents for FMI clients.

That workflow places the regulator-issued text of the 2016 guidance, its 2018-2020 derivative standards, and its current operative status at the centre of every AI-generated deliverable for consulting-operations teams.

Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confident, citable reconstructions of the CPMI-IOSCO 2016 Cyber Guidance (June 2016) that the regulator-issued primary text directly contradicts across nine findings spanning four failure classes: Source-Credit Fabrication (an asserted NIST Cybersecurity Framework citation that the 2016 guidance does not contain), Misattribution (the slogan 'secure the periphery, protect the core' located inside CPMI-IOSCO 2016 guidance or its 2018 wholesale-payments paper rather than the actual 2018 speech source), Anachronistic Cross-Reference (the 2016 guidance asserted as definitionally aligned with the November 2018 FSB Cyber Lexicon and the October 2020 FSB Effective Practices that postdate it), and Outdated Standing Claim (the 2016 guidance presented as the unchanged operative standard when CPMI-IOSCO has issued a May 2026 consultative document under active revision).

Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows consulting-operations teams use AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For consulting-operations teams, the failure pattern is operationally consequential. A client-deliverable target-operating-model document that records the 2016 guidance as containing an explicit NIST CSF citation documents the engagement foundation on a wrong reading of the source. A cyber-control gap-assessment that records the 2016 guidance as containing forensic-analysis-database operational depth introduces a regulator-criterion error into a billable client artefact.

The audit's nine findings are documented with immutable RLB Citation IDs. Representative entries include RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q019-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Opus47, and RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Sonnet46. The full audit is documented at the CPMI-IOSCO 2016 Cyber Resilience Guidance hub on RegLegBrief.com.

Saturday, 27 June 2026
Sector: Law Firms and Dept: Legal INT BIS-CPMI

Law Firms Legal teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO Cyber Resilience for FMIs (2016)

For Law Firms Legal teams working with Guidance on Cyber Resilience for Financial Market Infrastructures (CPMI-IOSCO 2016): Specialist-Panel-verified findings on where AI summaries diverge from the regulator's text,...

Legal teams at law firms advising FMIs, FMI participants, and cyber-supervisory bodies on the CPMI-IOSCO 2016 Cyber Guidance are increasingly relying on AI to draft client memoranda, validate cyber-supervisory citations, prepare partner-level briefings, and generate counsel-to-FMI-client briefings on the international guidance. In practice, AI is used to draft client memoranda on the CPMI-IOSCO 2016 Cyber Guidance, validate cyber-supervisory citation references in client deliverables, prepare partner-level briefings on FMI cyber-resilience standards, and generate counsel-to-FMI-client briefings on the 2016 guidance evolution.

That workflow places the regulator-issued text of the 2016 guidance, its 2018-2020 derivative standards, and its current operative status at the centre of every AI-generated deliverable for law-firm legal teams.

Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confident, citable reconstructions of the CPMI-IOSCO 2016 Cyber Guidance (June 2016) that the regulator-issued primary text directly contradicts across nine findings spanning four failure classes: Source-Credit Fabrication (an asserted NIST Cybersecurity Framework citation that the 2016 guidance does not contain), Misattribution (the slogan 'secure the periphery, protect the core' located inside CPMI-IOSCO 2016 guidance or its 2018 wholesale-payments paper rather than the actual 2018 speech source), Anachronistic Cross-Reference (the 2016 guidance asserted as definitionally aligned with the November 2018 FSB Cyber Lexicon and the October 2020 FSB Effective Practices that postdate it), and Outdated Standing Claim (the 2016 guidance presented as the unchanged operative standard when CPMI-IOSCO has issued a May 2026 consultative document under active revision).

Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows law-firm legal teams use AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For law-firm legal teams, the failure pattern is operationally consequential. A client memorandum that recites an explicit NIST CSF citation that the 2016 guidance does not contain misstates the regulatory foundation in counsel-to-client output. A partner-level briefing that records the 2016 guidance as the unchanged operative standard, when CPMI-IOSCO has issued a May 2026 consultative document, embeds a falsifiable status claim into a billable client deliverable.

The audit's nine findings are documented with immutable RLB Citation IDs. Representative entries include RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q019-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Opus47, and RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Sonnet46. The full audit is documented at the CPMI-IOSCO 2016 Cyber Resilience Guidance hub on RegLegBrief.com.

Sector: Cybersecurity and Dept: Operations INT BIS-CPMI

Cybersecurity Operations teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO Cyber Resilience for FMIs (2016)

For Cybersecurity Operations teams working with Guidance on Cyber Resilience for Financial Market Infrastructures (CPMI-IOSCO 2016): Specialist-Panel-verified findings on where AI summaries diverge from the...

Operations teams at cybersecurity firms delivering FMI cyber-resilience assessments and incident-response services are increasingly relying on AI to draft assessment programmes, generate incident-response playbooks, prepare client-deliverable cyber-control mappings, and produce cyber-resilience-testing scope documents against the CPMI-IOSCO 2016 framework. In practice, AI is used to draft FMI cyber-resilience assessment programmes, generate cyber-incident response playbooks citing CPMI-IOSCO 2016 expectations, prepare client-deliverable cyber-control mapping documents against the 2016 categories, and produce cyber-resilience-testing scope documents. That workflow places the regulator-issued text of the 2016 guidance, its 2018-2020 derivative standards, and its current operative status at the centre of every AI-generated deliverable for cybersecurity-operations teams.

Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confident, citable reconstructions of the CPMI-IOSCO 2016 Cyber Guidance (June 2016) that the regulator-issued primary text directly contradicts across nine findings spanning four failure classes: Source-Credit Fabrication (an asserted NIST Cybersecurity Framework citation that the 2016 guidance does not contain), Misattribution (the slogan 'secure the periphery, protect the core' located inside CPMI-IOSCO 2016 guidance or its 2018 wholesale-payments paper rather than the actual 2018 speech source), Anachronistic Cross-Reference (the 2016 guidance asserted as definitionally aligned with the November 2018 FSB Cyber Lexicon and the October 2020 FSB Effective Practices that postdate it), and Outdated Standing Claim (the 2016 guidance presented as the unchanged operative standard when CPMI-IOSCO has issued a May 2026 consultative document under active revision).

Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows cybersecurity-operations teams use AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For cybersecurity-operations teams, the failure pattern is operationally consequential. An assessment programme that records the 2016 guidance as containing an explicit NIST CSF citation documents the assessment's reference framework on a wrong reading of the source. A client-deliverable cyber-control mapping that records the 2016 guidance as containing forensic-analysis-database operational depth points the client engagement at a specification level the 2016 text does not contain. A cyber-resilience-testing scope document that records the 2016 guidance as the unchanged operative standard misstates the regulatory horizon.

The audit's nine findings are documented with immutable RLB Citation IDs. Representative entries include RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q019-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Opus47, and RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Sonnet46. The full audit is documented at the CPMI-IOSCO 2016 Cyber Resilience Guidance hub on RegLegBrief.com.

Sector: Statutory Boards & Agencies and Dept: Compliance INT BIS-CPMI

Statutory Boards & Agencies Compliance teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO Cyber Resilience for FMIs (2016)

For Statutory Boards & Agencies Compliance teams working with Guidance on Cyber Resilience for Financial Market Infrastructures (CPMI-IOSCO 2016): Specialist-Panel-verified findings on where AI summaries diverge from...

Compliance teams at statutory boards and agencies coordinating cyber-supervisory frameworks and FMI cyber-resilience oversight are increasingly relying on AI to update cyber-supervisory framework registers, generate inter-agency coordination briefings, and verify cyber-supervisory expectations against the CPMI-IOSCO 2016 source text. In practice, AI is used to update cyber-supervisory framework registers covering CPMI-IOSCO 2016 expectations, generate inter-agency cyber-coordination briefings, validate cyber-supervisory expectations against the 2016 source text, and prepare compliance reports on FMI cyber-resilience supervisory coverage.

That workflow places the regulator-issued text of the 2016 guidance, its 2018-2020 derivative standards, and its current operative status at the centre of every AI-generated deliverable for statutory-board and agency compliance teams.

Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confident, citable reconstructions of the CPMI-IOSCO 2016 Cyber Guidance (June 2016) that the regulator-issued primary text directly contradicts across nine findings spanning four failure classes: Source-Credit Fabrication (an asserted NIST Cybersecurity Framework citation that the 2016 guidance does not contain), Misattribution (the slogan 'secure the periphery, protect the core' located inside CPMI-IOSCO 2016 guidance or its 2018 wholesale-payments paper rather than the actual 2018 speech source), Anachronistic Cross-Reference (the 2016 guidance asserted as definitionally aligned with the November 2018 FSB Cyber Lexicon and the October 2020 FSB Effective Practices that postdate it), and Outdated Standing Claim (the 2016 guidance presented as the unchanged operative standard when CPMI-IOSCO has issued a May 2026 consultative document under active revision).

Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows statutory-board and agency compliance teams use AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For statutory-board and agency compliance teams, the failure pattern is operationally consequential. An inter-agency coordination briefing that records the 2016 guidance as containing an explicit NIST CSF citation misstates the international standard's actual framework references. A supervisory framework register that records the 2016 guidance and the FSB Cyber Lexicon as definitionally aligned collapses the two-year vocabulary gap. A compliance report that records the 2016 guidance as the unchanged operative standard at the reporting date misstates the regulatory horizon.

The audit's nine findings are documented with immutable RLB Citation IDs. Representative entries include RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q019-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Opus47, and RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Sonnet46. The full audit is documented at the CPMI-IOSCO 2016 Cyber Resilience Guidance hub on RegLegBrief.com.

↑ Back to top