AI Hallucination Research › Briefings

Briefings Blog

The running blog from the RLB Specialist Panel delves into real-world scenarios where the compliance, legal, or AI lab team interacts with frontier AI models under specific regulations. The blogs are anonymised to remove client-specific details and include insights from the RLB team analysing the hallucinations experienced in AI models while working on these cases. For example, when a model returns a confident answer that contradicts the regulator's primary text, such as a fabricated staff letter, a wrong appendix, or an inverted scope, these issues are discussed here. Each blog explains one set of findings and what it would have meant for the team that would have acted on it, sans this research initiative. This blog is frequently updated, a few times a day.

263 briefings in the archive · Subscribe via Atom: /briefings/feed.xml (this blog) · /feed.xml (all RegLegBrief publications)
Audience colours: AI Labs Practitioner (profession) Sector × Department
Audience
Jur.
Regulator
Profession
Sector
Dept
Range
Sort
Per page
Showing 5 of 263 · page 37 of 53
Tuesday, 22 September 2026
Sector: Management & Risk Consulting and Dept: Risk INT BIS-CPMI

Management & Risk Consulting Risk teams: documentation and reporting gaps possible from AI reading of PFMI Level 3 General Business Risk (2025)

For Management & Risk Consulting Risk teams working with Implementation Monitoring of the PFMI: Level 3 Assessment on General Business Risks: Specialist-Panel-verified findings on where AI summaries diverge from the...

Risk teams at management and risk consulting firms supporting CCP, CSD, and PS clients are increasingly using AI to design Principle 15 risk-mapping exercises, validate LNAFE sizing methodology for client liquidity-risk functions, draft scenario-analysis programme frameworks under KC2, and produce post-assessment remediation roadmaps drawing on the November 2025 CPMI-IOSCO Level 3 findings. The November 2025 CPMI-IOSCO Level 3 assessment of general business risk, recorded under PFMI Principle 15, is the supervisory exercise most directly bearing on this practice area in the current cycle.

As AI tooling enters the drafting layer, the question is no longer whether AI-assisted work product reaches client-facing deliverables; it is whether the work product reaches them with the regulator-text fidelity that consulting Risk teams need.

The RLB Specialist Panel tested two frontier AI models on a question set covering the LNAFE quantitative floor, the Basel/CRD equity carve-out condition, and the November 2025 assessment lifecycle. The Panel records 2 findings on this audience-specific cell. The failure pattern in scope: Quantitative-floor inflation into a fabricated composite minimum; Outright denial of a carve-out the rule records explicitly. Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows the respective audience uses AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For consulting Risk teams the operational consequence is direct. A remediation roadmap that inflates the KC3 six-month floor into a "greater of" dual-track minimum miscalibrates the client's regulatory baseline and produces a work programme that does not match the rule. PFMI Principle 15 is one of the cleanest primary-source surfaces in the cross-border CCP and CSD universe: a Key Consideration cited in a deliverable is either the right KC or it is not; a quantitative floor is either the regulator's text or it is not; an assessment-period date range is either accurate or it is not.

Each is recoverable on a routine line-by-line read.

The audit's 2 findings for this cell carry immutable RLB Citation IDs and are bound to verbatim regulator-issued source text held by the RLB Specialist Panel: RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q003-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q002-Sonnet46. The full audit on the November 2025 CPMI-IOSCO Level 3 assessment is published at the PFMI Level 3 General Business Risk hub on RegLegBrief.com.

Sector: Management & Risk Consulting and Dept: Compliance INT BIS-CPMI

Management & Risk Consulting Compliance teams: documentation and reporting gaps possible from AI reading of PFMI Level 3 General Business Risk (2025)

For Management & Risk Consulting Compliance teams working with Implementation Monitoring of the PFMI: Level 3 Assessment on General Business Risks: Specialist-Panel-verified findings on where AI summaries diverge...

Compliance teams at management and risk consulting firms advising CCP, CSD, and PS client portfolios are increasingly using AI to draft Principle 15 readiness gap analyses, build LNAFE compliance dashboards for client COOs, validate Basel-versus-LNAFE capital eligibility scoping memos, and prepare cross-client benchmarking decks on the November 2025 CPMI-IOSCO Level 3 assessment cycle. The November 2025 CPMI-IOSCO Level 3 assessment of general business risk, recorded under PFMI Principle 15, is the supervisory exercise most directly bearing on this practice area in the current cycle.

As AI tooling enters the drafting layer, the question is no longer whether AI-assisted work product reaches client-facing deliverables; it is whether the work product reaches them with the regulator-text fidelity that consulting Compliance teams need.

The RLB Specialist Panel tested two frontier AI models on a question set covering the LNAFE quantitative floor, the Basel/CRD equity carve-out condition, and the November 2025 assessment lifecycle. The Panel records 1 finding on this audience-specific cell. The failure pattern in scope: Source-text condition replacement with an invented overlay test. Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows the respective audience uses AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For consulting Compliance teams the operational consequence is direct. A client-facing gap analysis that frames the Basel/CRD equity carve-out as gated by a liquidity test the regulator has not issued recommends a stricter standard than the rule requires, and triggers downstream client decisions, on capital redeployment or buffer reshaping, that are not warranted by the underlying Principle.

PFMI Principle 15 is one of the cleanest primary-source surfaces in the cross-border CCP and CSD universe: a Key Consideration cited in a deliverable is either the right KC or it is not; a quantitative floor is either the regulator's text or it is not; an assessment-period date range is either accurate or it is not. Each is recoverable on a routine line-by-line read.

The audit's 1 finding for this cell carry immutable RLB Citation IDs and are bound to verbatim regulator-issued source text held by the RLB Specialist Panel: RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q002-Opus47. The full audit on the November 2025 CPMI-IOSCO Level 3 assessment is published at the PFMI Level 3 General Business Risk hub on RegLegBrief.com.

Practitioner: Public Auditors INT BIS-CPMI

Public Auditors: AI summaries of PFMI Level 3 General Business Risk (2025) may understate professional obligations

For Public Auditors working with Implementation Monitoring of the PFMI: Level 3 Assessment on General Business Risks: where Specialist-Panel-verified divergences between frontier AI summaries and the regulator's...

Public auditors performing financial-statement audits of CCPs, CSDs, and other FMIs are increasingly using AI to scope LNAFE buffer testing, draft Principle 15 compliance-walkthrough notes, validate Basel-versus-LNAFE capital eligibility assessments, and prepare audit-committee briefings on the November 2025 CPMI-IOSCO Level 3 findings cycle. The November 2025 CPMI-IOSCO Level 3 assessment of general business risk, recorded under PFMI Principle 15, is the supervisory exercise most directly bearing on this practice area in the current cycle.

As AI tooling enters the drafting layer, the question is no longer whether AI-assisted work product reaches client-facing deliverables; it is whether the work product reaches them with the regulator-text fidelity that Public Auditors need.

The RLB Specialist Panel tested two frontier AI models on a question set covering the LNAFE quantitative floor, the Basel/CRD equity carve-out condition, and the November 2025 assessment lifecycle. The Panel records 2 findings on this audience-specific cell. The failure pattern in scope: Source-text condition replacement with an invented overlay test; Quantitative-floor inflation into a fabricated composite minimum. Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows the respective audience uses AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For Public Auditors the operational consequence is direct. An audit work programme built on AI output that imports a fabricated KC4 liquidity test for Basel/CRD equity inclusion, or that frames the six-month floor as a "greater of" dual-track with a scenario-analysis sizing leg that does not appear in KC3, produces a gap analysis structurally misaligned with the Principle.

PFMI Principle 15 is one of the cleanest primary-source surfaces in the cross-border CCP and CSD universe: a Key Consideration cited in a deliverable is either the right KC or it is not; a quantitative floor is either the regulator's text or it is not; an assessment-period date range is either accurate or it is not. Each is recoverable on a routine line-by-line read.

The audit's 2 findings for this cell carry immutable RLB Citation IDs and are bound to verbatim regulator-issued source text held by the RLB Specialist Panel: RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q002-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q003-Opus47. The full audit on the November 2025 CPMI-IOSCO Level 3 assessment is published at the PFMI Level 3 General Business Risk hub on RegLegBrief.com.

Practitioner: Lawyers INT BIS-CPMI

Lawyers: AI summaries of PFMI Level 3 General Business Risk (2025) may understate professional obligations

For Lawyers working with Implementation Monitoring of the PFMI: Level 3 Assessment on General Business Risks: where Specialist-Panel-verified divergences between frontier AI summaries and the regulator's primary...

Lawyers advising on PFMI Principle 15 are increasingly using AI to draft Principle 15 compliance opinions for central counterparties, produce LNAFE eligibility briefings for liquidity-risk teams, prepare client memos on the November 2025 CPMI-IOSCO Level 3 assessment, and validate Key Consideration cross-references in regulatory submissions and consultation responses. The November 2025 CPMI-IOSCO Level 3 assessment of general business risk, recorded under PFMI Principle 15, is the supervisory exercise most directly bearing on this practice area in the current cycle.

As AI tooling enters the drafting layer, the question is no longer whether AI-assisted work product reaches client-facing deliverables; it is whether the work product reaches them with the regulator-text fidelity that Lawyers need.

The RLB Specialist Panel tested two frontier AI models on a question set covering the LNAFE quantitative floor, the Basel/CRD equity carve-out condition, and the November 2025 assessment lifecycle. The Panel records 3 findings on this audience-specific cell. The failure pattern in scope: Source-text condition replacement with an invented overlay test; Key Consideration mis-attribution of a quantitative threshold; and Supervisory-timeline truncation, dropping the validation phase. Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows the respective audience uses AI for.

The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For Lawyers the operational consequence is direct. A Principle 15 compliance opinion that misstates the KC3 Basel/CRD equity carve-out condition, or that attributes the six-month LNAFE floor to KC2 rather than KC3, is the kind of memorandum a regulator or counterparty due diligence team will challenge on first read.

PFMI Principle 15 is one of the cleanest primary-source surfaces in the cross-border CCP and CSD universe: a Key Consideration cited in a deliverable is either the right KC or it is not; a quantitative floor is either the regulator's text or it is not; an assessment-period date range is either accurate or it is not. Each is recoverable on a routine line-by-line read.

The audit's 3 findings for this cell carry immutable RLB Citation IDs and are bound to verbatim regulator-issued source text held by the RLB Specialist Panel: RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q002-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q003-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q005-Sonnet46. The full audit on the November 2025 CPMI-IOSCO Level 3 assessment is published at the PFMI Level 3 General Business Risk hub on RegLegBrief.com.

Sector: Statutory Boards & Agencies and Dept: Legal INT IMF-ELIB

Statutory Boards & Agencies Legal teams: documentation and reporting gaps possible from AI reading of IMF Financing Assurances & Sovereign Arrears Guidance (2024)

For Statutory Boards & Agencies Legal teams working with Guidance Note on the Financing Assurances and Sovereign Arrears Policies and the Fund's Role in Debt Restructurings (2024): Specialist-Panel-verified findings...

Legal teams at statutory boards and agencies engaging with the IMF Sovereign Arrears Financing-Assurances Guidance (2024) are increasingly using AI to draft inter-agency legal briefings, generate position papers on Strand 4 activation conditions, and validate IMF-policy citations in board-level, ministerial, and supervisory advice.

The RLB Specialist Panel put a set of practitioner-grade questions on the IMF Sovereign Arrears Financing-Assurances Guidance (2024) to two frontier AI models with web search active. Each question is prepared by the Panel based on the workflows that legal teams at statutory boards & agencies firms actually use AI for under this Guidance Note, covering the entry conditions for the Lending Into Official Arrears Strand 4 pathway, and the creditor-coverage rule for the 'sufficient set' in pre-emptive restructurings.

The Panel then binds every AI response to verbatim regulator-issued source text held as primary substrate, comparing the AI output line-by-line against the Guidance Note's published text. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced. On the IMF Sovereign Arrears Financing-Assurances Guidance (2024), the AI subjects returned a single hallucinated answer in the form of Fabricated-Activation-Test Hallucination for legal teams at statutory boards & agencies firms.

For legal teams at statutory boards & agencies firms advising on the IMF Sovereign Arrears Financing-Assurances Guidance (2024), treaty-style citation accuracy on IMF policy is load-bearing in legal opinions, contractual representations, due-diligence disclosures, and any pleading or position paper engaging a Fund-supported restructuring. A counterparty, opposing counsel, IMF staff reviewer, or treaty-body monitoring reviewer who identifies a fabricated Strand 4 entry condition or a fabricated pre-emptive 'sufficient set' threshold on first reading calls the entire piece of advice into question. The Strand 4 entry conditions are the gate to the Fund's most consequential financing assurance pathway.

A legal opinion built on the fabricated entry conditions either endorses premature Strand 4 invocation, or fails to identify the actual structural triggers, or both.

The published Specialist Panel findings carry the following citation identifiers:

↑ Back to top