AI Hallucination Research › Briefings

Briefings Blog

The running blog from the RLB Specialist Panel delves into real-world scenarios where the compliance, legal, or AI lab team interacts with frontier AI models under specific regulations. The blogs are anonymised to remove client-specific details and include insights from the RLB team analysing the hallucinations experienced in AI models while working on these cases. For example, when a model returns a confident answer that contradicts the regulator's primary text, such as a fabricated staff letter, a wrong appendix, or an inverted scope, these issues are discussed here. Each blog explains one set of findings and what it would have meant for the team that would have acted on it, sans this research initiative. This blog is frequently updated, a few times a day.

263 briefings in the archive · Subscribe via Atom: /briefings/feed.xml (this blog) · /feed.xml (all RegLegBrief publications)
Audience colours: AI Labs Practitioner (profession) Sector × Department
Audience
Jur.
Regulator
Profession
Sector
Dept
Range
Sort
Per page
Showing 5 of 263 · page 17 of 53
Tuesday, 14 July 2026
Sector: Investment Banking and Dept: Risk INT BIS-CPMI

Investment Banking Risk teams: documentation and reporting gaps possible from AI reading of PFMI Level 3 General Business Risk (2025)

For Investment Banking Risk teams working with Implementation Monitoring of the PFMI: Level 3 Assessment on General Business Risks: Specialist-Panel-verified findings on where AI summaries diverge from the...

Risk teams at investment banks with significant FMI counterparty exposures are increasingly using AI to draft FMI counterparty risk scoring memos, validate LNAFE sufficiency reads for the credit committee, generate scenario-analysis commentary on FMI buffer adequacy, and prepare cross-counterparty benchmarking decks on the November 2025 CPMI-IOSCO Level 3 cycle. The November 2025 CPMI-IOSCO Level 3 assessment of general business risk, recorded under PFMI Principle 15, is the supervisory exercise most directly bearing on this practice area in the current cycle.

As AI tooling enters the drafting layer, the question is no longer whether AI-assisted work product reaches client-facing deliverables; it is whether the work product reaches them with the regulator-text fidelity that IB Risk teams need.

The RLB Specialist Panel tested two frontier AI models on a question set covering the LNAFE quantitative floor, the Basel/CRD equity carve-out condition, and the November 2025 assessment lifecycle. The Panel records 2 findings on this audience-specific cell. The failure pattern in scope: Quantitative-floor inflation into a fabricated composite minimum; Outright denial of a carve-out the rule records explicitly. Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows the respective audience uses AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For IB Risk teams the operational consequence is direct. A counterparty risk scoring memo that frames KC3 as a "greater of" composite minimum or that excludes Basel CET1 on a fabricated liquidity test miscalibrates the counterparty's regulatory baseline and drives risk decisions on a wrong floor.

PFMI Principle 15 is one of the cleanest primary-source surfaces in the cross-border CCP and CSD universe: a Key Consideration cited in a deliverable is either the right KC or it is not; a quantitative floor is either the regulator's text or it is not; an assessment-period date range is either accurate or it is not. Each is recoverable on a routine line-by-line read.

The audit's 2 findings for this cell carry immutable RLB Citation IDs and are bound to verbatim regulator-issued source text held by the RLB Specialist Panel: RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q003-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q002-Sonnet46. The full audit on the November 2025 CPMI-IOSCO Level 3 assessment is published at the PFMI Level 3 General Business Risk hub on RegLegBrief.com.

Sector: Investment Banking and Dept: Legal INT BIS-CPMI

Investment Banking Legal teams: documentation and reporting gaps possible from AI reading of PFMI Level 3 General Business Risk (2025)

For Investment Banking Legal teams working with Implementation Monitoring of the PFMI: Level 3 Assessment on General Business Risks: Specialist-Panel-verified findings on where AI summaries diverge from the...

Legal teams at investment banks acting for FMI counterparties and for derivatives clients clearing through CCPs are increasingly using AI to draft Principle 15 opinion sections for transaction documentation, prepare counterparty disclosure summaries on FMI capital sufficiency, validate cross-references in clearing-arrangement memos, and produce regulatory-engagement briefings on the November 2025 CPMI-IOSCO Level 3 cycle. The November 2025 CPMI-IOSCO Level 3 assessment of general business risk, recorded under PFMI Principle 15, is the supervisory exercise most directly bearing on this practice area in the current cycle.

As AI tooling enters the drafting layer, the question is no longer whether AI-assisted work product reaches client-facing deliverables; it is whether the work product reaches them with the regulator-text fidelity that IB Legal teams need.

The RLB Specialist Panel tested two frontier AI models on a question set covering the LNAFE quantitative floor, the Basel/CRD equity carve-out condition, and the November 2025 assessment lifecycle. The Panel records 1 finding on this audience-specific cell. The failure pattern in scope: Supervisory-timeline truncation, dropping the validation phase. Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows the respective audience uses AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For IB Legal teams the operational consequence is direct. A regulatory-engagement briefing that records the CPMI-IOSCO Level 3 assessment as a 2023-2024 exercise truncates the supervisory lifecycle and misrepresents the scope of regulator engagement with industry, and downstream client communications built on the briefing inherit the same procedural inaccuracy.

PFMI Principle 15 is one of the cleanest primary-source surfaces in the cross-border CCP and CSD universe: a Key Consideration cited in a deliverable is either the right KC or it is not; a quantitative floor is either the regulator's text or it is not; an assessment-period date range is either accurate or it is not. Each is recoverable on a routine line-by-line read.

The audit's 1 finding for this cell carry immutable RLB Citation IDs and are bound to verbatim regulator-issued source text held by the RLB Specialist Panel: RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q005-Sonnet46. The full audit on the November 2025 CPMI-IOSCO Level 3 assessment is published at the PFMI Level 3 General Business Risk hub on RegLegBrief.com.

Sector: Investment Banking and Dept: Compliance INT BIS-CPMI

Investment Banking Compliance teams: documentation and reporting gaps possible from AI reading of PFMI Level 3 General Business Risk (2025)

For Investment Banking Compliance teams working with Implementation Monitoring of the PFMI: Level 3 Assessment on General Business Risks: Specialist-Panel-verified findings on where AI summaries diverge from the...

Compliance teams at investment banks whose client coverage includes CCPs, CSDs, and other FMIs are increasingly using AI to draft Principle 15 counterparty due diligence summaries, generate exposure-monitoring memos on FMI capital sufficiency, prepare cross-counterparty benchmarking decks for the credit committee, and update regulatory-change registers on the November 2025 CPMI-IOSCO Level 3 cycle. The November 2025 CPMI-IOSCO Level 3 assessment of general business risk, recorded under PFMI Principle 15, is the supervisory exercise most directly bearing on this practice area in the current cycle.

As AI tooling enters the drafting layer, the question is no longer whether AI-assisted work product reaches client-facing deliverables; it is whether the work product reaches them with the regulator-text fidelity that IB Compliance teams need.

The RLB Specialist Panel tested two frontier AI models on a question set covering the LNAFE quantitative floor, the Basel/CRD equity carve-out condition, and the November 2025 assessment lifecycle. The Panel records 3 findings on this audience-specific cell. The failure pattern in scope: Source-text condition replacement with an invented overlay test; Key Consideration mis-attribution of a quantitative threshold; and Supervisory-timeline truncation, dropping the validation phase. Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows the respective audience uses AI for.

The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For IB Compliance teams the operational consequence is direct. A counterparty due diligence memo that misstates the KC3 Basel carve-out, that attributes the six-month floor to KC2, or that records the assessment as a 2023-2024 exercise is the kind of document a credit-committee chair, a regulator, or a counterparty challenger will catch on first read.

PFMI Principle 15 is one of the cleanest primary-source surfaces in the cross-border CCP and CSD universe: a Key Consideration cited in a deliverable is either the right KC or it is not; a quantitative floor is either the regulator's text or it is not; an assessment-period date range is either accurate or it is not. Each is recoverable on a routine line-by-line read.

The audit's 3 findings for this cell carry immutable RLB Citation IDs and are bound to verbatim regulator-issued source text held by the RLB Specialist Panel: RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q002-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q003-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q005-Sonnet46. The full audit on the November 2025 CPMI-IOSCO Level 3 assessment is published at the PFMI Level 3 General Business Risk hub on RegLegBrief.com.

Sector: Payment Institutions and Dept: Treasury INT BIS-CPMI

Payment Institutions Treasury teams: documentation and reporting gaps possible from AI reading of PFMI Level 3 General Business Risk (2025)

For Payment Institutions Treasury teams working with Implementation Monitoring of the PFMI: Level 3 Assessment on General Business Risks: Specialist-Panel-verified findings on where AI summaries diverge from the...

Treasury teams at payment institutions are increasingly using AI to draft LNAFE buffer composition memos for the group treasurer, validate Basel-versus-LNAFE capital eligibility across legal entities, prepare quarterly liquidity-buffer trend commentaries, and scope cross-cycle treasury planning against the November 2025 CPMI-IOSCO Level 3 cycle. The November 2025 CPMI-IOSCO Level 3 assessment of general business risk, recorded under PFMI Principle 15, is the supervisory exercise most directly bearing on this practice area in the current cycle.

As AI tooling enters the drafting layer, the question is no longer whether AI-assisted work product reaches client-facing deliverables; it is whether the work product reaches them with the regulator-text fidelity that PI Treasury teams need.

The RLB Specialist Panel tested two frontier AI models on a question set covering the LNAFE quantitative floor, the Basel/CRD equity carve-out condition, and the November 2025 assessment lifecycle. The Panel records 2 findings on this audience-specific cell. The failure pattern in scope: Quantitative-floor inflation into a fabricated composite minimum; Outright denial of a carve-out the rule records explicitly. Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows the respective audience uses AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For PI Treasury teams the operational consequence is direct. A treasurer's memo that frames KC3 as a "greater of" dual-track minimum overstates the regulatory floor, and a Basel eligibility memo that imports a liquidity test that does not appear in KC3 understates the eligible equity pool; either framing miscalibrates the treasury plan.

PFMI Principle 15 is one of the cleanest primary-source surfaces in the cross-border CCP and CSD universe: a Key Consideration cited in a deliverable is either the right KC or it is not; a quantitative floor is either the regulator's text or it is not; an assessment-period date range is either accurate or it is not. Each is recoverable on a routine line-by-line read.

The audit's 2 findings for this cell carry immutable RLB Citation IDs and are bound to verbatim regulator-issued source text held by the RLB Specialist Panel: RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q003-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q002-Sonnet46. The full audit on the November 2025 CPMI-IOSCO Level 3 assessment is published at the PFMI Level 3 General Business Risk hub on RegLegBrief.com.

Sector: Payment Institutions and Dept: Risk INT BIS-CPMI

Payment Institutions Risk teams: documentation and reporting gaps possible from AI reading of PFMI Level 3 General Business Risk (2025)

For Payment Institutions Risk teams working with Implementation Monitoring of the PFMI: Level 3 Assessment on General Business Risks: Specialist-Panel-verified findings on where AI summaries diverge from the...

Risk teams at payment institutions are increasingly using AI to design Principle 15 risk-mapping artefacts, draft general-business-risk scenario suites for the CRO, validate LNAFE sufficiency calculations under stress, and prepare cross-cycle benchmarking commentary on the November 2025 CPMI-IOSCO Level 3 findings. The November 2025 CPMI-IOSCO Level 3 assessment of general business risk, recorded under PFMI Principle 15, is the supervisory exercise most directly bearing on this practice area in the current cycle.

As AI tooling enters the drafting layer, the question is no longer whether AI-assisted work product reaches client-facing deliverables; it is whether the work product reaches them with the regulator-text fidelity that PI Risk teams need.

The RLB Specialist Panel tested two frontier AI models on a question set covering the LNAFE quantitative floor, the Basel/CRD equity carve-out condition, and the November 2025 assessment lifecycle. The Panel records 2 findings on this audience-specific cell. The failure pattern in scope: Quantitative-floor inflation into a fabricated composite minimum; Outright denial of a carve-out the rule records explicitly. Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows the respective audience uses AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For PI Risk teams the operational consequence is direct. A risk-mapping artefact that attributes the six-month LNAFE floor to KC2 collapses the structural distinction between the KC2 scenario-analysis obligation and the KC3 quantitative minimum, producing a risk register that does not match the Principle's architecture.

PFMI Principle 15 is one of the cleanest primary-source surfaces in the cross-border CCP and CSD universe: a Key Consideration cited in a deliverable is either the right KC or it is not; a quantitative floor is either the regulator's text or it is not; an assessment-period date range is either accurate or it is not. Each is recoverable on a routine line-by-line read.

The audit's 2 findings for this cell carry immutable RLB Citation IDs and are bound to verbatim regulator-issued source text held by the RLB Specialist Panel: RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q003-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q002-Sonnet46. The full audit on the November 2025 CPMI-IOSCO Level 3 assessment is published at the PFMI Level 3 General Business Risk hub on RegLegBrief.com.

↑ Back to top