AI Hallucination Research › Briefings

Briefings Blog

The running blog from the RLB Specialist Panel delves into real-world scenarios where the compliance, legal, or AI lab team interacts with frontier AI models under specific regulations. The blogs are anonymised to remove client-specific details and include insights from the RLB team analysing the hallucinations experienced in AI models while working on these cases. For example, when a model returns a confident answer that contradicts the regulator's primary text, such as a fabricated staff letter, a wrong appendix, or an inverted scope, these issues are discussed here. Each blog explains one set of findings and what it would have meant for the team that would have acted on it, sans this research initiative. This blog is frequently updated, a few times a day.

263 briefings in the archive · Subscribe via Atom: /briefings/feed.xml (this blog) · /feed.xml (all RegLegBrief publications)
Audience colours: AI Labs Practitioner (profession) Sector × Department
Audience
Jur.
Regulator
Profession
Sector
Dept
Range
Sort
Per page
Showing 5 of 263 · page 6 of 53
Wednesday, 22 July 2026
Practitioner: Company Secretaries INT BIS-CPMI

Company Secretaries: AI summaries of CPMI FPS Interlinking Governance 2024 may understate professional obligations

For Company Secretaries working with Linking Fast Payment Systems Across Borders: Governance and Oversight — Final Report: where Specialist-Panel-verified divergences between frontier AI summaries and the regulator's...

Misstated recommendation count, fabricated consultation respondents, and scoping drift on the CPMI October 2024 final report on FPS interlinking governance. Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confident, citable answers across six distinct questions on the October 2024 CPMI final report on linking fast payment systems across borders that the regulator's own primary text in publication d223 directly contradicts. The audit covers the count and scope of the report's oversight recommendations, the named list of public consultation respondents in Annex 1, and the distinction between the interim publication d219 and the final publication d223.

For Company Secretaries working CPMI FPS interlinking governance matters, the failure pattern is operationally consequential. The audit tested six questions designed by the RLB Specialist Panel to mirror how lawyers, compliance officers, risk officers, operations leads, and board secretariats at FPS operators, hub entities, payment institutions, and banks actually use AI on this practice area: board packs and committee memoranda on FPS interlinking governance, secretariat notes on oversight expectations, written resolutions on cross-border payment interlinking arrangements, and minute records of board deliberation on the report. Each question is bound to verbatim regulator-issued primary substrate.

Across the six findings the AI subjects committed, in board-style and analyst-style deliverables, to approximately ten oversight recommendations (against the seven set out in d223 Section 5.2), to consultation-respondent lists of fifteen to twenty named organisations (against the seven specific respondents recorded in d223 Annex 1), and to a scoping treatment that places the single access point gateway model inside the report's recommendations (against d223 Section 2.2, which records that the single access point is not the focus of the report).

The findings are operationally consequential for legal teams, compliance teams, risk teams, operations teams, and board secretariats at payment institutions, banks, hub entities, and FPS operators whose practice touches the October 2024 CPMI final report. A board-level briefing memo that records the report as setting out approximately ten recommendations conflates the interim d219's ten considerations with the final d223's seven recommendations and embeds that confusion into the board's understanding of the oversight regime. A legal opinion that scopes a single access point arrangement inside the d223 recommendation set creates a falsifiable regulatory-interpretation error in a partner-level deliverable.

An analyst report that lists fifteen to twenty named consultation respondents (including fabricated organisation names) attributes positions and counts to stakeholders that the regulator's Annex 1 does not record.

The audit's six findings are published with immutable RLB Citation IDs. Representative entries include RLB-H-INT-BIS-CPMI-CPMI-FPS-INTERLINKING-GOVERNANCE-2024-Q001-Opus47, RLB-H-INT-BIS-CPMI-CPMI-FPS-INTERLINKING-GOVERNANCE-2024-Q002-Sonnet46, RLB-H-INT-BIS-CPMI-CPMI-FPS-INTERLINKING-GOVERNANCE-2024-Q003-Opus47, RLB-H-INT-BIS-CPMI-CPMI-FPS-INTERLINKING-GOVERNANCE-2024-Q006-Opus47. The full audit is published at the CPMI FPS Interlinking Governance 2024 hub on RegLegBrief.com.

Practitioner: Lawyers INT BIS-CPMI

Lawyers: AI summaries of CPMI FPS Interlinking Governance 2024 may understate professional obligations

For Lawyers working with Linking Fast Payment Systems Across Borders: Governance and Oversight — Final Report: where Specialist-Panel-verified divergences between frontier AI summaries and the regulator's primary...

Misstated recommendation count, fabricated consultation respondents, and scoping drift on the CPMI October 2024 final report on FPS interlinking governance. Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confident, citable answers across six distinct questions on the October 2024 CPMI final report on linking fast payment systems across borders that the regulator's own primary text in publication d223 directly contradicts. The audit covers the count and scope of the report's oversight recommendations, the named list of public consultation respondents in Annex 1, and the distinction between the interim publication d219 and the final publication d223.

For Lawyers working CPMI FPS interlinking governance matters, the failure pattern is operationally consequential. The audit tested six questions designed by the RLB Specialist Panel to mirror how lawyers, compliance officers, risk officers, operations leads, and board secretariats at FPS operators, hub entities, payment institutions, and banks actually use AI on this practice area: legal opinions on FPS interlinking arrangements, board memoranda on cross-border fast-payment governance, counsel notes for in-house teams at FPS operators and hub entities, and submissions to oversight authorities on interlinking-related risks. Each question is bound to verbatim regulator-issued primary substrate.

Across the six findings the AI subjects committed, in board-style and analyst-style deliverables, to approximately ten oversight recommendations (against the seven set out in d223 Section 5.2), to consultation-respondent lists of fifteen to twenty named organisations (against the seven specific respondents recorded in d223 Annex 1), and to a scoping treatment that places the single access point gateway model inside the report's recommendations (against d223 Section 2.2, which records that the single access point is not the focus of the report).

The findings are operationally consequential for legal teams, compliance teams, risk teams, operations teams, and board secretariats at payment institutions, banks, hub entities, and FPS operators whose practice touches the October 2024 CPMI final report. A board-level briefing memo that records the report as setting out approximately ten recommendations conflates the interim d219's ten considerations with the final d223's seven recommendations and embeds that confusion into the board's understanding of the oversight regime. A legal opinion that scopes a single access point arrangement inside the d223 recommendation set creates a falsifiable regulatory-interpretation error in a partner-level deliverable.

An analyst report that lists fifteen to twenty named consultation respondents (including fabricated organisation names) attributes positions and counts to stakeholders that the regulator's Annex 1 does not record.

The audit's six findings are published with immutable RLB Citation IDs. Representative entries include RLB-H-INT-BIS-CPMI-CPMI-FPS-INTERLINKING-GOVERNANCE-2024-Q001-Opus47, RLB-H-INT-BIS-CPMI-CPMI-FPS-INTERLINKING-GOVERNANCE-2024-Q003-Opus47, RLB-H-INT-BIS-CPMI-CPMI-FPS-INTERLINKING-GOVERNANCE-2024-Q005-Opus47, RLB-H-INT-BIS-CPMI-CPMI-FPS-INTERLINKING-GOVERNANCE-2024-Q006-Opus47. The full audit is published at the CPMI FPS Interlinking Governance 2024 hub on RegLegBrief.com.

AI Labs INT BIS-CPMI

Alert: Frontier AI models misread CPMI FPS Interlinking Governance 2024

RegLegBrief's Specialist Panel finds frontier AI models with web search enabled diverge from the regulator's verbatim text of Linking Fast Payment Systems Across Borders: Governance and Oversight, Final Report....

Misstated recommendation count, fabricated consultation respondents, and scoping drift on the CPMI October 2024 final report on FPS interlinking governance. Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confident, citable answers across six distinct questions on the October 2024 CPMI final report on linking fast payment systems across borders that the regulator's own primary text in publication d223 directly contradicts. The audit covers the count and scope of the report's oversight recommendations, the named list of public consultation respondents in Annex 1, and the distinction between the interim publication d219 and the final publication d223.

For AI lab teams working CPMI FPS interlinking governance matters, the failure pattern is operationally consequential. The audit tested six questions designed by the RLB Specialist Panel to mirror how AI lab teams fielding frontier models into cross-border fast-payment, payment-system oversight, and central-bank advisory deployments actually use AI on this practice area: evaluation harnesses on international standard-setting publications, internal probing of model behaviour on CPMI publication records, and partnership conversations with regulators and supervisors on AI-assisted oversight work. Each question is bound to verbatim regulator-issued primary substrate.

Across the six findings the AI subjects committed, in board-style and analyst-style deliverables, to approximately ten oversight recommendations (against the seven set out in d223 Section 5.2), to consultation-respondent lists of fifteen to twenty named organisations (against the seven specific respondents recorded in d223 Annex 1), and to a scoping treatment that places the single access point gateway model inside the report's recommendations (against d223 Section 2.2, which records that the single access point is not the focus of the report).

The findings are operationally consequential for legal teams, compliance teams, risk teams, operations teams, and board secretariats at payment institutions, banks, hub entities, and FPS operators whose practice touches the October 2024 CPMI final report. A board-level briefing memo that records the report as setting out approximately ten recommendations conflates the interim d219's ten considerations with the final d223's seven recommendations and embeds that confusion into the board's understanding of the oversight regime. A legal opinion that scopes a single access point arrangement inside the d223 recommendation set creates a falsifiable regulatory-interpretation error in a partner-level deliverable.

An analyst report that lists fifteen to twenty named consultation respondents (including fabricated organisation names) attributes positions and counts to stakeholders that the regulator's Annex 1 does not record.

The audit's six findings are published with immutable RLB Citation IDs. Representative entries include RLB-H-INT-BIS-CPMI-CPMI-FPS-INTERLINKING-GOVERNANCE-2024-Q001-Opus47, RLB-H-INT-BIS-CPMI-CPMI-FPS-INTERLINKING-GOVERNANCE-2024-Q002-Sonnet46, RLB-H-INT-BIS-CPMI-CPMI-FPS-INTERLINKING-GOVERNANCE-2024-Q003-Opus47, RLB-H-INT-BIS-CPMI-CPMI-FPS-INTERLINKING-GOVERNANCE-2024-Q004-Sonnet46, RLB-H-INT-BIS-CPMI-CPMI-FPS-INTERLINKING-GOVERNANCE-2024-Q005-Opus47, RLB-H-INT-BIS-CPMI-CPMI-FPS-INTERLINKING-GOVERNANCE-2024-Q006-Opus47. The full audit is published at the CPMI FPS Interlinking Governance 2024 hub on RegLegBrief.com.

Sector: Hedge Funds and Dept: Compliance INT BIS-CPMI

Hedge Funds Compliance teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO VM Effective Practices 2025

For Hedge Funds Compliance teams working with Streamlining Variation Margin in Centrally Cleared Markets — Examples of Effective Practices: Specialist-Panel-verified findings on where AI summaries diverge from the...

Compliance teams at hedge funds clearing derivatives through central counterparties are increasingly using AI to draft regulatory monitoring updates citing CPMI-IOSCO d226, prepare board compliance committee briefings on VM operational expectations, classify the January 2025 publication in the firm's horizon-scanning log, and validate proposed updates to client-clearing onboarding language for institutional investors. Leading AI assistants tested by the RLB Specialist Panel produced confident, citable answers on the binding force of d226 that the document itself directly contradicts.

The RLB Specialist Panel tested whether two frontier AI models could correctly characterise the legal status of d226, asking them to classify each of the eight effective practices set out in the document as either a mandatory requirement with enforcement consequences, a supervisory expectation that regulators will test against, or voluntary guidance with no binding legal force. The exercise targeted what the Panel calls inverted modality: AI commitments that flip the binding force of a source text from voluntary illustration to supervisory or mandatory rule.

The frontier model under test produced a complete compliance obligations memo that classified every one of the eight effective practices as either a supervisory expectation in its own right or as overlapping with mandatory national rules, with a threshold classification asserting that d226 carries "a strong gravitational pull into (B) SUPERVISORY EXPECTATION." The document's own stated purpose paragraph, by contrast, records that d226 sets out "examples of how standards set out in the CPMI-IOSCO Principles for financial market infrastructures, as supplemented by the relevant guidance, can be met."

For Hedge fund Compliance, the operational consequence is direct. Horizon-scanning entries and compliance committee briefings that characterise d226 as a binding regulator obligation trigger remediation workstreams sized against a voluntary publication, distort the compliance function's prioritisation of regulatory-change resource, and create inconsistencies between the fund manager's d226 position and the binding national-rule position that supervisors will actually examine. The pattern is also reproducible: it surfaces wherever a deliverable asks the model to commit to a legal characterisation of an international standard-setter publication, and it is not addressed by general-purpose prompting.

The RLB Specialist Panel records the finding under the misstated-rule failure category and binds it to verbatim regulator text drawn from the d226 final report held as primary substrate.

The full finding is recorded under Citation ID RLB-H-INT-BIS-CPMI-CPMI-IOSCO-VARIATION-MARGIN-CCPs-2025-Q004-Opus47. The regulation hub is at /regulators/j1/INT/BIS-CPMI-INT-001/CPMI-IOSCO-VARIATION-MARGIN-CCPs-2025/. Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows the respective audience uses AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

Sector: Hedge Funds and Dept: Risk INT BIS-CPMI

Hedge Funds Risk teams: documentation and reporting gaps possible from AI reading of CPMI-IOSCO VM Effective Practices 2025

For Hedge Funds Risk teams working with Streamlining Variation Margin in Centrally Cleared Markets — Examples of Effective Practices: Specialist-Panel-verified findings on where AI summaries diverge from the...

Risk teams at hedge funds clearing derivatives through central counterparties via their clearing brokers are increasingly using AI to draft updates to liquidity stress-testing assumptions referencing CPMI-IOSCO d226, prepare investor-due-diligence response packs on VM operational resilience, classify each d226 effective practice in the fund manager's regulatory monitoring log, and validate proposed amendments to client-clearing-agreement collateral terms. Leading AI assistants tested by the RLB Specialist Panel produced confident, citable answers on the binding force of d226 that the document itself directly contradicts.

The RLB Specialist Panel tested whether two frontier AI models could correctly characterise the legal status of d226, asking them to classify each of the eight effective practices set out in the document as either a mandatory requirement with enforcement consequences, a supervisory expectation that regulators will test against, or voluntary guidance with no binding legal force. The exercise targeted what the Panel calls inverted modality: AI commitments that flip the binding force of a source text from voluntary illustration to supervisory or mandatory rule.

The frontier model under test produced a complete compliance obligations memo that classified every one of the eight effective practices as either a supervisory expectation in its own right or as overlapping with mandatory national rules, with a threshold classification asserting that d226 carries "a strong gravitational pull into (B) SUPERVISORY EXPECTATION." The document's own stated purpose paragraph, by contrast, records that d226 sets out "examples of how standards set out in the CPMI-IOSCO Principles for financial market infrastructures, as supplemented by the relevant guidance, can be met."

For Hedge fund Risk, the operational consequence is direct. Stress-testing assumptions, investor due-diligence packs, and clearing-broker negotiation notes that characterise d226 as a binding regulator obligation distort the fund's liquidity-buffer sizing, inflate the framing of regulatory risk in investor communications, and weaken the fund's negotiating position on collateral terms with its clearing brokers. The pattern is also reproducible: it surfaces wherever a deliverable asks the model to commit to a legal characterisation of an international standard-setter publication, and it is not addressed by general-purpose prompting.

The RLB Specialist Panel records the finding under the misstated-rule failure category and binds it to verbatim regulator text drawn from the d226 final report held as primary substrate.

The full finding is recorded under Citation ID RLB-H-INT-BIS-CPMI-CPMI-IOSCO-VARIATION-MARGIN-CCPs-2025-Q004-Opus47. The regulation hub is at /regulators/j1/INT/BIS-CPMI-INT-001/CPMI-IOSCO-VARIATION-MARGIN-CCPs-2025/. Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows the respective audience uses AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

↑ Back to top