Briefings Blog

SINGAPORE, June 10, 2026. Two frontier AI models running with web search enabled, both tested by the RLB Specialist Panel, produced confidently wrong reconstructions of the headline baseline figure in the International Monetary Fund's October 2024 surcharge reform, in findings released today by the RegLeg Brief Specialist Panel. Asked how many IMF member countries were paying surcharges immediately before the reform took effect on 1 November 2024, both models committed to a specific integer that diverges from the Fund's own published count, and arrived at the same wrong number through different failure paths.

Claude Opus 4.7, queried on the immediate impact of the reform and the projected count through fiscal year 2026, answered that "before reform: 19 IMF member countries were paying surcharges" and that "after 1 November 2024: 11 countries continue to pay surcharges," with a net of eight countries released. The IMF Executive Board's published record, in press release PR/24/385 dated 11 October 2024, states that the number of surcharge payers is expected to decline from 20 to 13 countries in FY2026.

The model dropped one country from the pre-reform baseline and reconstructed the post-reform count from a net-release figure rather than from the regulator's published projection.

Claude Sonnet 4.6, on the same question, answered that "before the reform: 19 countries were paying surcharges" and that "after the reform took effect: 11 countries remain subject to surcharges." Sonnet 4.6 attributed the figures to Green Central Banking reporting on IMF Board data, then offered a separate "pre-reform baseline was 20 surcharge-paying countries" line in the same response, surfacing both the regulator figure and the wrong figure without resolving the conflict in favour of the regulator.

A sovereign debt economist, IMF country team, finance ministry desk officer, or research analyst drafting a surcharge impact note against either output would publish a pre-reform baseline one country short of the Fund's own count and a post-reform projection short of the FY2026 figure the Board approved.

Two frontier AI models running with web search enabled, both tested by the RLB Specialist Panel, produced confidently wrong reconstructions of the CPMI Harmonised ISO 20022 Data Requirements for Enhancing Cross-Border Payments, the Updated Report that anchors the messaging architecture for the G20 cross-border payments roadmap and binds correspondent banks, payment scheme operators, and real-time gross settlement systems to a common data model.

The RegLeg Brief Specialist Panel tested both models on the regulator's adoption metrics, on Fedwire's hybrid/end-state postal address format, on the CPMI working-group chair attribution, and on the operational statistics published in BIS-channel speeches, and documents findings in which the models blended distinct subcategory adoption percentages into a single composite figure, over-specified the mandatory tier of a published technical schema, misattributed the working-group chair to a higher-frequency central bank, and evaded a precisely-stated operational statistic by returning a false negative.

Claude Opus 4.7, asked what share of faster payment systems and RTGS systems currently use ISO 20022 messaging, wrote that "approximately 79% of both real-time gross settlement (RTGS) systems and fast payment systems (FPS) had either already implemented ISO 20022 or had concrete plans to do so." The regulator's record, drawn from Bank of England Governor Andrew Bailey's 12 March 2026 speech, reads: "more than three-quarters of faster payment systems and approaching half of RTGS systems now use ISO 20022." The model collapsed two distinct figures into one symmetric percentage that matches neither, applied to both system types simultaneously.

Asked separately about Fedwire's postal address format under the hybrid/end-state approach, Opus 4.7 elevated Building Number, Post Code, and Country Sub-Division into a structured mandatory tier; the implementing body's published FAQ places those elements in the optional tier and prescribes country code plus town name plus optional free-format lines of 70 characters as the binding format.

Claude Sonnet 4.6 reproduced the same 79% conflation on the adoption-rate question and added two further failures. Asked which central bank chairs the relevant CPMI working group, Sonnet 4.6 named the Federal Reserve Bank of New York; the working-group co-chair role belongs to the Reserve Bank of Australia.

Asked for the official statistics on payment inquiry rates and manual touchpoints under the existing cross-border architecture, the model returned a false negative, claiming no specific figure existed; the regulator's March 2026 speech gives the precise figures of 1 to 3 per cent of payments generating inquiries requiring 5 to 10 manual touchpoints, with resolution times reducible by up to 80 per cent through harmonised ISO 20022 implementation.

A correspondent-bank compliance officer, payment-scheme operator, fintech integrator, or regtech tool advising on cross-border implementation timelines and relying on either output would misadvise a client on implementation readiness, pursue the wrong central-bank counterparty on standards governance, implement a more restrictive Fedwire address schema than the regulator requires, and miss a quantitative baseline the regulator itself published. That is the failure mode these findings document.

Two frontier AI models running with web search enabled, both tested by the RLB Specialist Panel, produced confidently wrong reconstructions of the CPMI-IOSCO Level 3 Assessment Report on Authorities' Implementation of the PFMI Standards for Financial Market Infrastructures regarding General Business Risk, published by the Bank for International Settlements and IOSCO in November 2025 as BIS CPMI Papers No. 228 / IOSCOPD807.

The RegLeg Brief Specialist Panel tested both models on the assessment's text and on PFMI Principle 15, and documents findings in which the models invented a quantitative six-months-of-operating-expenses floor for Principle 15 key consideration 3 the standard does not state, fabricated named co-chairs and team co-leads for the Implementation Monitoring Standing Group, and compressed the 2023 to 2025 assessment window into "2023 and 2024" while attributing the answer to the published report.

Claude Opus 4.7, asked what the current PFMI Principle 15 minimum standard is for liquid net assets funded by equity, wrote that the floor is "the greater of the resources required to execute the firm's recovery or orderly wind-down plan, and six months of current operating expenses". The assessment report's own reproduction of Principle 15 states the minimum as the liquid net assets needed to implement the firm's recovery or orderly wind-down plan, and separately references the further CPMI-IOSCO guidance on recovery planning issued since 2014; the report does not state a six-months-of-operating-expenses figure as the binding KC3 floor.

The model converted a recovery-plan-sized obligation into a numerically anchored floor that reads as authoritative but does not appear in the source.

Asked who co-chaired the IMSG running the Level 3 exercise, Opus 4.7 declined to name individuals and directed the reader to the report's inside cover. Sonnet 4.6, in the parallel finding, asserted that the IMSG was co-chaired by the US Securities and Exchange Commission's Elizabeth L Fitzgerald and the European Central Bank's Fiona van Echelpoel, with team co-leads Corinna Freund of the ECB and Vishal Shukla of the Securities and Exchange Board of India. None of the four named individuals appears in the published report in those roles; the names, affiliations and roles are the model's construction.

Asked when the assessment was conducted, Sonnet 4.6 wrote that "the assessment work was carried out during 2023 and 2024". The published report states the work was carried out during 2023 to 2025 by the IMSG and a team of experts from CPMI and IOSCO member jurisdictions.

A CCP capital management team, central-bank supervisor, or trade-repository compliance lead drafting a Principle 15 sufficiency policy, a board paper, or a benchmarking note against either output would record a six-months floor the PFMI standard itself does not anchor, would cite IMSG co-chairs and team co-leads who do not appear in the report, and would mis-state the assessment window. That is the failure mode these findings document.

Two frontier AI models running with web search enabled, both tested by the RLB Specialist Panel, produced confidently wrong reconstructions of the CPMI-IOSCO Guidance on Cyber Resilience for Financial Market Infrastructures (June 2016), the global standard for cyber resilience at systemically important payment systems, central counterparties, and securities settlement infrastructures.

The RegLeg Brief Specialist Panel tested the models on the guidance's content, its relationship to post-2016 standards (FSB Cyber Lexicon 2018, FSB Effective Practices 2020), and its current operative status, and documents findings in which the models fabricated an explicit citation to the NIST Cybersecurity Framework, imported 2020-era operational detail into the 2016 text, and asserted the document remained the unchanged operative standard when it had moved into active revision in May 2026.

Claude Opus 4.7, asked whether the 2016 guidance explicitly cites the NIST Cybersecurity Framework, wrote that the document "acknowledges and considers prevailing industry frameworks, including the NIST CSF, ISO/IEC 27001/27002, COBIT, and the ISF Standard of Good Practice." No verbatim NIST CSF citation appears in the 2016 guidance. The guidance's five categories (Governance, Identification, Protection, Detection, Response and Recovery) are structurally similar to the NIST CSF's five functions, but architectural resemblance is not the same as an explicit textual reference, and the model converted the resemblance into a confident affirmative citation claim.

Claude Sonnet 4.6, asked whether the 2016 guidance itself specifies detailed operational practices for cyber incident response, wrote that it "dedicates specific sections to cyber incident response and recovery" and described detailed expectations including secondary-site use, recovery and resumption planning, and incident communication protocols. The operational specificity the model described is characteristic of the FSB's Effective Practices for Cyber Incident Response and Recovery, published in October 2020, four years after the guidance. Both models, asked separately about the document's current status, asserted the 2016 guidance remained the operative international standard, despite CPMI-IOSCO having published a consultative revision in May 2026.

An FMI cyber-resilience officer, supervisor, or compliance lead relying on either output would draft policy frameworks, supervisory submissions, or board disclosures that misrepresent what the 2016 guidance actually contains and overstate its current standing. That is the failure mode these findings document.

Two frontier AI models running with web search enabled, both tested by the RLB Specialist Panel, produced confidently wrong reconstructions of the CPMI-IOSCO Consultative Report on Updated Guidance and Public Disclosures to Implement Initial Margin Proposals, the May 2026 consultative document (d232) that codifies how central counterparties must disclose initial margin responsiveness, override frameworks, and simulation tools.

The RegLeg Brief Specialist Panel tested both models on the consultation's text and on the January 2025 BCBS-CPMI-IOSCO report it implements, and documents findings in which the models softened mandatory CCP obligations into discretionary "should consider" language, invented a three-category taxonomy the source report does not contain, and fabricated a three-element public disclosure structure for override frameworks the guidance does not enumerate.

Claude Opus 4.7, asked what obligation the final CPMI-IOSCO guidance places on CCPs to provide margin simulation tools, wrote that CCPs "should consider" making the tools available to clearing members and, where feasible, their clients. The 2024 consultative text from which the obligation derives reads: "Margin simulation tools with certain minimum functionality should be made available by CCPs to clearing members and their clients." The model converted a positive obligation into a discretionary consideration, the difference between a CCP being expected to provide a tool and a CCP being expected to think about providing one.

Asked about the structure of the January 2025 BCBS-CPMI-IOSCO report, Opus 4.7 also asserted the report's ten policy proposals fall into "three broad categories" of CCP transparency, governance and clearing-member transparency. The report's published text describes ten proposals aimed at resilience of the centrally cleared market ecosystem; no three-category taxonomy is stated in the source.

Claude Sonnet 4.6 reproduced the same obligation-softening on margin simulation tools and added a fabricated disclosure structure for the override framework. Asked what CCPs must publicly disclose about their override framework, Sonnet 4.6 enumerated three elements: instances or circumstances where overrides may be warranted, the key decision-makers authorised to exercise override discretion, and the permissible types of adjustments. The consultative text says only: "CCPs should publicly disclose relevant information on their override framework." The three-element list is the model's construction, not the regulator's.

A CCP risk officer, clearing-member compliance lead, or supervisor drafting a comment letter, board paper, or implementation plan against either output would understate CCP obligations on simulation tools, structure their override-framework disclosure against a taxonomy that does not exist in the source, and cite a category framework for the underlying policy proposals that the BIS press release does not endorse. That is the failure mode these findings document.