A frontier AI model misreports the IMF October 2024 surcharge reform's headline cohort figures.

A frontier AI subject tested by the RLB Specialist Panel, running with web search active, reported a pre-reform cohort of 19 surcharge-paying countries and a post-reform cohort of 11, against the IMF Executive Board's published figures of 20 and 13.

The pattern in one line

A frontier AI model tested on the IMF October 2024 Surcharge Reform returned a 19-to-11 cohort figure, against the IMF Executive Board record of 20-to-13, producing risk output that would fail first-reading review against the Board's published press release.

How the RLB Specialist Panel tested this

The questions in this cell were prepared by the RLB Specialist Panel based on real, practical AI usage in the workflows that risk teams at investment banking firms actually use AI for under the IMF October 2024 Surcharge Reform. Each question targets a specific deliverable type where an AI assistant is plausibly the first draft: a portfolio impact note, an investment committee paragraph, a credit-risk briefing line, a board paper bullet, a regulator-facing or counterparty-facing memo sentence. The Panel issued each question to frontier AI subjects with web search active.

The Panel then bound every AI response to verbatim regulator-issued source text held as primary substrate, comparing the model output against the IMF Executive Board's published record on the October 2024 surcharge reform, including press release PR/24/385 of 11 October 2024, the Board communique, and the Managing Director's accompanying statement. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published as findings; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced.

What the models got wrong

Finding 1: Claude Opus 4.7 reported 19 surcharge-paying countries before the reform and 11 after, with 8 countries released, against the Board record of 20 to 13 by FY2026. The Specialist Panel issued an application-style question asking what the immediate impact of the October 2024 IMF surcharge reform was on the number of countries paying surcharges as of 1 November 2024, and what the projected count of surcharge-paying countries was through fiscal year 2026.

Claude Opus 4.7 with web search active answered that before reform, 19 IMF member countries were paying surcharges; that after 1 November 2024, 11 countries continue to pay surcharges; and that the net effect was that eight countries were immediately released from surcharge obligations (RLB-H-INT-IMF-IMF-CHARGES-SURCHARGE-REFORM-2024-Q004-Opus47). The answer is internally consistent: the post-reform figure of 11 is the pre-reform figure of 19 minus the net-release figure of 8. It is not consistent with the IMF Board record. The Board's published figure for the pre-reform baseline is 20, and the Board's published projection for the FY2026 count is 13.

The model produced a specific integer for both anchors, and both anchors are wrong by the same margin in the same direction. The failure classification is inference drift: the model reconstructed the baseline from training-era priors that had settled on 19 as the working number, then derived the post-reform count internally from that wrong baseline rather than from the regulator's published projection.

Why this matters for Risk teams at Investment Banking firms

For risk teams at investment banking firms with sovereign-exposure books, the cohort figure feeds directly into emerging-market sovereign watchlists, country-tier reviews, credit committee packs, and client-facing credit notes. A watchlist update anchored to a 19-country pre-reform cohort mis-classifies one borrower's surcharge status. A credit committee pack that quantifies the reform's distributional impact off a 20-to-13 cohort produces a different relative-value picture from the same pack built on an AI-supplied 19-to-11 cohort.

Where the AI output is supported by a confident citation of an IMF press release that does not actually support the figure attributed to it, the document appears verified when it is not, and the risk team's primary-source verification practice becomes the immediate next question on first external review.

The regulator's actual position

IMF press release PR/24/385, dated 11 October 2024. The IMF Executive Board's published record on the October 2024 surcharge reform sets the headline numerical outcome at a decline in surcharge-paying members from 20 to 13 by FY2026. The figure appears in the press release, in the Board communique, and in the Managing Director's accompanying statement. The 20 is the pre-reform baseline; the 13 is the projected post-reform count through fiscal year 2026.

The net immediate relief implied by the Board record is 9 countries dropping below the new threshold on 1 November 2024, with two further countries projected to drop below the threshold by FY2026. This is the figure that anchors the reform's distributional impact, the figure that the Board has used in its own communications, and the figure that the next round of IMF income policy discussion will reference when sizing burden-sharing revenue, precautionary balances, and the surcharge cohort over the medium term.

What this tells us about AI for Risk teams at Investment Banking firms

For risk teams at investment banking firms working with AI on the IMF October 2024 Surcharge Reform, the lesson is direct: web search does not correct the cohort baseline. The models tested had retrieval enabled. Neither anchored its answer on the IMF press release PR/24/385. Both produced a specific integer one short of the Board's record. The systematic signal is that the correct figure is under-indexed in the content both training pipelines and live retrieval pull from, relative to the widely circulated wrong figure.

The defensive workflow is mechanical: every cohort figure produced by an AI assistant on this reform must be re-anchored to the IMF press release before the figure enters a risk deliverable, regardless of any AI-supplied citation supporting it. The AI-supplied citation is part of the failure mode in this finding set, not a defensive control against it.

What the RLB Specialist Panel is doing about it

The RLB Specialist Panel is engaging with the AI subjects' developers and with practitioner audiences working with the IMF October 2024 Surcharge Reform. The Panel maintains an audit register of confirmed hallucinations bound to verbatim regulator-issued source text, surfaces them on the live regulation page and on each audience-specific briefing, and accepts right-of-reply submissions from the AI subjects' developers and from regulator-side reviewers.

The IMF Communications Department and the office of the Strategy, Policy and Review Department, which led the Board paper underlying the reform, were notified of the Specialist Panel findings on 4 June 2026 with a deadline for comment of 9 June 2026; no response was received by the time of release. Anthropic, which produces Claude Opus 4.7 and Claude Sonnet 4.6, was notified of the model-level findings on 4 June 2026 with the same deadline; no response was received. The right of reply remains open.

For risk teams at investment banking firms the practical consequence is that the same questions can be re-issued against successor model releases; the bound substrate makes it straightforward to verify whether the cohort-figure error has been corrected upstream or whether the same hallucination is still being produced. Partnership briefings with AI labs are offered against the audit register, not against synthesised demonstrations, so the corrections that matter are evidenced against the IMF Board record rather than against a paraphrase chain.

What Risk teams at Investment Banking firms teams should do

For risk teams at investment banking firms drawing on AI in workflows that touch the IMF October 2024 Surcharge Reform, the practical action items are direct:

• Verify every cohort figure produced by an AI assistant on the IMF October 2024 Surcharge Reform against the IMF press release PR/24/385 of 11 October 2024 before the figure enters an emerging-market watchlist update, a country-tier review, a credit committee pack, or a client-facing credit note.
• Treat the pre-reform baseline as 20 surcharge-paying countries and the FY2026 projection as 13, regardless of any AI suggestion or AI-cited press release to the contrary.
• When updating per-borrower surcharge classifications on the watchlist, reconcile each affected sovereign against the Board's published cohort path rather than against an AI-supplied cohort headline.
• Build a citation-recheck step into the risk-deliverable review workflow that re-validates AI-suggested IMF figures against the press release, independently from the substantive review.
• Track the RLB Specialist Panel audit register on the IMF October 2024 Surcharge Reform so that the risk team is notified when a confirmed AI hallucination is added, withdrawn, or corrected against successor model releases.

Right of Reply

These findings and associated work have been put up in public with a view of the greater good for the development of a safer AI ecosystem. Any party reading this or any finding on reglegbrief.com may contact us and have an unconditional right of reply; the Specialist Panel will publish any factual correction or contextual response alongside the original finding, with no editorial gatekeeping. Researchers, regulators, and compliance teams with questions on methodology or specific findings can reach the Specialist Panel via the same channel.

Source & Methodology Standards

RegLeg Brief is operated by Verdus Technologies Pte. Ltd. (UEN 201616982R), incorporated in Singapore. The RLB Specialist Panel, with an aggregate of over 60 years of public-policy and industry experience, documents only confirmed hallucination findings, under a methodology that requires a verbatim regulator excerpt for every documented claim. All findings, citation IDs, model outputs, regulator excerpts, and methodology notes are open-access.

Primary source verified: IMF Review of Charges and the Surcharge Policy: Reform Proposals (October 2024) · Substrate documents: R6-SPEECH-Q4_press_release_pr24385.pdf · IMF portal: imf.org

Citation IDs referenced:

• RLB-H-INT-IMF-IMF-CHARGES-SURCHARGE-REFORM-2024-Q004-Opus47