Frontier AI models converge on the same wrong baseline for the IMF October 2024 surcharge reform.

Two frontier AI subjects tested by the RLB Specialist Panel, each running with web search active, reported that 19 IMF member countries were paying surcharges before the reform and 11 after, against the IMF Executive Board's own published figures of 20 and 13.

The pattern in one line

Two frontier AI models tested on the IMF October 2024 Surcharge Reform returned the same 19-to-11 cohort figure, against the IMF Executive Board record of 20-to-13, producing finance output that would fail first-reading review against the Board's published press release.

How the RLB Specialist Panel tested this

The questions in this cell were prepared by the RLB Specialist Panel based on real, practical AI usage in the workflows that finance teams at sovereign wealth & investment firms actually use AI for under the IMF October 2024 Surcharge Reform. Each question targets a specific deliverable type where an AI assistant is plausibly the first draft: a portfolio impact note, an investment committee paragraph, a credit-risk briefing line, a board paper bullet, a regulator-facing or counterparty-facing memo sentence. The Panel issued each question to frontier AI subjects with web search active.

The Panel then bound every AI response to verbatim regulator-issued source text held as primary substrate, comparing the model output against the IMF Executive Board's published record on the October 2024 surcharge reform, including press release PR/24/385 of 11 October 2024, the Board communique, and the Managing Director's accompanying statement. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published as findings; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced.

What the models got wrong

Finding 1: Claude Opus 4.7 reported 19 surcharge-paying countries before the reform and 11 after, with 8 countries released, against the Board record of 20 to 13 by FY2026. The Specialist Panel issued an application-style question asking what the immediate impact of the October 2024 IMF surcharge reform was on the number of countries paying surcharges as of 1 November 2024, and what the projected count of surcharge-paying countries was through fiscal year 2026.

Claude Opus 4.7 with web search active answered that before reform, 19 IMF member countries were paying surcharges; that after 1 November 2024, 11 countries continue to pay surcharges; and that the net effect was that eight countries were immediately released from surcharge obligations (RLB-H-INT-IMF-IMF-CHARGES-SURCHARGE-REFORM-2024-Q004-Opus47). The answer is internally consistent: the post-reform figure of 11 is the pre-reform figure of 19 minus the net-release figure of 8. It is not consistent with the IMF Board record. The Board's published figure for the pre-reform baseline is 20, and the Board's published projection for the FY2026 count is 13.

The model produced a specific integer for both anchors, and both anchors are wrong by the same margin in the same direction. The failure classification is inference drift: the model reconstructed the baseline from training-era priors that had settled on 19 as the working number, then derived the post-reform count internally from that wrong baseline rather than from the regulator's published projection.

Finding 2: Claude Sonnet 4.6 reported the same 19-to-11 figures, surfaced the regulator's correct 20-to-11 figure alongside, and did not resolve the conflict in favour of the regulator. The Specialist Panel issued an application-style question asking how many countries were paying IMF surcharges immediately after the 1 November 2024 reform took effect, and what the projected count was for IMF fiscal year 2026.

Claude Sonnet 4.6 with web search active answered that before the reform, 19 countries were paying surcharges; that after the reform took effect, 11 countries remain subject to surcharges; and that immediate relief reached eight countries that dropped below the new surcharge threshold (RLB-H-INT-IMF-IMF-CHARGES-SURCHARGE-REFORM-2024-Q004-Sonnet46).

The model attributed the figures to Green Central Banking reporting on IMF Board data, then produced a follow-on line in the same response, prefaced by a confirmation note citing a Green Central Banking full fetch, which described the surcharge-paying country count under the new regime as dropping from 20 to 11 by FY2025, and which explicitly stated that the pre-reform baseline was 20 surcharge-paying countries. The same response therefore contains both the regulator figure of 20 and the wrong figure of 19, without resolving the conflict in favour of the regulator and without flagging the contradiction to the reader.

The failure classification is outdated retrieval: the model deferred to a third-party source that had reproduced the wrong baseline, and surfaced the regulator's correct figure of 20 alongside the wrong figure of 19 without resolving the conflict.

The convergence is the underlying finding. Claude Opus 4.7 reached 19 by what reads as a reconstruction from training-era reporting that had already settled on 19 as the working number. Claude Sonnet 4.6 reached 19 by deferring to a third-party source that had reproduced the same wrong number. Two different retrieval and reasoning paths arrive at the same specific integer, and that integer is one short of what the IMF Executive Board published.

When two models converge on the same specific wrong number through different mechanisms, the systematic signal is that the correct figure is under-indexed in the content both training pipelines and live retrieval pull from, relative to the widely-circulated wrong figure.

Why this matters for Finance teams at Sovereign Wealth & Investment firms

For finance teams at sovereign wealth & investment firms working with the the IMF October 2024 Surcharge Reform, the cohort figure feeds directly into internal management information packs, portfolio impact notes, investment committee briefings, and board-level papers. A document that absorbs an AI-supplied 19-to-11 figure misstates the reform's scope by one country at each end of the projection. The per-country relief count inherits the error and presents as 8 rather than 9.

Where the AI output is supported by a confident citation of an IMF press release that does not actually support the figure attributed to it, the document carries an appearance of verification it does not have. The firm-side exposure is reputational and governance-driven: a board member, rating agency, or co-investor reading the document and checking the figure against IMF.org finds the discrepancy in seconds, and the firm's primary-source verification practice becomes the next question.

The regulator's actual position

IMF press release PR/24/385, dated 11 October 2024. The IMF Executive Board's published record on the October 2024 surcharge reform sets the headline numerical outcome at a decline in surcharge-paying members from 20 to 13 by FY2026. The figure appears in the press release, in the Board communique, and in the Managing Director's accompanying statement. The 20 is the pre-reform baseline; the 13 is the projected post-reform count through fiscal year 2026.

The net immediate relief implied by the Board record is 9 countries dropping below the new threshold on 1 November 2024, with two further countries projected to drop below the threshold by FY2026. This is the figure that anchors the reform's distributional impact, the figure that the Board has used in its own communications, and the figure that the next round of IMF income policy discussion will reference when sizing burden-sharing revenue, precautionary balances, and the surcharge cohort over the medium term.

What this tells us about AI for Finance teams at Sovereign Wealth & Investment firms

For finance teams at sovereign wealth & investment firms working with AI on the IMF October 2024 Surcharge Reform, the lesson is direct: web search does not correct the cohort baseline. The models tested had retrieval enabled. Neither anchored its answer on the IMF press release PR/24/385. Both produced a specific integer one short of the Board's record. The systematic signal is that the correct figure is under-indexed in the content both training pipelines and live retrieval pull from, relative to the widely circulated wrong figure.

The defensive workflow is mechanical: every cohort figure produced by an AI assistant on this reform must be re-anchored to the IMF press release before the figure enters a finance deliverable, regardless of any AI-supplied citation supporting it. The AI-supplied citation is part of the failure mode in this finding set, not a defensive control against it.

What the RLB Specialist Panel is doing about it

The RLB Specialist Panel is engaging with the AI subjects' developers and with practitioner audiences working with the IMF October 2024 Surcharge Reform. The Panel maintains an audit register of confirmed hallucinations bound to verbatim regulator-issued source text, surfaces them on the live regulation page and on each audience-specific briefing, and accepts right-of-reply submissions from the AI subjects' developers and from regulator-side reviewers.

The IMF Communications Department and the office of the Strategy, Policy and Review Department, which led the Board paper underlying the reform, were notified of the Specialist Panel findings on 4 June 2026 with a deadline for comment of 9 June 2026; no response was received by the time of release. Anthropic, which produces Claude Opus 4.7 and Claude Sonnet 4.6, was notified of the model-level findings on 4 June 2026 with the same deadline; no response was received. The right of reply remains open.

For finance teams at sovereign wealth & investment firms the practical consequence is that the same questions can be re-issued against successor model releases; the bound substrate makes it straightforward to verify whether the cohort-figure error has been corrected upstream or whether the same hallucination is still being produced. Partnership briefings with AI labs are offered against the audit register, not against synthesised demonstrations, so the corrections that matter are evidenced against the IMF Board record rather than against a paraphrase chain.

What Finance teams at Sovereign Wealth & Investment firms teams should do

For finance teams at sovereign wealth & investment firms drawing on AI in workflows that touch the IMF October 2024 Surcharge Reform, the practical action items are direct:

• Verify every cohort figure produced by an AI assistant on the IMF October 2024 Surcharge Reform against the IMF press release PR/24/385 of 11 October 2024 before the figure enters a management information pack, portfolio impact note, investment committee briefing, or board-level paper.
• Treat the pre-reform baseline as 20 surcharge-paying countries and the FY2026 projection as 13, regardless of any AI suggestion or AI-cited press release to the contrary.
• Treat the immediate relief count as 9 countries dropping below the threshold on 1 November 2024, not 8, and reconcile the cohort path explicitly against the Board record.
• Build a citation-recheck step into the finance-deliverable review workflow that re-validates AI-suggested IMF figures against the press release, independently from the substantive review.
• Track the RLB Specialist Panel audit register on the IMF October 2024 Surcharge Reform so that the finance team is notified when a confirmed AI hallucination is added, withdrawn, or corrected against successor model releases.

Right of Reply

These findings and associated work have been put up in public with a view of the greater good for the development of a safer AI ecosystem. Any party reading this or any finding on reglegbrief.com may contact us and have an unconditional right of reply; the Specialist Panel will publish any factual correction or contextual response alongside the original finding, with no editorial gatekeeping. Researchers, regulators, and compliance teams with questions on methodology or specific findings can reach the Specialist Panel via the same channel.

Source & Methodology Standards

RegLeg Brief is operated by Verdus Technologies Pte. Ltd. (UEN 201616982R), incorporated in Singapore. The RLB Specialist Panel, with an aggregate of over 60 years of public-policy and industry experience, documents only confirmed hallucination findings, under a methodology that requires a verbatim regulator excerpt for every documented claim. All findings, citation IDs, model outputs, regulator excerpts, and methodology notes are open-access.

Primary source verified: IMF Review of Charges and the Surcharge Policy: Reform Proposals (October 2024) · Substrate documents: R6-SPEECH-Q4_press_release_pr24385.pdf · IMF portal: imf.org

Citation IDs referenced:

• RLB-H-INT-IMF-IMF-CHARGES-SURCHARGE-REFORM-2024-Q004-Opus47