AI Labs · updated 2026-06-06 · methodology v2.1

Finding on 'Q004 Probe' for Claude Sonnet 4.6 with web search ON

RLB Citation ID: RLB-H-INT-IMF-IMF-CHARGES-SURCHARGE-REFORM-2024-Q004-Sonnet46

What the RLB Specialist Panel found

Finding on 'Q004 Probe' for Claude Sonnet 4.6 with web search ON

Question (paraphrased to protect IP): How many countries are paying IMF surcharges immediately after the November 1, 2024 reform takes effect, and what is the projected count for IMF fiscal year 2026?
AI's response: "Before the reform: 19 countries were paying surcharges... After the reform took effect: 11 countries remain subject to surcharges... Immediate relief: 8 countries dropped below the new surcharge threshold"
Regulator's text: Surcharge-paying countries under new regime: drop from 20 to 11 by FY2025. The pre-reform baseline was 20 surcharge-paying countries.
Why the AI went wrong: The model retrieved external content and deferred to it, but the retrieved source had already introduced the wrong baseline (19 rather than 20). The model treated this secondary account as authoritative without checking it against the IMF's primary text. The post-reform figure (11) and the net reduction (8 countries) are arithmetically consistent with the wrong baseline — which means the internal logic holds together while the foundational figure is off, making the error harder for the reader to detect.
Cited source(s):

Impact for this audience

This error implicates the retrieval-authority ranking in the web-search stack: the model deferred to a third-party source that had already introduced the wrong baseline (19 rather than 20), without cross-checking against the IMF's primary document. The downstream arithmetic — 11 remaining, 8 relieved — is internally consistent with the wrong baseline, meaning the response passed its own coherence check while the foundational figure was off. The retrieval ranker is treating third-party regulatory commentary as co-equal in authority to the regulator's primary text for numeric threshold queries, which is the proximate cause of this class of failure.

References — raw findings (per AI model)

RLB-H-INT-IMF-IMF-CHARGES-SURCHARGE-REFORM-2024-Q004-Opus47 Claude Opus 4.7 (web search on)
RLB-H-INT-IMF-IMF-CHARGES-SURCHARGE-REFORM-2024-Q004-Sonnet46 Claude Sonnet 4.6 (web search on)

This finding also affects

← Previous finding Finding on 'Q004 Probe' for Claude Opus 4.7 with web search ON

Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-H-INT-IMF-IMF-CHARGES-SURCHARGE-REFORM-2024-Q004-Sonnet46

Plain text Download

RegLeg Specialist Panel (2026). "Finding on 'Q004 Probe' for Claude Sonnet 4.6 with web search ON — AI Labs." Citation ID: RLB-H-INT-IMF-IMF-CHARGES-SURCHARGE-REFORM-2024-Q004-Sonnet46. RegLegBrief AI Hallucination Research, published 2026-06-06. https://reglegbrief.com/regulators/j1/int/imf/imf-charges-surcharge-reform-2024/whitepaper/finding/INT-IMF-INT-001-IMF-CHARGES-SURCHARGE-REFORM-2024-v1-004--sonnet-46-websearch/

APA 7th edition Download

RegLeg Specialist Panel. (2026). Finding on 'Q004 Probe' for Claude Sonnet 4.6 with web search ON [Hallucination finding RLB-H-INT-IMF-IMF-CHARGES-SURCHARGE-REFORM-2024-Q004-Sonnet46]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j1/int/imf/imf-charges-surcharge-reform-2024/whitepaper/finding/INT-IMF-INT-001-IMF-CHARGES-SURCHARGE-REFORM-2024-v1-004--sonnet-46-websearch/

Bluebook / OSCOLA (US + UK legal) Download

RegLeg Specialist Panel, Finding on 'Q004 Probe' for Claude Sonnet 4.6 with web search ON [RLB-H-INT-IMF-IMF-CHARGES-SURCHARGE-REFORM-2024-Q004-Sonnet46], RegLegBrief AI Hallucination Research (June 06, 2026), https://reglegbrief.com/regulators/j1/int/imf/imf-charges-surcharge-reform-2024/whitepaper/finding/INT-IMF-INT-001-IMF-CHARGES-SURCHARGE-REFORM-2024-v1-004--sonnet-46-websearch/.

BibTeX Download

@misc{reglegbrief_RLB_H_INT_IMF_IMF_CHARGES_SURCHARGE_REFORM_2024_Q004_Sonnet46,
  author    = {RegLeg Specialist Panel},
  title     = {Finding on 'Q004 Probe' for Claude Sonnet 4.6 with web search ON},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-H-INT-IMF-IMF-CHARGES-SURCHARGE-REFORM-2024-Q004-Sonnet46},
  url       = {https://reglegbrief.com/regulators/j1/int/imf/imf-charges-surcharge-reform-2024/whitepaper/finding/INT-IMF-INT-001-IMF-CHARGES-SURCHARGE-REFORM-2024-v1-004--sonnet-46-websearch/}
}

← Back to case study summary Case study detail →

About Citation IDs →

Every finding on this page compares an AI subject's account of the rule against the regulator's verbatim text from the regulator's own portal. Both are linked. Each delta, its root causes, and impact analysis are documented and published with immutable Citation IDs.