AI Labs · Last updated 7 Jun 2026 · methodology vv2.3 · Hallucination Register

Finding on 'Q010 Probe' for Claude Opus 4.7 with web search ON

RLB Citation ID: RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010-Opus47

What the RLB Specialist Panel found

Finding on 'Q010 Probe' for Claude Opus 4.7 with web search ON

Question (paraphrased to protect IP): A market briefing on the global fast payment system landscape needs CPMI data on how many domestic fast payment systems are currently operational globally, how many have already enabled cross-border payment exchanges, how many are planning cross-border linkages, and what proportion are operated by central banks versus private entities.
AI's response: "the 2025 monitoring survey covers 57 fast payment systems (56 for one of the graph cuts), up from prior surveys; this is the most recent CPMI count"
Regulator's text: Speech sp231115 (Nov 2023, Tara Rice): '70+ domestic fast payment systems currently operational globally'; '14 fast payment systems already enabling cross-border exchanges'; '24 systems planning linkages within five years'; '40% operated by central banks; 35% by private entities'
Why the AI went wrong: The model retrieved a 2025 monitoring-survey figure (57 systems) and presented it as the answer to a question about the global FPS universe. The November 2023 Tara Rice speech documents the 70+ universe figure, but the model defaulted to the more recently retrieved sample figure without distinguishing between universe and sample. The user-facing response gives no signal that a sample-versus-universe substitution has occurred.
Cited source(s):

Impact for this audience

The model substituted a survey-sample count (57 systems from the 2025 CPMI monitoring survey) for the regulator's stated universe figure (70+ from the November 2023 Tara Rice speech). The error is statistical-substrate confusion: the model has retrieved a sample from one CPMI publication and presented it as the universe figure that a different CPMI publication actually states.

The implication for the retrieval-and-generation pipeline is that when multiple CPMI sources publish related-but-different numbers (universe versus sample, current-state versus monitoring-snapshot), the model is not disambiguating between them; it surfaces the most recently retrieved figure as if it were the answer to the question. For an AI lab, this is a high-yield eval probe: regulatory benchmark questions that have universe-versus-sample distinctions in the primary source corpus should surface this confusion pattern reliably.

References — raw findings (per AI model)

RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010-Opus47 Claude Opus 4.7 (web search on)
RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010-Sonnet46 Claude Sonnet 4.6 (web search on)

This finding also affects

← Previous finding Finding on 'Q009 Probe' for Claude Sonnet 4.6 with web search ON Next finding → Finding on 'Q010 Probe' for Claude Sonnet 4.6 with web search ON

Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010-Opus47

Plain text Download

RegLeg Specialist Panel (2026). "Finding on 'Q010 Probe' for Claude Opus 4.7 with web search ON — AI Labs." Citation ID: RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010-Opus47. RegLegBrief AI Hallucination Research, published 2026-06-07. https://reglegbrief.com/regulators/j1/INT/BIS-CPMI/CPMI-API-HARMONISATION-CROSS-BORDER-2024/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-010--opus-47-websearch/

APA 7th edition Download

RegLeg Specialist Panel. (2026). Finding on 'Q010 Probe' for Claude Opus 4.7 with web search ON [Hallucination finding RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010-Opus47]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j1/INT/BIS-CPMI/CPMI-API-HARMONISATION-CROSS-BORDER-2024/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-010--opus-47-websearch/

Bluebook / OSCOLA (US + UK legal) Download

RegLeg Specialist Panel, Finding on 'Q010 Probe' for Claude Opus 4.7 with web search ON [RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010-Opus47], RegLegBrief AI Hallucination Research (June 07, 2026), https://reglegbrief.com/regulators/j1/INT/BIS-CPMI/CPMI-API-HARMONISATION-CROSS-BORDER-2024/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-010--opus-47-websearch/.

BibTeX Download

@misc{reglegbrief_RLB_H_INT_BIS_CPMI_API_HARMONISATION_CROSS_BORDER_2024_Q010_Opus47,
  author    = {RegLeg Specialist Panel},
  title     = {Finding on 'Q010 Probe' for Claude Opus 4.7 with web search ON},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010-Opus47},
  url       = {https://reglegbrief.com/regulators/j1/INT/BIS-CPMI/CPMI-API-HARMONISATION-CROSS-BORDER-2024/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-010--opus-47-websearch/}
}

← Back to case study summary Case study detail →

About Citation IDs →

Every finding on this page compares an AI subject's account of the rule against the regulator's verbatim text from the regulator's own portal. Both are linked. Each delta, its root causes, and impact analysis are documented and published with immutable Citation IDs.