AI Labs · updated 2026-05-26 · methodology v2.1

Finding on 'Q020 Probe' for Claude Sonnet 4.6 with web search ON

RLB Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46

What the RLB Specialist Panel found

Finding on 'Q020 Probe' for Claude Sonnet 4.6 with web search ON

Question (paraphrased to protect IP): How does the CPMI-IOSCO 2016 Cyber Guidance define 'cyber resilience', and is that definition aligned with the 2018 FSB Cyber Lexicon?
AI's response: > The CPMI-IOSCO 2016 Cyber Guidance defines cyber resilience as the ability to anticipate, absorb, adapt to, rapidly respond to, and recover from disruption caused by a cyber attack... the FSB Lexicon... explicitly drew on the CPMI-IOSCO definition.
Regulator's text: The FSB Cyber Lexicon was published in November 2018 — two years after the 2016 guidance. Its standardised definitions postdate the 2016 document and may not correspond to how the 2016 text used the same terms.
Why the AI went wrong: The model not only compared the two definitions but asserted a specific causal relationship — that the FSB Lexicon explicitly drew on the CPMI-IOSCO definition — for which no basis was found. This converts a plausible inference (that a 2018 lexicon would be informed by a prominent 2016 document from the same regulatory community) into a stated fact. The model also presented the 2016 definition in confident detail without flagging that the Lexicon postdates it and the relationship between the two definitions remains unconfirmed.
Regulator portal (if any cited link is dud): https://www.bis.org

Impact for this audience

This finding reveals that the model not only collapsed a temporal gap but asserted a specific causal relationship (that the FSB Lexicon drew on the CPMI-IOSCO definition) for which no evidential basis was found. This is a more advanced failure than simple conflation: the model constructed a plausible-sounding provenance claim that goes beyond what the documents support. This class of error — inferred causation stated as documented fact — is particularly hazardous in legal and compliance contexts and is likely to evade generic hallucination red-teaming that focuses on factual accuracy rather than provenance accuracy.

References — raw findings (per AI model)

RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47 Claude Opus 4.7 (web search on)
RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46 Claude Sonnet 4.6 (web search on)

This finding also affects

← Previous finding Finding on 'Q019 Probe' for Claude Sonnet 4.6 with web search ON Next finding → Finding on 'Q022 Probe' for Claude Sonnet 4.6 with web search ON

Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46

Plain text Download

RegLeg Specialist Panel (2026). "Finding on 'Q020 Probe' for Claude Sonnet 4.6 with web search ON — AI Labs." Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46. RegLegBrief AI Hallucination Research, published 2026-05-26. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-cyber-resilience-fmi-2016/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-v1-020--sonnet-46-websearch/

APA 7th edition Download

RegLeg Specialist Panel. (2026). Finding on 'Q020 Probe' for Claude Sonnet 4.6 with web search ON [Hallucination finding RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-cyber-resilience-fmi-2016/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-v1-020--sonnet-46-websearch/

Bluebook / OSCOLA (US + UK legal) Download

RegLeg Specialist Panel, Finding on 'Q020 Probe' for Claude Sonnet 4.6 with web search ON [RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46], RegLegBrief AI Hallucination Research (May 26, 2026), https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-cyber-resilience-fmi-2016/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-v1-020--sonnet-46-websearch/.

BibTeX Download

@misc{reglegbrief_RLB_H_INT_BIS_CPMI_IOSCO_CYBER_RESILIENCE_FMI_2016_Q020_Sonnet46,
  author    = {RegLeg Specialist Panel},
  title     = {Finding on 'Q020 Probe' for Claude Sonnet 4.6 with web search ON},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46},
  url       = {https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-cyber-resilience-fmi-2016/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-v1-020--sonnet-46-websearch/}
}

← Back to case study summary Case study detail →

About Citation IDs →

Every finding on this page compares an AI subject's account of the rule against the regulator's verbatim text from the regulator's own portal. Both are linked. Each delta, its root causes, and impact analysis are documented and published with immutable Citation IDs.