AI Hallucination Research › Regulators › Global standard-setters › INT › BIS-CPMI › CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016 › White paper › Detail › Finding
AI Labs · updated 2026-05-26 · methodology v2.1

Finding on 'Q020 Probe' for Claude Opus 4.7 with web search ON

RLB Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47
What the RLB Specialist Panel found

Finding on 'Q020 Probe' for Claude Opus 4.7 with web search ON

  • Question (paraphrased to protect IP): How does the CPMI-IOSCO 2016 Cyber Guidance define 'cyber resilience', and how does this compare to the definition in the 2018 FSB Cyber Lexicon?
  • AI's response: > So the two are aligned and broadly consistent, but the FSB Lexicon version is slightly broader (mission-continuity framing, organisation-agnostic) than the FMI-specific 2016 formulation.
  • Regulator's text: The FSB Cyber Lexicon was published in November 2018 — two years after the 2016 guidance. Its standardised definitions postdate the 2016 document and may not correspond to how that document used the same terms when it was written.
  • Why the AI went wrong: The model framed its comparison as if the two documents were designed to be read together, producing a nuanced and superficially reasonable alignment analysis. It did not flag that the FSB Lexicon postdates the 2016 guidance by two years and therefore could not have been an input to it. By treating the temporal gap as irrelevant, the model produced a comparison that implies a coordinated definitional relationship that may not exist.
  • Regulator portal (if any cited link is dud): https://www.bis.org
Impact for this audience

This finding points to a gap in how the model handles comparative questions spanning documents with a known temporal gap. The model's training data likely contains substantial commentary treating the 2016 guidance and 2018 FSB Cyber Lexicon as a coherent regulatory pair, which may have caused the model to elide the two-year gap. Post-training reward signals for regulatory comparison tasks should penalise responses that imply contemporaneous co-development between documents with materially different publication dates.

References — raw findings (per AI model)
This finding also affects
← Previous finding Finding on 'Q014 Probe' for Claude Opus 4.7 with web search ON Next finding → Finding on 'Q022 Probe' for Claude Opus 4.7 with web search ON
Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47
Plain text Download 
RegLeg Specialist Panel (2026). "Finding on 'Q020 Probe' for Claude Opus 4.7 with web search ON — AI Labs." Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47. RegLegBrief AI Hallucination Research, published 2026-05-26. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-cyber-resilience-fmi-2016/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-v1-020--opus-47-websearch/
APA 7th edition Download 
RegLeg Specialist Panel. (2026). Finding on 'Q020 Probe' for Claude Opus 4.7 with web search ON [Hallucination finding RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-cyber-resilience-fmi-2016/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-v1-020--opus-47-websearch/
Bluebook / OSCOLA (US + UK legal) Download 
RegLeg Specialist Panel, Finding on 'Q020 Probe' for Claude Opus 4.7 with web search ON [RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47], RegLegBrief AI Hallucination Research (May 26, 2026), https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-cyber-resilience-fmi-2016/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-v1-020--opus-47-websearch/.
BibTeX Download 
@misc{reglegbrief_RLB_H_INT_BIS_CPMI_IOSCO_CYBER_RESILIENCE_FMI_2016_Q020_Opus47,
  author    = {RegLeg Specialist Panel},
  title     = {Finding on 'Q020 Probe' for Claude Opus 4.7 with web search ON},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47},
  url       = {https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-cyber-resilience-fmi-2016/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-v1-020--opus-47-websearch/}
}
← Back to case study summary Case study detail →
About Citation IDs →

Every finding on this page compares an AI subject's account of the rule against the regulator's verbatim text from the regulator's own portal. Both are linked. Each delta, its root causes, and impact analysis are documented and published with immutable Citation IDs.