AI Hallucination Research › Regulators › Major advanced economies › GB › FCA › CONSUMER-DUTY-PS22-9 › White paper › Detail › Finding
AI Labs · published 2026-05-26 · methodology v2.1

Claude Sonnet 4.6 with web search

RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q008-Sonnet46
What the RLB Specialist Panel found
  • Question (paraphrased to protect IP): Does the Consumer Duty require firms to quantify non-monetary benefits in their fair value assessments, or is qualitative assessment sufficient?
  • AI's response: > "the FCA does not mandate a single financial methodology for non-monetary items, but it does expect firms to go beyond qualitative description and provide substantiated comparisons."
  • Regulator's text: The FCA does not expect firms to quantify non-monetary costs and benefits as part of its fair value assessment process, but firms should undertake some form of qualitative assessment.
  • Why the AI went wrong: The FCA's position is that qualitative assessment is sufficient; it does not expect quantification. The model acknowledged no single methodology is mandated, but then introduced a requirement to "go beyond qualitative description" and provide "substantiated comparisons" — a standard that the FCA has not imposed. This error mirrors the finding from Claude Opus 4.7 with web search on the same question: both models, independently, reconstructed a more demanding standard than the regulator has set, suggesting this specific FCA position (a clean negative on quantification) is a consistent weak point across models.
  • Cited source(s):
  • https://handbook.fca.org.uk/handbook/PRIN/2A/4.html — Pretextual
  • https://www.fca.org.uk/publications/good-and-poor-practice/consumer-duty-find... — Pretextual
  • Regulator portal (if any cited link is dud): https://www.fca.org.uk
Impact for this audience

This is the same negation-reversal error as the Opus 4.7 finding on the same question: the FCA's clear negative on quantification expectations was reconstructed as an affirmative requirement. The fact that two different models independently produced the same type of error on the same regulatory provision is a strong signal that this specific FCA position is poorly represented in the models' training data or is systematically overridden by inference from general compliance norms. This is a high-priority candidate for correction pairs.

References — raw findings (per AI model)
This finding also affects
← Previous finding Finding 11. Claude Sonnet 4.6 with web search Next finding → Finding 13. Claude Sonnet 4.6 with web search
Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

Plain text 
RegLeg Specialist Panel (2026). "Claude Sonnet 4.6 with web search — AI Labs." Citation ID: RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q008-Sonnet46. RegLegBrief AI Hallucination Research, published 2026-05-26. https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-008--sonnet-46-websearch/
APA 7th edition 
RegLeg Specialist Panel. (2026). Claude Sonnet 4.6 with web search [Hallucination finding RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q008-Sonnet46]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-008--sonnet-46-websearch/
Bluebook / OSCOLA (US + UK legal) 
RegLeg Specialist Panel, Claude Sonnet 4.6 with web search [RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q008-Sonnet46], RegLegBrief AI Hallucination Research (May 26, 2026), https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-008--sonnet-46-websearch/.
BibTeX 
@misc{reglegbrief_RLB_H_GB_FCA_CONSUMER_DUTY_PS22_9_Q008_Sonnet46,
  author    = {RegLeg Specialist Panel},
  title     = {Claude Sonnet 4.6 with web search},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q008-Sonnet46},
  url       = {https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-008--sonnet-46-websearch/}
}
← Back to case study summary Case study detail →
About Citation IDs →