AI Hallucination Research › Regulators › Major advanced economies › GB › FCA › CONSUMER-DUTY-PS22-9 › White paper › Detail › Finding
AI Labs · published 2026-05-26 · methodology v2.1

Claude Sonnet 4.6 with web search

RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q002-Sonnet46
What the RLB Specialist Panel found
  • Question (paraphrased to protect IP): Was the Consumer Duty created through primary legislation or FCA rulemaking? What is the legal basis for Principle 12 and PRIN 2A, and did FSMA 2023 have any role in creating it?
  • AI's response: > "The Consumer Duty was introduced by FCA rules, not by an Act of Parliament. The legal basis is the FCA's statutory rule-making power under the Financial Services and Markets Act 2000 (FSMA 2000)."
  • Regulator's text: FSMA 2023 did not create the Consumer Duty.
  • Why the AI went wrong: The model correctly identified that the Consumer Duty is FCA rulemaking under FSMA 2000, but it did not address the role of FSMA 2023, which was specifically raised in the question. The response omits any engagement with whether FSMA 2023 had any role — a question for which the FCA's published record provides a clear answer. By not addressing this element, the model left a material gap that could mislead a reader into thinking FSMA 2023 is simply irrelevant rather than affirmatively excluded.
  • Cited source(s):
  • https://www.akingump.com/en/insights/alerts/new-fca-principle-12-consumer-duty — Pretextual
  • Regulator portal (if any cited link is dud): https://www.fca.org.uk
Impact for this audience

This finding implicates the model's handling of compound regulatory questions where one element has a clear published answer (the general legal basis) and another element requires engagement with a specific recent Act (FSMA 2023). The model answered the first element correctly and silently omitted the second. This is a selective engagement pattern — the model responded to the part of the question it had strong training coverage on and left the novel element unaddressed. Eval probes that include a secondary specific element alongside a general question would test this pattern.

References — raw findings (per AI model)
This finding also affects
← Previous finding Finding 8. Claude Opus 4.7 with web search Next finding → Finding 10. Claude Sonnet 4.6 with web search
Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

Plain text 
RegLeg Specialist Panel (2026). "Claude Sonnet 4.6 with web search — AI Labs." Citation ID: RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q002-Sonnet46. RegLegBrief AI Hallucination Research, published 2026-05-26. https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-002--sonnet-46-websearch/
APA 7th edition 
RegLeg Specialist Panel. (2026). Claude Sonnet 4.6 with web search [Hallucination finding RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q002-Sonnet46]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-002--sonnet-46-websearch/
Bluebook / OSCOLA (US + UK legal) 
RegLeg Specialist Panel, Claude Sonnet 4.6 with web search [RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q002-Sonnet46], RegLegBrief AI Hallucination Research (May 26, 2026), https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-002--sonnet-46-websearch/.
BibTeX 
@misc{reglegbrief_RLB_H_GB_FCA_CONSUMER_DUTY_PS22_9_Q002_Sonnet46,
  author    = {RegLeg Specialist Panel},
  title     = {Claude Sonnet 4.6 with web search},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q002-Sonnet46},
  url       = {https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-002--sonnet-46-websearch/}
}
← Back to case study summary Case study detail →
About Citation IDs →