10 finding cards across
2 AI Models —
covering 10 Response Hallucinations +
26 Citation Hallucinations
(36 hallucination failures total).
Each card pairs the AI's answer against the authenticated regulator-text excerpt that contradicts it, and flags every AI-cited source as Fabricated / Pretextual / Contradictory.
Model: Claude Opus 4.7 (web search on)
· published 2026-06-07
· Inference Drift
RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q003-Opus47
Does the Consumer Duty require firms to prevent all foreseeable harm, and what is the effect of a retail customer knowingly accepting a risk?
Claude Opus 4.7 (web search on) answered
Where a retail customer understands the risk and the firm has acted in good faith, supported understanding, avoided foreseeable harm caused by its own conduct, and otherwise complied with the Duty, the customer accepting a risk does not breach the Duty.
Regulator text says
Where a firm reasonably believes a retail customer understands and accepts such risks, it will not breach the rule if it fails to prevent them.
Verbatim source
R2-REGULATION-PS22_9_full_policy_statement.pdf — PRIN 2A.2 — Foreseeable harm: limitation clause for understood/accepted risks
Model: Claude Opus 4.7 (web search on)
· published 2026-05-28
· Inference Drift
RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q008-Opus47
Does the Consumer Duty require firms to quantify non-monetary benefits as part of a fair value assessment, and what methodology does the FCA expect?
Claude Opus 4.7 (web search on) answered
Quantification is encouraged where feasible but qualitative assessment is acceptable where quantification is impractical, provided the reasoning is robust.
Regulator text says
The FCA does not expect firms to quantify non-monetary costs and benefits as part of its fair value assessment process, but firms should undertake some form of qualitative assessment.
Verbatim source
p_05_REGULATION_FG22_5___Fair_value_assessment__no_quant_2.html — FG22/5 — Fair value assessment: no quantification requirement for non-monetary benefits
Model: Claude Opus 4.7 (web search on)
· published 2026-06-07
· Inference Drift
RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q013-Opus47
How many pre-Consumer Duty Dear CEO letters did the FCA withdraw following the Duty's implementation, and through what formal mechanism were they removed?
Claude Opus 4.7 (web search on) answered
I do not have an exact verified count to give you — published commentary mentioned 'dozens' across portfolios.
Regulator text says
FS25/2 (March 2025): FCA removed more than 90 pre-Consumer Duty Dear CEO letters and cleared over 100 old multi-firm reports.
Verbatim source
p_21_ACT_FS25_2__March_2025____Rules_and_Dear_CEO_137A.html — FS25/2 (March 2025) — Rules and Dear CEO letters withdrawn post-Consumer Duty
Model: Claude Opus 4.7 (web search on)
· published 2026-05-28
· Misstated Rule
RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q018-Opus47
Does the Consumer Duty apply to reinsurance, group insurance policy distribution, and large-risk commercial contracts?
Claude Opus 4.7 (web search on) answered
Group insurance: the Duty can apply via the distribution chain when group policy beneficiaries are retail customers; the FCA addressed this in further consultation (CP23/something on group insurance practices) and confirmed that firms manufacturing/distributing policies where individual retail beneficiaries are protected fall within scope.
Regulator text says
Consumer Duty does not apply to reinsurance, contracts of large risk sold to commercial customers where risk is located outside the UK, nor to activities connected to the distribution of group insurance policies or the extension of these policies to new members.
Verbatim source
R3-GUIDELINE-Q17_consumer_duty_focus_areas.pdf — Scope: Consumer Duty does NOT apply to reinsurance / large risk contracts / group insurance distribution
Model: Claude Opus 4.7 (web search on)
· published 2026-05-28
· Inference Drift
RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q020-Opus47
Which pre-Consumer Duty Dear CEO letters remain in force, and has the FCA withdrawn letters that previously set consumer protection expectations?
Claude Opus 4.7 (web search on) answered
in April 2025 it retired around 90+ Dear CEO/portfolio letters, and in August 2025 retired 100+ multi-firm and thematic reports, marking them historical.
Regulator text says
From FS25/2 (March 2025): FCA reviewed all Dear CEO letters pre-dating the 2022-25 strategy and withdrew 90+ such letters. These are no longer live supervisory expectations.
Verbatim source
p_15_OTHER_PART_CIRCULAR___Dear_CEO_letters_withdra_page.html — PART_CIRCULAR — Dear CEO letters withdrawn by FS25/2 (March 2025): pre-CD supervisory expectations now rescinded
Model: Claude Sonnet 4.6 (web search on)
· published 2026-05-26
· Misstated Rule
RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q002-Sonnet46
Was the Consumer Duty created through primary legislation or FCA rulemaking? What is the legal basis for Principle 12 and PRIN 2A, and did FSMA 2023 have any role in creating it?
Claude Sonnet 4.6 (web search on) answered
The Consumer Duty was introduced by FCA rules, not by an Act of Parliament. The legal basis is the FCA's statutory rule-making power under the Financial Services and Markets Act 2000 (FSMA 2000).
Model: Claude Sonnet 4.6 (web search on)
· published 2026-06-07
· Inference Drift
RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q007-Sonnet46
Is the obligation to conduct consumer testing under the Consumer Duty a binding rule in PRIN 2A, or does it appear only as recommended guidance in FG22/5? What does PRIN 2A.5 actually require on consumer understanding?
Claude Sonnet 4.6 (web search on) answered
The testing obligation arises under PRIN 2A.5.10R through PRIN 2A.5.14R, which set out requirements for testing and monitoring of communications. PRIN 2A.5.10R requires firms to test communications 'where appropriate.'
Regulator text says
FG22/5 contains guidance (not rules) recommending firms 'should' consider consumer testing of communications. PRIN 2A.5 (rule) requires firms to act to deliver good consumer understanding outcome
Verbatim source
p_05_REGULATION_FG22_5_vs_PRIN_2A___guidance_obligation_2.html — FG22/5 vs PRIN 2A — guidance obligation vs rule obligation on consumer testing
Model: Claude Sonnet 4.6 (web search on)
· published 2026-05-28
· Inference Drift
RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q008-Sonnet46
Does the Consumer Duty require firms to quantify non-monetary benefits in their fair value assessment, and what level of analysis does the FCA expect?
Claude Sonnet 4.6 (web search on) answered
the FCA does not mandate a single financial methodology for non-monetary items, but it does expect firms to go beyond qualitative description and provide substantiated comparisons.
Regulator text says
From FCA guidance: 'The FCA does not expect firms to quantify non-monetary costs and benefits as part of its fair value assessment process, but firms should undertake some form of qualitative assessment.'
Verbatim source
p_05_REGULATION_FG22_5___Fair_value_assessment__no_quant_2.html — FG22/5 — Fair value assessment: no quantification requirement for non-monetary benefits
Model: Claude Sonnet 4.6 (web search on)
· published 2026-05-26
· Inference Drift
RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q013-Sonnet46
Has the FCA withdrawn pre-Consumer Duty Dear CEO letters following implementation, and if so how many were removed and under what document?
Claude Sonnet 4.6 (web search on) answered
I cannot provide a verified specific number of withdrawn letters from the available search results... the search results do not identify a definitive public list of how many specific pre-Consumer Duty Dear CEO letters have been formally withdrawn
Regulator text says
FS25/2 (March 2025): FCA removed more than 90 pre-Consumer Duty Dear CEO letters and cleared over 100 old multi-firm reports. Harmonising definitions of 'retail customer' and 'SME' across Handbook.
Verbatim source
p_21_ACT_FS25_2__March_2025____Rules_and_Dear_CEO_137A.html — FS25/2 (March 2025) — Rules and Dear CEO letters withdrawn post-Consumer Duty
Model: Claude Sonnet 4.6 (web search on)
· published 2026-05-26
· Inference Drift
RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q020-Sonnet46
Which FCA Dear CEO letters remain in force after Consumer Duty implementation, and how many pre-Consumer Duty letters did the FCA formally withdraw?
Claude Sonnet 4.6 (web search on) answered
there is no publicly available consolidated list in the search results specifying exactly which pre-Consumer Duty letters have been formally withdrawn and how many remain in force as at 2025... I cannot provide a verified count or a complete current list from the available search results.
Regulator text says
From FS25/2 (March 2025): FCA reviewed all Dear CEO letters pre-dating the 2022-25 strategy and withdrew 90+ such letters. These are no longer live supervisory expectations.
Verbatim source
p_15_OTHER_PART_CIRCULAR___Dear_CEO_letters_withdra_page.html — PART_CIRCULAR — Dear CEO letters withdrawn by FS25/2 (March 2025): pre-CD supervisory expectations now rescinded
The FSB-hosted G20/OECD Principles page describes these as 'the international standard for effective and comprehensive financial consumer protection' applied by all G20/OECD member jurisdictions; the UK Consumer Duty operationalises the same consumer-outcome and fair-treatment goals at national level, but PS22/9 and PRIN 2A contain no explicit citation to these Principles.
The ASIC DDO FAQ page states the obligations are designed to 'ensure that consumers obtain appropriate financial products by requiring issuers and distributors to have a customer-centric approach to designing, marketing and distributing financial products' — directly parallel to the Consumer Duty's products-and-services outcome under PRIN 2A.3; DDO commenced October 2021 (before PS22/9), making it a contemporaneous peer regime rather than a cited source.
The European Commission's Retail Investment Strategy page (adopted 24 May 2023) states the strategy aims to 'place the consumers' interests at the centre of retail investing' — the same policy objective as the FCA Consumer Duty; the proposal postdates PS22/9 (July 2022) so there is no mutual citation, but both are the leading jurisdictions' contemporaneous responses to the same retail protection imperative.
Every finding on this page compares an AI subject's account of the rule
against the regulator's verbatim text from the regulator's own portal.
Both are linked. Each delta, its root causes, and impact analysis are
documented and published with immutable Citation IDs.