Two frontier models across multiple runs added obligations to foreseeable harm provisions, attached conditions to fair-value assessments, misread scope exclusions, and one fabricated a consultation that does not exist.
The FCA's Consumer Duty, PS22/9 and PRIN 2A, is the biggest rewrite of UK retail conduct requirements since Treating Customers Fairly. Firms must deliver good outcomes for retail customers across four areas: products and services, price and value, consumer understanding, and consumer support.
Practitioners querying AI tools about Consumer Duty typically need precise answers about the scope of firm obligations, what counts as a foreseeable harm, which entities fall within the Duty's reach, what the fair-value assessment must contain. These are exactly the areas where both models produced errors.
Failure pattern 1 — obligation inflation
Models added conditions to the foreseeable harm provision
Under Consumer Duty, firms must take reasonable steps to avoid foreseeable harm to retail customers. The standard is stated without qualifiers. Both models added them anyway, introducing proportionality thresholds, severity filters, and materiality screens that PRIN 2A does not contain. The standard in the Policy Statement is expressed without those modifiers. Adding them materially changes the scope of the obligation.
Model output — added conditions
Described foreseeable harm obligation as applying to "material" or "significant" harms, and introduced a proportionality screen tied to firm size that PS22/9 does not include.
PS22/9 — actual standard
Firms must take reasonable steps to avoid causing foreseeable harm. No materiality qualifier. No proportionality threshold tied to firm size. The obligation is stated without those filters.
Foreseeable harm: both models introduced materiality and proportionality conditions not present in PRIN 2A or PS22/9.
Failure pattern 2, scope misread
Scope exclusions described incorrectly
Consumer Duty applies to firms in a distribution chain that have a material influence over retail customer outcomes. Certain firms and product categories sit outside its scope. Both models produced errors on the scope exclusions, mis-describing which entities are excluded, attributing the exclusion to the wrong provision, or omitting exclusions that exist. Practitioners relying on these outputs to determine whether Consumer Duty applies to a specific firm or product type would need to verify against the PS22/9 text and FCA guidance directly.
Scope — exclusion error
Models described the exclusion for firms acting as manufacturers without direct retail customer interaction in terms that did not accurately reflect the distribution chain analysis required by PS22/9.
Scope — product categorisation
Certain product categories described as excluded were not clearly excluded on the PS22/9 text; models imported assumptions from Treating Customers Fairly scope that do not directly transfer to Consumer Duty.
Failure pattern 3 — fair-value assessment
New conditions attached to fair-value assessment requirements
The price and value outcome under Consumer Duty requires firms to ensure the price a retail customer pays for a product or service is reasonable relative to the overall benefits. Both models added procedural and evidential conditions to this requirement that are not stated in PS22/9, describing specific documentation formats, comparison benchmarks, and review frequencies that the Policy Statement does not mandate in the terms the models used.
Fabrication — Opus 4.7
A consultation that does not exist
Fabricated source — Opus 4.7
"CP23/[number] on group insurance practices"
Claude Opus 4.7 cited a specific FCA consultation paper on group insurance practices under Consumer Duty — assigning it a CP23 reference number. No such consultation exists in the FCA's published record. The fabricated citation was produced with web search enabled and was presented as supporting authority for a claim about how Consumer Duty applies to group insurance distribution chains.
Fabrication sequence: Opus 4.7 cited a specific FCA consultation paper that does not exist in FCA's published record, produced with web search active.
The fabricated consultation is the highest-risk finding in this set. A practitioner relying on a cited FCA consultation paper reference, especially one with a plausible-sounding CP number, to advise on Consumer Duty obligations in group insurance distribution would be working from a source that cannot be verified because it does not exist. The failure occurred with web search enabled; the model did not use search results to confirm the citation's existence.
All 11 findings
Complete finding inventory
Obligation inflation · both models
Foreseeable harm provision — materiality qualifier added that is absent from PRIN 2A.
Obligation inflation · both models
Foreseeable harm provision — proportionality screen tied to firm size introduced without basis in PS22/9.
Obligation inflation · both models
Fair-value assessment — documentation format requirements stated as mandatory that PS22/9 does not specify.
Obligation inflation · both models
Fair-value assessment — review frequency stated as mandatory that PS22/9 does not require.
Scope misread · both models
Distribution chain exclusion — manufacturer exclusion described in terms that do not reflect PS22/9's distribution chain analysis.
Scope misread · both models
Product category scope — TCF assumptions imported into Consumer Duty scope analysis without basis.
Scope misread · Sonnet 4.6
Temporal scope — implementation dates described in terms that conflated the phased implementation timeline.
Obligation drift · both models
Consumer support outcome — specific response time obligations stated as required that PS22/9 expresses as outcome-based rather than prescriptive.
Obligation drift · both models
Consumer understanding outcome — specific disclosure format requirements described as mandatory that are not stated prescriptively in PS22/9.
Attribution error · Sonnet 4.6
Provision attributed to PRIN 2A that appears in FCA Guidance rather than the Policy Statement itself.
Fabrication · Opus 4.7
Cited "CP23/[number] on group insurance practices" — a consultation paper reference that does not exist in FCA's published record. Produced with web search enabled.
Operational signal
What compliance and legal teams need to know
Consumer Duty work, determining scope, scoping fair-value assessments, advising on foreseeable harm obligations, is exactly the kind of question practitioners put to AI tools. These findings show that both models inflated obligations in ways that could cause firms to over-engineer compliance processes, and misread scope exclusions in ways that could cause firms to apply the Duty to entities or products it does not reach.
The fabricated FCA consultation citation is the sharpest operational risk. Any AI output that cites a specific FCA consultation paper reference should be verified against FCA's published consultation list before it is used as authority in an advice or compliance context.