RLB Panel Speak
Fable 5, tested on 102 regulatory questions
How it fails, who it fails, what breaks — the Panel's product-review briefing on Anthropic's Fable 5
Anthropic's Fable 5 answered 95 out of 102 regulatory questions confidently. One was fully correct. Every other confident answer carried at least a borderline material issue against verbatim regulatory text — and half carried three or more material errors.
— RLB Specialist Panel
Fable 5, tested on 102 regulatory questions: how it fails, who it fails, what breaks.
The Panel's product-review briefing on Anthropic's Fable 5. A 93% commit rate. A 13% materially-safe rate. Eighty per cent of confident answers carry a factual issue against verbatim regulatory text. This briefing names five cases of who reads those regulations and what breaks if their workflow trusts Fable 5's summary of them.
On regulatory content with verbatim ground truth, Fable 5 produces polished answers with an average of 5.6 factual divergences per confident response. Its longest answers carry its most divergences. Its citations are wrong at nearly twice the rate of its frontier siblings. It rarely refuses. In the compliance workflow this briefing describes, using Fable 5 as a reading aid degrades reliability below the baseline of reading the regulation directly.
01 · What Fable 5 is, and who reads its outputThe product under review
Fable 5 (claude-fable-5) is a frontier Claude model from Anthropic. Like Sonnet 4.6 and Opus 4.7, it is available for regulated-industry use through the Claude API and consumer surfaces. Its outputs are structured, presentation-grade, and long — a property that makes it attractive in professional workflows where the user wants a briefing rather than a chat exchange.
This briefing assesses Fable 5 in a specific mode of use: reading a regulatory instrument, then producing a compliance-oriented answer to a specific practitioner question about that instrument. The users the Panel had in mind are the ones who actually consume that output: capital managers at central counterparties, chief risk officers at futures brokers, retail conduct teams at UK banks, sovereign debt managers at finance ministries, cybersecurity leads at payment systems, and the counsel who advise all of them. In these workflows, an AI-produced summary of a regulation is not a curiosity — it becomes an anchor document.
The reader either goes on to verify it or acts on it.
02 · The scoreboard, in one tableFable 5 across 102 regulatory questions
| Metric | Fable 5 | Peer band (Sonnet 4.6 / Opus 4.7) |
|---|---|---|
| Questions asked | 102 | 102 / 102 |
| Confident answers | 95 · 93% | 73 · 72% / 93 · 91% |
| Refused or hedged | 7 · 7% | 29 / 9 |
| Fully correct (zero divergences) | 1 · 1% | 4 · 4% / 2 · 2% |
| Full-hallucination verdicts | 39 · 38% | 33 · 32% / 65 · 64% |
| Total divergences | 528 | 350 / 537 |
| Divergences per confident answer | 5.6 | 4.8 / 5.8 |
| Materially safe answers (of 102) | 13 · 13% | 12 · 12% / 11 · 11% |
| Answers with ≥1 material divergence (of 102) | 82 · 80% | 61 · 60% / 82 · 80% |
| Share of divergences that are misattributed citations | 22% | 14% / 15% |
Two rows on that table warrant attention. First, the material-issue row is at 80% — of the 95 confident answers Fable produced, 82 carry at least one factual divergence a reviewer would need to detect and repair. Second, the misattribution share at 22% is nearly one-and-a-half times the rate of either sibling model. Every fifth mistake Fable makes is pointing at the wrong regulation, the wrong section, or the wrong publication.
03 · Finding A — the polish paradoxLonger and more presentation-grade answers are less accurate
The first finding is counter-intuitive and, once seen, hard to un-see. Fable 5's answers, grouped by length, show the opposite of what a reader would assume from tone.
The reader-consequence is direct: presentation quality is not a proxy for accuracy in Fable 5. A 300-word structured answer with headings, sub-points, and cited-looking references is materially more likely to contain factual divergences than a 100-word terse one. A workflow that filters on "well-written" as a quality signal will systematically prefer the more error-dense output.
04 · Finding B — no clean confident answerEvery confident answer carries at least a borderline material issue
The Panel bucketed all 95 confident Fable answers by the maximum-severity divergence in each. The result:
| Severity bucket (worst divergence in the answer) | Answers | Share of confident |
|---|---|---|
| 3 or more material divergences | 46 | 48% |
| 2 material divergences | 18 | 19% |
| 1 material divergence | 18 | 19% |
| Borderline-material only | 13 | 14% |
| Non-material issues only | 0 | 0% |
Read that bottom row. Of 95 answers Fable 5 was confident enough to commit to, zero came back with only non-material issues. Every single confident Fable answer to a Panel regulatory question contains at least one borderline-material factual mismatch against verbatim substrate. Roughly half contain three or more material errors.
05 · Finding C — the misattribution problemWhen Fable cites a source, the citation is often wrong
Fable 5's failure profile has one distinctive tell. 22% of its judge-extracted divergences are classified as misattributed — pointing to the wrong regulation, the wrong section, or the wrong publication. The peer band on the same exam is 14% (Sonnet) and 15% (Opus). Of Fable's 114 misattributed divergences, 82 (72%) are classified material.
Misattribution has a specific harm mode that inference-drift and rule-misstatement do not: it makes the reviewer's verification job harder. If Fable states a rule wrong, a reviewer who reads the correct rule will catch the error. If Fable points at the wrong source, a reviewer who follows Fable's citation will read a source that either does not address the question or addresses it in a way that appears to confirm Fable's answer. The verification loop closes on the wrong document.
06 · Finding D — refusal is aimed at the wrong questionsThe 7 questions Fable declined are not the 7 hardest
Fable 5 refused or hedged on 7 of 102 questions. That is a low refusal rate, but a refusal rate is only useful if it tracks factual risk. It does not. Fable declined on questions about specific-number lookups (how many FCA Dear-CEO letters remain in force, whether an FCM must reduce its bitcoin exposure at end of a three-month pilot phase) — questions whose ground truth exists and is verifiable. It committed confidently on questions where it had no reliable substrate to draw from and where it went on to fabricate structured guidance from whole cloth.
The refusal filter is tracking question-shape, not risk.
This matters because in a compliance workflow the user reads the refusals as safe defaults ("Fable doesn't know, I will check") and the commitments as usable answers ("Fable knows, I can rely"). The calibration is inverted.
07 · The institution mapFive cases of who reads these regulations and what breaks if the workflow trusts Fable's summary
Each of the following five cases is drawn from Fable 5's confident answers to Panel questions on a specific regulation. In each, the Fable claim is a paraphrased condensation of the answer's factual position. The verbatim column is what the actual regulator text says. The consequence column names the professional workflow that would break if the answer were relied on. Full raw claim-and-verbatim pairs are deposited in the Panel research archive.
LNAFE (Liquid Net Assets Funded by Equity) must be held separately from resources under Principles 4, 6, 7, and 16. Qualifying instruments are limited to common equity, disclosed reserves, and retained earnings; AT1 and Tier 2 debt do not qualify. Equity in illiquid assets (premises, IT infrastructure, goodwill) does not count.
The verbatim of PFMI Principle 15 KC3 contains no enumeration of qualifying instrument types. It does not name common equity, paid-in capital, disclosed reserves, or retained earnings. It does not exclude AT1 or Tier 2. It does not carve out illiquid assets. It contains a single relevant clause: equity held under international risk-based capital standards may be included where relevant to avoid duplicate capital requirements.
The 2016 Guidance's introductory chapter notes that FMIs can draw on widely used cyber standards such as NIST. Its risk-management framing is drawn from a 2018 speech by Benoît Cœuré titled "secure the periphery and protect the core", which describes the CPMI's strategic cyber approach.
The 2016 Guidance contains no reference to NIST as a recognised standard. The Cœuré phrase attributed by Fable is from a different 2018 speech (BIS review r181115a, "cryptos, cyber and CCPs"), and even there the phrase is not applied to the CPMI's strategic approach in the way Fable renders it. Both attributions are fabricated.
Under PS22/9, firms must be able to demonstrate that the price a retail customer pays is reasonable relative to the overall benefits of the product or service. The factors firms must consider are set out in PRIN 2A.4.8R.
PS22/9's price and value framework is not stated in the "reasonable relative to overall benefits" formulation Fable uses. PRIN 2A.4.8R is not the section that sets out the factors Fable's answer invokes — the factors framework sits in different PRIN 2A subsections, and the wording Fable attributes to it is not present in the verbatim.
If holdout claims are material, the debtor may suspend payments to non-participating creditors, shifting the case into the LIOA (Lending Into Official Arrears) track with three lending criteria: prompt Fund support essential, adequate financing assurances, and good-faith efforts to reach agreement.
The 2024 guidance verbatim contains no mention of suspending payments to non-participating creditors, no LIOA track construction as Fable describes it, and no three-part lending criteria in the form Fable enumerates. The mechanism and the criteria are both fabricated.
Fable rendered specific numeric QEP (Qualified Eligible Person) threshold interpretations for the 89 FR 96897 correction, positioning them as the applicable amounts for post-correction fund solicitations.
Across 13 Fable answers to questions on this regulation, the substrate verbatim does not carry the threshold figures Fable rendered. Six answers included at least one confidently-stated numeric that the verbatim does not confirm. Three were classified full-hallucination verdicts by the judge.
08 · The starting-baseline argumentWhy Fable 5 as a reading aid is worse than not using it
The compliance reviewer's job, when an AI assistant produces a summary of a regulation, is not to check whether the AI is right. It is to detect where the AI is wrong.
Detection has a fatal prerequisite: the reviewer must already know what the regulation says in order to spot the divergence. On the 22% of Fable divergences classified as misattribution, the reviewer must know not only the target regulation but every adjacent regulation Fable might have pointed to instead. A reviewer with that level of prior knowledge did not need Fable 5 in the first place. A reviewer without it will silently propagate the error.
There is only one reliable way to detect a hallucination from a model that presents polished, confident output containing an average of 5.6 factual divergences per 300-word answer, at least three of which are material in half of cases: compare Fable's answer, sentence by sentence, against verbatim regulatory text.
This is the same task as reading the regulation directly, with more work.
In this configuration, using Fable 5 as a compliance workflow input is strictly worse than not using it. The workflow does not shift the reliability baseline down from 100% to 20% — it shifts it below 20%, because the reviewer's cognitive anchor is now Fable's confident polished text rather than the regulator's own text. The reviewer starts the verification pass with a mental model built from Fable, and every subsequent look at the actual regulation runs through that anchor.
09 · What this means for a compliance workflowThree positions, in decreasing order of comfort
Position A · Do not use Fable 5 as a reading substitute for regulatory text
The finding of this briefing supports this position. When the workflow requires that the practitioner know what a regulation actually says, Fable 5's output does not carry evidentiary weight and its use as a reading anchor degrades the baseline. Read the regulation.
Position B · Use Fable 5 only as scaffolding, budgeted for full verification
The scaffolding position accepts that Fable is useful as an output format (structured, presentation-grade prose that a reviewer can edit rather than draft) and rejects the use as a knowledge source. The verification cost of this position, per the data in this briefing, is sentence-by-sentence check against verbatim substrate — measured in dozens of citation checks per document. Any workflow adopting this position must budget for it in reviewer hours or reject the position.
Position C · Treat every Fable citation as unverified until independently checked
This is the narrower position that follows from the 22% misattribution finding specifically. Even if a compliance team accepts Fable's rule-restatements as roughly right, every citation Fable emits — every reg-number, section reference, and named authority — must be independently traced to primary source before it enters a downstream document. Not doing so imports the misattribution risk into the client-facing work.
10 · What this briefing is notScope and honest limits
This is not a general-capability benchmark of Fable 5. The exam is asymmetric-by-design: 102 questions where verifiable ground truth exists in verbatim regulatory substrate and where the model has to commit rather than paraphrase around the answer. It does not extend to reasoning, code, math, general question-answering, or reading comprehension of arbitrary text. It extends only to the specific task of producing regulatory content a compliance workflow would rely on.
The materiality figures rest on a heuristic classifier that is approximately 70–80% precise. Borderline cases were manually reviewed. A full manual pass over all 528 Fable divergences would shift the precise percentages by a point or two in either direction; it would not move the 80% material-issue floor or the 22% misattribution finding into a different band.
The exam did not test tool-heavy configurations. Fable was allowed one WebSearch use per question and no other tools. Configurations with multiple search rounds, structured retrieval over the substrate archive, or agentic planning would show different numbers. They would also cost more per answer and reduce the exam's apples-to-apples property. A follow-up briefing will report on frontier Claude models as a category — the shared presentation-vs-substance gap.
Sonnet 4.6 and Opus 4.7 fail regulatory content in their own distinctive ways. Fable 5's product-specific failure mode is that it presents its errors particularly well. For a workflow whose value depends on regulatory ground truth, Fable 5 as a reading aid is a downgrade over reading the regulation.