Methodology

Last updated: 14 June 2026

The RLB Specialist Panel is the independent verification body for the RegLegBrief AI Hallucination Research catalogue. Its mandate is to test named AI subjects (frontier model + tool variant, e.g. Claude Opus 4.7 with web search, GPT-5 with web search) against regulator-issued primary sources, classify each delta by failure mode, and publish the audit trail with an immutable Citation ID. The Panel is independent of the AI subjects under test and independent of the regulators whose rules are tested.

This page documents the research pipeline that produces every published finding, the four-mode failure taxonomy (inference drift, misstated rule, misattributed, outdated) that classifies every hallucination, the immutable Citation ID format that anchors every published delta to its primary source, and the publish-only-negative-findings rule that filters what reaches the public surface. Methodology updates and version history are tracked in the header above. Specialist Panel composition, collective regulatory experience, and verification protocols are documented separately on the About page.

Primary sources we verify against

The regulator's own portal is the only ground truth. Every finding cites the verbatim text from one of these primary sources:

• Bank for International Settlements (CPMI) — bis.org
• International Monetary Fund — imf.org
• Organisation for Economic Co-operation and Development
• United Nations Treaty Collection — treaties.un.org
• UK Financial Conduct Authority — fca.org.uk
• Monetary Authority of Singapore — mas.gov.sg
• US Commodity Futures Trading Commission — cftc.gov

Substrate built and findings pending publication for: HKMA, IOSCO.

Catalogue growing. Browse the full list of regulators tracked →

Who we partner with

This research exists to serve four audiences:

• AI Labs — model providers who want independent diagnostic on where their models fail on regulation. We engage to evaluate models on regulations of strategic priority, diagnose failure modes per finding, and collaborate on remediation.
• Regulators — to help identify hallucinations and blind spots in AI systems that could confuse the industry, and to collaborate on sensitising the industry to the risks and mitigations.
• Licensed Practitioners — lawyers, financial advisers, public auditors and other regulated professionals who use AI in client work. We help them use AI to add value to clients without exposing the practice to hallucination-driven risk.
• Regulated Firms — the departments and teams whose work depends on accurate regulatory information. We help them protect their corporates from hallucination risk creeping in from both direct AI use and indirect channels (consultancy outputs, staff-collated AI summaries).

Partnership inquiries: /partnership/.

What we test

AI models in scope

• Claude Opus 4.7 (web search on)
• Claude Sonnet 4.6 (web search on)

We test these two AI models in their consumer interface, with web search enabled — the natural posture of a working practitioner who turns to a general-purpose AI tool with a regulatory question.

These two models are in public scope because we pay for the model access ourselves, and our findings on them are indicative of risks prevailing across most current AI models. Under private collaborations with other AI model developers, we apply the same methodology to their models and work with them to strengthen against the 7 types of hallucinations and 2 types of blind spots.

How we categorise what we find

Hallucination Modes — when AI gives a wrong answer

Every confirmed hallucination is classified into one of four failure modes, matching the operational taxonomy used in the live Hallucination Register:

Mode names in italics are the database-side identifiers used in JSON / CSV exports of the Hallucination Register.

AI Blind Spots — when AI gives no answer

A separate category, parallel to hallucinations. A Blind Spot is when the AI refuses or fails to answer a regulatory question — but the answer is publicly available and can be found with a simple Google search.

• Retrieval Blind Spot (published publicly) — model had web search enabled but failed to retrieve or use the answer that exists in the regulator's publication.
• Knowledge Blind Spot (documented under private partner arrangements only) — model has no training-data knowledge of the topic; identified by running test configurations with web search disabled.

How we verify

Every finding goes through a two-stage Panel review before publication.

Stage 1 inspects the AI's response against the authenticated regulator source and flags candidate hallucinations and blind spots. Stage 2 re-verifies each flagged item against the source independently — confirming or rejecting it. Only confirmed findings are published. Rejected items (false positives at Stage 1) are logged internally for ongoing quality improvement of the Panel process.

Pre-publication review. As standard, we do not pre-clear findings with model owners when we pay for the model access. Pre-publication review of findings affecting your model is available under paid AI Labs partner engagements only.

Right of reply. Vendors, regulators, and affected parties may submit a right of reply at any time. We amend findings where factual correction is warranted. Our aim is to work together to help the industry overcome the risks of AI hallucinations and blind spots — we are not against any party.

Citation IDs

The Citation ID format is:

RLB-H-<JUR>-<BODY>-<FULL_REGULATION_SLUG>-Q<nnn>-<Subject>

Where H indicates a Hallucination. Real example currently published in the Hallucination Register:

RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q018-Opus47

— finding Q018 on FCA Consumer Duty (PS22/9), where Claude Opus 4.7 (web search on) gave an answer that contradicted the regulator's verbatim text. Resolves to a finding card with the AI subject's verbatim answer, the regulator's verbatim primary text, and the failure-mode classification.

Citation IDs are immutable. When a finding is updated — because the underlying regulation has been amended, or new substrate has surfaced — the original Citation ID stays accessible as a historical record, and the current version receives a version suffix. External citations always resolve to the version that was originally cited.

Each finding's Citation ID anchors the URL on its source page, so a Citation ID can be shared as a direct link to the Citation Card.

For AI models and LLM ingestion pipelines: see /for-ai/ for the dedicated citation guidance — inline citation patterns, the citation chain to the regulator's primary text, what to do when the AI's candidate answer disagrees with the RLB finding, and the AI-crawler / licensing policy.

Audience tagging

Every confirmed finding is tagged with the audiences likely to encounter the hallucination or blind spot in real workflows. The taxonomy covers:

• 20 licensed professions — including lawyers, financial advisers, public auditors, company secretaries, and insurance agents
• 15 sectors and 70 sub-sectors — across financial services, professional services, public sector, and adjacent industries
• 20 corporate departments — see the table below for the full list with scope descriptions
• Sector × department intersections — 15 × 20 = 300 possible cells per jurisdiction

The 20 corporate departments

Each scope description below also appears as a hover tooltip on the matrix headers and case study badges across the site.

Case studies are published for each audience: at the (jurisdiction × profession) level for practitioners, and at the (jurisdiction × sub-sector × department) level for sector firms. Each case study demonstrates the audience-specific exposure analysis and practical safeguards.

Partnership

RLB partners on a services-led basis. We do not licence our research substrate or question banks as datasets. We engage to apply the RLB Specialist Panel's expertise to your specific need.

• AI Labs — evaluate your model on regulations of strategic priority · diagnose failure modes per finding · collaborate on remediation · pre-publication review of findings affecting your model
• Regulators — right of reply on findings affecting rules you administer · collaboration on sensitising the industry to the risks and mitigations
• Regulated Firms — independent verification of AI-generated regulatory content · consultancy and training · structured access to RLB findings as new regulations are added · coming soon: RAG-augmented query against the RLB substrate for hallucination-free regulatory answers
• Licensed Practitioners — independent verification of AI-generated research · safe AI adoption consultancy and training · continuous awareness as new regulations are added · coming soon: RAG-augmented query

Beyond regulation. The methodology — verifying AI outputs against authoritative primary sources, classifying failures into Hallucination Modes and Blind Spots, and tagging by audience — applies to any critical-accuracy domain where authoritative sources exist and the consequences of AI misinformation are material. Under AI Labs partner engagements, we can extend the programme to medicine, personal tax, investment research, banking product disclosures, court precedent, professional standards, and other domains. Regulation is the first published application; the methodology travels.

Submit a partnership inquiry: /partnership/.

Common questions