AI Hallucination Research › Briefings

Briefings Blog

The running blog from the RLB Specialist Panel delves into real-world scenarios where the compliance, legal, or AI lab team interacts with frontier AI models under specific regulations. The blogs are anonymised to remove client-specific details and include insights from the RLB team analysing the hallucinations experienced in AI models while working on these cases. For example, when a model returns a confident answer that contradicts the regulator's primary text, such as a fabricated staff letter, a wrong appendix, or an inverted scope, these issues are discussed here. Each blog explains one set of findings and what it would have meant for the team that would have acted on it, sans this research initiative. This blog is frequently updated, a few times a day.

263 briefings in the archive · Subscribe via Atom: /briefings/feed.xml (this blog) · /feed.xml (all RegLegBrief publications)
Audience colours: AI Labs Practitioner (profession) Sector × Department
Audience
Jur.
Regulator
Profession
Sector
Dept
Range
Sort
Per page
Showing 5 of 263 · page 30 of 53
Saturday, 04 July 2026
Practitioner: Company Secretaries GB FCA

Company Secretaries: AI summaries of FCA Consumer Duty (PS22/9) may understate professional obligations

For Company Secretaries working with Consumer Duty (PS22/9 + PRIN 2A): where Specialist-Panel-verified divergences between frontier AI summaries and the regulator's primary source can affect client work, professional...

Company secretaries supporting boards of regulated firms are increasingly using AI to draft board-pack summaries of Consumer Duty annual board reports, validate Principle 12 mapping for committee minutes, prepare director briefings on PRIN 2A obligations, and reconcile FCA Feedback Statements such as FS25/2 against existing supervisory expectations recorded in board papers. The output sits at the centre of director-attestation packs and audit-committee minutes that auditors and the regulator can request.

Two frontier AI models tested by the RLB Specialist Panel produced 9 substantive failures on this regulation under audit conditions. The failure classes recorded are: Misstated Statutory Architecture, Inference Drift on the Foreseeable-Harm Safe Harbour, Confused Guidance with Rule on Consumer Testing, Inference Drift on Fair Value Quantification Expectation, Hedge in Place of Verified FS25/2 Figure, Refusal to Confirm a Documented FS25/2 Count, Reversed the PRIN 2A Group-Insurance Exclusion, Invented Dual-Event Timeline for a Single FS25/2 Withdrawal, Refusal to Confirm FS25/2 Withdrawal Count.

Questions were prepared by the RLB Specialist Panel based on real practical AI usage in the workflows the respective audience uses AI for, and each finding is bound to verbatim regulator-issued source text held as primary substrate. The Consumer Duty (PS22/9 introducing Principle 12 and PRIN 2A, in force for open products from 31 July 2023 and for closed products from 31 July 2024) is the central retail-conduct regime the FCA now uses to grade firm behaviour, and the failure modes seen here all land inside the day-to-day work product that company secretaries sign off on.

For company secretaries, the operational consequence is direct. A board-pack summary or audit-committee briefing built on the AI's framing imports a defect into director attestations. The next supervisory visit, an internal-audit pull of the board record, or an external review of governance materials will surface the gap, and the secretariat carries the governance-quality exposure.

Citation IDs for the findings in this brief: RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q002-Sonnet46, RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q003-Opus47, RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q007-Sonnet46, RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q008-Opus47, RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q013-Opus47, RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q013-Sonnet46, RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q018-Opus47, RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q020-Opus47, RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q020-Sonnet46. Each citation links to the per-finding record, the AI subject answer, and the regulator-issued substrate excerpt the answer was tested against. The RLB Specialist Panel maintains an audit-traceable record of which model produced which answer, against which substrate passage, and the binding is what makes the finding referenceable in firm work product and in supervisory correspondence.

The findings below are the ones that company secretaries working under the Consumer Duty are most likely to encounter in the AI tools they already use, and the briefing sections that follow read each finding against the regulator-issued text.

Friday, 03 July 2026
Sector: Telecommunications and Dept: ESG & Sustainability INT OECD

Telecommunications ESG & Sustainability teams: documentation and reporting gaps possible from AI reading of Recommendation of the Council on Digital Technologies and the Environment

For Telecommunications ESG & Sustainability teams working with Recommendation of the Council on Digital Technologies and the Environment (2025 Revision): Specialist-Panel-verified findings on where AI summaries...

ESG & Sustainability teams at Telecommunications firms operating under digital infrastructure environmental impact and data-centre energy reporting are increasingly using AI to populate annual sustainability reports with OECD-cited national data-centre energy benchmarks, draft board briefings on digital-infrastructure footprint across European operations, prepare regulatory data submissions referencing OECD national baselines, and validate disclosed figures against regulator-cited primary sources.

The OECD's 2025 Revision of the Recommendation on Digital Technologies and the Environment carries a named, citable statistic on Ireland's data-centre share of metered electricity, drawn from Ireland's Central Statistics Office, that ESG & Sustainability teams at telecommunications firms will reach for when populating sustainability disclosures, ESG investor responses, and regulatory briefings on digital-infrastructure environmental impact. That statistic is exactly the kind of figure the RLB Specialist Panel tested two frontier AI subjects against.

The RLB Specialist Panel issued a Specialist Panel application-style question on the share of Ireland's 2021 metered electricity that data centres accounted for, per the figure cited in the OECD Digital Economy Outlook 2024 chapter referenced by the 2025 Recommendation, sourced from Ireland's CSO (2023). Two frontier AI models tested by the RLB Specialist Panel returned the figure as 14 per cent and extended the answer with a four-point time series running from 5 per cent in 2015 through 21 per cent in 2023. The regulator's verbatim text records 11 per cent in 2021, with no multi-year trajectory.

The failure class is Fabricated Fact: a confidently delivered, citably attributed statistic that does not match the source document, compounded by a fabricated time series that does not appear anywhere in the OECD or CSO published record.

For ESG & Sustainability teams at telecommunications firms, this is operationally consequential because the wrong figure is not a vague paraphrase. It is delivered with a real source chain, CSO 2023 via OECD Digital Economy Outlook 2024, that survives standard reference-check review. An ESG team at a telecommunications firm consulting AI for the OECD's Ireland data-centre energy figure would receive 14 per cent, not the verbatim 11 per cent, along with a fabricated four-point time series (2015 to 2023) that appears nowhere in the source.

If that figure enters an annual sustainability report, a board briefing, or a regulatory data submission, the firm has published a material misstatement attributed to a real and checkable primary source. The exposure is compounded in international operating environments where multiple national regulators independently cite the same OECD benchmark: a telecommunications group with European operations may find its disclosed figure contradicted not only by the OECD document but by the regulator's own published baseline, creating a discrepancy that requires formal correction and explanation.

The audit's finding on this question is published with an immutable RLB Citation ID. The relevant entry is RLB-H-INT-OECD-OECD-DIGITAL-TECHNOLOGIES-ENVIRONMENT-2025-Q006-Sonnet46. The full audit is published at the OECD Digital Technologies and the Environment Recommendation (2025 Revision) hub on RegLegBrief.com.

Sector: Statutory Boards & Agencies and Dept: ESG & Sustainability INT OECD

Statutory Boards & Agencies ESG & Sustainability teams: documentation and reporting gaps possible from AI reading of Recommendation of the Council on Digital Technologies and the Environment

For Statutory Boards & Agencies ESG & Sustainability teams working with Recommendation of the Council on Digital Technologies and the Environment (2025 Revision): Specialist-Panel-verified findings on where AI...

ESG & Sustainability teams at Statutory Boards & Agencies firms operating under digital infrastructure environmental impact and data-centre energy reporting are increasingly using AI to extract OECD-cited data-centre energy statistics for ministerial briefings, populate official sustainability publications with verbatim OECD figures, draft policy position papers on digital-infrastructure environmental impact, and validate analytical references in signed government publications.

The OECD's 2025 Revision of the Recommendation on Digital Technologies and the Environment carries a named, citable statistic on Ireland's data-centre share of metered electricity, drawn from Ireland's Central Statistics Office, that ESG & Sustainability teams at statutory board and agency firms will reach for when populating sustainability disclosures, ESG investor responses, and regulatory briefings on digital-infrastructure environmental impact. That statistic is exactly the kind of figure the RLB Specialist Panel tested two frontier AI subjects against.

The RLB Specialist Panel issued a Specialist Panel application-style question on the share of Ireland's 2021 metered electricity that data centres accounted for, per the figure cited in the OECD Digital Economy Outlook 2024 chapter referenced by the 2025 Recommendation, sourced from Ireland's CSO (2023). Two frontier AI models tested by the RLB Specialist Panel returned the figure as 14 per cent and extended the answer with a four-point time series running from 5 per cent in 2015 through 21 per cent in 2023. The regulator's verbatim text records 11 per cent in 2021, with no multi-year trajectory.

The failure class is Fabricated Fact: a confidently delivered, citably attributed statistic that does not match the source document, compounded by a fabricated time series that does not appear anywhere in the OECD or CSO published record.

For ESG & Sustainability teams at statutory board and agency firms, this is operationally consequential because the wrong figure is not a vague paraphrase. It is delivered with a real source chain, CSO 2023 via OECD Digital Economy Outlook 2024, that survives standard reference-check review.

When an ESG and Sustainability team at a statutory board or agency asks AI tools to extract the data-centre energy consumption figure cited in the OECD's Recommendation, the AI returned 14 per cent, attributed by name to Ireland's Central Statistics Office (2023) and to the OECD Digital Economy Outlook 2024, when the actual figure in the text is 11 per cent. The AI compounded the error with a fabricated time series showing the share rising to 18 per cent in 2022 and 21 per cent in 2023, figures that do not exist in the source material.

If this response is used to populate a sustainability report, a ministerial briefing, or a policy position on digital infrastructure environmental impact, the statutory board documents wrong numbers attributed to a verifiable official source. The correction obligation that follows, amending a published government document, notifying the ministry, and re-examining any policy conclusions the figure was used to support, carries institutional reputational cost that is disproportionate to the original research shortcut.

The OECD has no direct enforcement powers over statutory boards and agencies under this recommendation, but the credibility damage from a publicly-visible factual error in an official sustainability publication is the operative risk.

The audit's finding on this question is published with an immutable RLB Citation ID. The relevant entry is RLB-H-INT-OECD-OECD-DIGITAL-TECHNOLOGIES-ENVIRONMENT-2025-Q006-Sonnet46. The full audit is published at the OECD Digital Technologies and the Environment Recommendation (2025 Revision) hub on RegLegBrief.com.

Sector: Software & SaaS and Dept: ESG & Sustainability INT OECD

Software & SaaS ESG & Sustainability teams: documentation and reporting gaps possible from AI reading of Recommendation of the Council on Digital Technologies and the Environment

For Software & SaaS ESG & Sustainability teams working with Recommendation of the Council on Digital Technologies and the Environment (2025 Revision): Specialist-Panel-verified findings on where AI summaries diverge...

ESG & Sustainability teams at Software & SaaS firms operating under digital infrastructure environmental impact and data-centre energy reporting are increasingly using AI to surface OECD data-centre energy benchmarks for sustainability-report sections, draft investor data room briefs on digital-infrastructure footprint, populate regulatory mapping documents with verbatim OECD statistics, and contextualise product sustainability claims against national data-centre energy trends.

The OECD's 2025 Revision of the Recommendation on Digital Technologies and the Environment carries a named, citable statistic on Ireland's data-centre share of metered electricity, drawn from Ireland's Central Statistics Office, that ESG & Sustainability teams at software and SaaS firms will reach for when populating sustainability disclosures, ESG investor responses, and regulatory briefings on digital-infrastructure environmental impact. That statistic is exactly the kind of figure the RLB Specialist Panel tested two frontier AI subjects against.

The RLB Specialist Panel issued a Specialist Panel application-style question on the share of Ireland's 2021 metered electricity that data centres accounted for, per the figure cited in the OECD Digital Economy Outlook 2024 chapter referenced by the 2025 Recommendation, sourced from Ireland's CSO (2023). Two frontier AI models tested by the RLB Specialist Panel returned the figure as 14 per cent and extended the answer with a four-point time series running from 5 per cent in 2015 through 21 per cent in 2023. The regulator's verbatim text records 11 per cent in 2021, with no multi-year trajectory.

The failure class is Fabricated Fact: a confidently delivered, citably attributed statistic that does not match the source document, compounded by a fabricated time series that does not appear anywhere in the OECD or CSO published record.

For ESG & Sustainability teams at software and SaaS firms, this is operationally consequential because the wrong figure is not a vague paraphrase. It is delivered with a real source chain, CSO 2023 via OECD Digital Economy Outlook 2024, that survives standard reference-check review.

When ESG teams at SaaS firms use AI tools to surface OECD data-centre energy benchmarks, the AI produced the wrong figure for Ireland's 2021 share of metered electricity, 14 per cent versus the correct 11 per cent per CSO 2023 as cited in OECD Digital Economy Outlook 2024, and fabricated a multi-year trajectory that does not exist in the source document. Any sustainability report section, investor data room brief, or regulatory mapping document that incorporates this figure carries a materially incorrect OECD statistic with correct-looking source attribution, a combination that passes standard QA review undetected.

For SaaS firms benchmarking their digital infrastructure footprint against OECD-cited sector data, or making product sustainability claims that reference national data-centre energy trends, the downstream exposure includes audit findings, investor relations issues, and reputational damage if the discrepancy surfaces through primary source verification by a third party.

The audit's finding on this question is published with an immutable RLB Citation ID. The relevant entry is RLB-H-INT-OECD-OECD-DIGITAL-TECHNOLOGIES-ENVIRONMENT-2025-Q006-Sonnet46. The full audit is published at the OECD Digital Technologies and the Environment Recommendation (2025 Revision) hub on RegLegBrief.com.

Sector: Management & Risk Consulting and Dept: ESG & Sustainability INT OECD

Management & Risk Consulting ESG & Sustainability teams: documentation and reporting gaps possible from AI reading of Recommendation of the Council on Digital Technologies and the Environment

For Management & Risk Consulting ESG & Sustainability teams working with Recommendation of the Council on Digital Technologies and the Environment (2025 Revision): Specialist-Panel-verified findings on where AI...

ESG & Sustainability teams at Management & Risk Consulting firms operating under digital infrastructure environmental impact and data-centre energy reporting are increasingly using AI to draft client-facing ESG benchmarking sections referencing OECD-cited national data, populate regulatory gap-analysis deliverables with verbatim OECD statistics, prepare client strategy documents that use OECD trend data for forward projection, and validate benchmark citations in signed consulting deliverables.

The OECD's 2025 Revision of the Recommendation on Digital Technologies and the Environment carries a named, citable statistic on Ireland's data-centre share of metered electricity, drawn from Ireland's Central Statistics Office, that ESG & Sustainability teams at management and risk consulting firms will reach for when populating sustainability disclosures, ESG investor responses, and regulatory briefings on digital-infrastructure environmental impact. That statistic is exactly the kind of figure the RLB Specialist Panel tested two frontier AI subjects against.

The RLB Specialist Panel issued a Specialist Panel application-style question on the share of Ireland's 2021 metered electricity that data centres accounted for, per the figure cited in the OECD Digital Economy Outlook 2024 chapter referenced by the 2025 Recommendation, sourced from Ireland's CSO (2023). Two frontier AI models tested by the RLB Specialist Panel returned the figure as 14 per cent and extended the answer with a four-point time series running from 5 per cent in 2015 through 21 per cent in 2023. The regulator's verbatim text records 11 per cent in 2021, with no multi-year trajectory.

The failure class is Fabricated Fact: a confidently delivered, citably attributed statistic that does not match the source document, compounded by a fabricated time series that does not appear anywhere in the OECD or CSO published record.

For ESG & Sustainability teams at management and risk consulting firms, this is operationally consequential because the wrong figure is not a vague paraphrase. It is delivered with a real source chain, CSO 2023 via OECD Digital Economy Outlook 2024, that survives standard reference-check review. An ESG and Sustainability team using AI tools to retrieve Ireland's 2021 data-centre electricity intensity figure for a client deliverable will receive 14 per cent, not the 11 per cent the OECD text actually cites from Ireland's Central Statistics Office (2023).

The AI additionally fabricates a multi-year time series, spanning 2015 to 2023, that does not appear in the primary document, giving the wrong anchor number a false analytical foundation that could underpin trend analysis or forward-projection work in client strategy documents. The immediate exposure is a factually incorrect statistic in a client-facing report or regulatory gap analysis, attributed to a named and verifiable OECD/CSO source. Because the citation chain is structurally real, the error survives casual review and surfaces only when a client's investor relations team, internal audit function, or external ESG assurance provider checks the primary OECD text.

For a consulting firm, that discovery, a wrong number in a signed-off deliverable, is an engagement-quality failure with direct consequences for client retention and the firm's reputational standing on future mandates.

The audit's finding on this question is published with an immutable RLB Citation ID. The relevant entry is RLB-H-INT-OECD-OECD-DIGITAL-TECHNOLOGIES-ENVIRONMENT-2025-Q006-Sonnet46. The full audit is published at the OECD Digital Technologies and the Environment Recommendation (2025 Revision) hub on RegLegBrief.com.

↑ Back to top