AI Hallucination Research › Briefings

Briefings Blog

The running blog from the RLB Specialist Panel delves into real-world scenarios where the compliance, legal, or AI lab team interacts with frontier AI models under specific regulations. The blogs are anonymised to remove client-specific details and include insights from the RLB team analysing the hallucinations experienced in AI models while working on these cases. For example, when a model returns a confident answer that contradicts the regulator's primary text, such as a fabricated staff letter, a wrong appendix, or an inverted scope, these issues are discussed here. Each blog explains one set of findings and what it would have meant for the team that would have acted on it, sans this research initiative. This blog is frequently updated, a few times a day.

263 briefings in the archive · Subscribe via Atom: /briefings/feed.xml (this blog) · /feed.xml (all RegLegBrief publications)
Audience colours: AI Labs Practitioner (profession) Sector × Department
Audience
Jur.
Regulator
Profession
Sector
Dept
Range
Sort
Per page
Showing 5 of 263 · page 23 of 53
Thursday, 09 July 2026
Sector: Law Firms and Dept: Legal US CFTC

Law Firms Legal teams: documentation and reporting gaps possible from AI reading of CFTC Swap Dealer Business Conduct & Documentation (2025)

For Law Firms Legal teams working with Revisions to Business Conduct and Swap Documentation Requirements for Swap Dealers and Major Swap Participants: Specialist-Panel-verified findings on where AI summaries diverge...

Legal teams at US Law Firms advising swap dealer clients on the December 2025 CFTC final rule are increasingly using AI to draft client alerts and regulatory memos on the External Business Conduct Standards, generate partner-level briefings on the January 2026 correction notice, prepare cross-border execution opinions on the CFTC's staff no-action letter regime, and validate threshold language and venue scope claims against the published rule. The same tools are used to draft sign-off letters for swap dealer clients on §§ 23.431, 23.434, and 23.440 amendments.

Two frontier AI models tested by the RLB Specialist Panel on the workflows law firm legal teams actually use AI for on the December 2025 CFTC final rule on swap dealer business conduct and documentation produced three discrete hallucinations bound to verbatim regulator-issued source text. The Panel records two distinct failure classes, Exposed Fabrication and Inference Drift across the set. Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows law firm legal teams use AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For Legal teams at Law Firms, each hallucination has a direct read-through into the client alert, regulatory memorandum, partner-level briefing, or sign-off letter on swap dealer business conduct compliance. The Panel's testing surfaces the January 2026 correction notice and the identity of the restored appendix, CFTC Staff Letter 25-49's trading venue scope, misidentified as US SEFs and DCMs rather than eligible UK trading venues, and the PTMMM elimination scope, overstated to include cleared CDS where the prior provision had never applied to cleared swaps.

Where these errors flow into a deliverable, the exposure is PI exposure, an inaccurate regulatory advice trail that enters the client's audit record, and a discoverable error in advice that propagates to multiple swap dealer counterparties.

The Specialist Panel records the citation IDs as follows: RLB-H-US-CFTC-SWAP-DEALER-BUSINESS-CONDUCT-DOCUMENTATION-2025-Q002-Opus47 (Claude Opus 4.7 (web search on), Inference Drift); RLB-H-US-CFTC-SWAP-DEALER-BUSINESS-CONDUCT-DOCUMENTATION-2025-Q003-Opus47 (Claude Opus 4.7 (web search on), Exposed Fabrication); RLB-H-US-CFTC-SWAP-DEALER-BUSINESS-CONDUCT-DOCUMENTATION-2025-Q004-Opus47 (Claude Opus 4.7 (web search on), Exposed Fabrication). Each citation links to the verbatim regulator-issued source text, the tested AI question, and the recorded AI response, so the Panel's assessment is traceable end to end. The full audit is published at the the CFTC swap dealer business conduct and documentation hub on RegLegBrief.com.

Sector: Investment Banking and Dept: Legal US CFTC

Investment Banking Legal teams: documentation and reporting gaps possible from AI reading of CFTC Swap Dealer Business Conduct & Documentation (2025)

For Investment Banking Legal teams working with Revisions to Business Conduct and Swap Documentation Requirements for Swap Dealers and Major Swap Participants: Specialist-Panel-verified findings on where AI summaries...

Legal teams at Investment Banking firms with swap dealer registration under the December 2025 CFTC final rule are increasingly using AI to draft swap documentation playbooks for the rates, credit, and FX derivatives desks, generate counterparty advisories on the External Business Conduct Standards, prepare regulatory opinions on cross-border execution and Special Entity recommendations, and validate threshold language and staff no-action letter scope against the published rule. The same tools are used to draft client alerts on the January 2026 correction notice and to brief the GC on §§ 23.431, 23.434, and 23.440 amendments.

Two frontier AI models tested by the RLB Specialist Panel on the workflows investment-banking legal teams actually use AI for on the December 2025 CFTC final rule on swap dealer business conduct and documentation produced three discrete hallucinations bound to verbatim regulator-issued source text. The Panel records two distinct failure classes, Exposed Fabrication and Inference Drift across the set. Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows investment-banking legal teams use AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For Legal teams at Investment Banking firms, each hallucination has a direct read-through into the swap documentation playbook, regulatory opinion, counterparty advisory, or GC briefing on swap dealer business conduct. The Panel's testing surfaces the January 2026 correction notice and the identity of the restored appendix, CFTC Staff Letter 25-49's trading venue scope, misidentified as US SEFs and DCMs rather than eligible UK trading venues, and the PTMMM elimination scope, overstated to include cleared CDS where the prior provision had never applied to cleared swaps.

Where these errors flow into a deliverable, the exposure is regulatory opinion error that propagates across the firm's counterparty documentation, examination posture exposure on cross-border execution, and documentation gaps that surface only under CFTC review.

The Specialist Panel records the citation IDs as follows: RLB-H-US-CFTC-SWAP-DEALER-BUSINESS-CONDUCT-DOCUMENTATION-2025-Q002-Opus47 (Claude Opus 4.7 (web search on), Inference Drift); RLB-H-US-CFTC-SWAP-DEALER-BUSINESS-CONDUCT-DOCUMENTATION-2025-Q003-Opus47 (Claude Opus 4.7 (web search on), Exposed Fabrication); RLB-H-US-CFTC-SWAP-DEALER-BUSINESS-CONDUCT-DOCUMENTATION-2025-Q004-Opus47 (Claude Opus 4.7 (web search on), Exposed Fabrication). Each citation links to the verbatim regulator-issued source text, the tested AI question, and the recorded AI response, so the Panel's assessment is traceable end to end. The full audit is published at the the CFTC swap dealer business conduct and documentation hub on RegLegBrief.com.

Sector: Investment Banking and Dept: Compliance US CFTC

Investment Banking Compliance teams: documentation and reporting gaps possible from AI reading of CFTC Swap Dealer Business Conduct & Documentation (2025)

For Investment Banking Compliance teams working with Revisions to Business Conduct and Swap Documentation Requirements for Swap Dealers and Major Swap Participants: Specialist-Panel-verified findings on where AI...

Compliance teams at Investment Banking firms operating swap dealer franchises under the December 2025 CFTC final rule are increasingly using AI to update written supervisory procedures on the External Business Conduct Standards, generate trade-monitoring rule-update bulletins for the rates, credit, and FX derivatives desks, validate threshold language and venue-scope claims for ITBC swap counterparty disclosure, and prepare CCO briefings on the §§ 23.431 and 23.434 amendments. The same tools are used to map staff no-action letters and the January 2026 correction notice into standing policy text ahead of CFTC examination cycles.

Two frontier AI models tested by the RLB Specialist Panel on the workflows investment-banking compliance officers actually use AI for on the December 2025 CFTC final rule on swap dealer business conduct and documentation produced three discrete hallucinations bound to verbatim regulator-issued source text. The Panel records two distinct failure classes, Exposed Fabrication and Inference Drift across the set. Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows investment-banking compliance officers use AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For Compliance teams at Investment Banking firms, each hallucination has a direct read-through into the written supervisory procedure, compliance attestation, counterparty communication template, or audit walkthrough narrative on swap dealer business conduct. The Panel's testing surfaces the January 2026 correction notice and the identity of the restored appendix, CFTC Staff Letter 25-49's trading venue scope, misidentified as US SEFs and DCMs rather than eligible UK trading venues, and the PTMMM elimination scope, overstated to include cleared CDS where the prior provision had never applied to cleared swaps.

Where these errors flow into a deliverable, the exposure is CFTC examination findings, remediation across desk-level procedures and training materials, and supervisory exposure on a multi-product swap dealer book where documentation gaps cascade into desk-level execution practice.

The Specialist Panel records the citation IDs as follows: RLB-H-US-CFTC-SWAP-DEALER-BUSINESS-CONDUCT-DOCUMENTATION-2025-Q002-Opus47 (Claude Opus 4.7 (web search on), Inference Drift); RLB-H-US-CFTC-SWAP-DEALER-BUSINESS-CONDUCT-DOCUMENTATION-2025-Q003-Opus47 (Claude Opus 4.7 (web search on), Exposed Fabrication); RLB-H-US-CFTC-SWAP-DEALER-BUSINESS-CONDUCT-DOCUMENTATION-2025-Q004-Opus47 (Claude Opus 4.7 (web search on), Exposed Fabrication). Each citation links to the verbatim regulator-issued source text, the tested AI question, and the recorded AI response, so the Panel's assessment is traceable end to end. The full audit is published at the the CFTC swap dealer business conduct and documentation hub on RegLegBrief.com.

Practitioner: Lawyers US CFTC

Lawyers: AI summaries of CFTC Swap Dealer Business Conduct & Documentation (2025) may understate professional obligations

For Lawyers working with Revisions to Business Conduct and Swap Documentation Requirements for Swap Dealers and Major Swap Participants: where Specialist-Panel-verified divergences between frontier AI summaries and...

Lawyers advising on the December 2025 CFTC swap dealer business conduct and documentation rulemaking are increasingly using AI to draft 2-page board memos on amendment scope, generate client-facing investor-eligibility summaries on the External Business Conduct Standards, prepare partner-level briefings on the January 2026 correction notice, and validate threshold language and venue-scope claims against the published rule. The same tools are used to summarise CFTC staff letters for cross-border swap dealer clients and to track how the rule reshapes pre-trade mid-market mark disclosure obligations across cleared and uncleared swap product books.

Two frontier AI models tested by the RLB Specialist Panel on the workflows lawyers actually use AI for on the December 2025 CFTC final rule on swap dealer business conduct and documentation produced three discrete hallucinations bound to verbatim regulator-issued source text. The Panel records two distinct failure classes, Exposed Fabrication and Inference Drift across the set. Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows lawyers use AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For Lawyers, each hallucination has a direct read-through into the regulatory opinion, partner-level memorandum, client alert, or sign-off letter on swap dealer business conduct compliance. The Panel's testing surfaces the January 2026 correction notice and the identity of the restored appendix, CFTC Staff Letter 25-49's trading venue scope, misidentified as US SEFs and DCMs rather than eligible UK trading venues, and the PTMMM elimination scope, overstated to include cleared CDS where the prior provision had never applied to cleared swaps.

Where these errors flow into a deliverable, the exposure is PI exposure, client correction, and a discoverable error in opinion drafts and client advisories that propagate to multiple swap dealer counterparties.

The Specialist Panel records the citation IDs as follows: RLB-H-US-CFTC-SWAP-DEALER-BUSINESS-CONDUCT-DOCUMENTATION-2025-Q002-Opus47 (Claude Opus 4.7 (web search on), Inference Drift); RLB-H-US-CFTC-SWAP-DEALER-BUSINESS-CONDUCT-DOCUMENTATION-2025-Q003-Opus47 (Claude Opus 4.7 (web search on), Exposed Fabrication); RLB-H-US-CFTC-SWAP-DEALER-BUSINESS-CONDUCT-DOCUMENTATION-2025-Q004-Opus47 (Claude Opus 4.7 (web search on), Exposed Fabrication). Each citation links to the verbatim regulator-issued source text, the tested AI question, and the recorded AI response, so the Panel's assessment is traceable end to end. The full audit is published at the the CFTC swap dealer business conduct and documentation hub on RegLegBrief.com.

Sector: Hedge Funds and Dept: Operations US CFTC

Hedge Funds Operations teams: documentation and reporting gaps possible from AI reading of CFTC Regulation 1.44 (Margin Adequacy + Separate Accounts)

For Hedge Funds Operations teams working with Regulations to Address Margin Adequacy and to Account for the Treatment of Separate Accounts by Futures Commission Merchants (17 CFR § 1.44): Specialist-Panel-verified...

Hedge fund operations teams running multi-currency client accounts cleared through Futures Commission Merchants are increasingly using AI to configure margin processing system parameters, generate end-of-day reconciliation rule sets, produce CFTC counterparty deadline reference cards for treasury staff, validate FCM margin call timing against the firm's internal monitoring thresholds, and draft operations procedure documentation for new currency pairs. CFTC Regulation 1.44 (17 CFR Section 1.44) governs margin adequacy and the treatment of separate accounts by FCMs, and its three-tier currency deadline schedule defines the timing parameters that every operations system supporting an FCM relationship must reflect correctly.

Two frontier AI models tested by the RLB Specialist Panel produced Regulation 1.44 currency deadline output that contradicts the rule on the exact operational parameters operations teams configure their systems against. The RLB Specialist Panel classes the failure pattern as Enumeration Collapse: the models reconstructed the regulation's three-tier deadline structure from intuitive priors rather than from Section 1.44(f) verbatim. One model collapsed three tiers into two, assigning Appendix A currencies T+1 when the rule requires T+2. The second model added an intraday Eastern Time cutoff to the T+1 default tier that does not appear in the rule.

Both AI subjects answered the operations brief with web search enabled, mirroring how operations and treasury teams at hedge funds actually use AI assistants when setting up a new FCM counterparty or onboarding a new currency pair; the failure pattern surfaced regardless of the retrieval pathway. The Specialist Panel binds each finding to the verbatim eCFR text of Section 1.44 and Appendix A held as primary substrate, and records the failure mode classifications (outdated for the Opus 4.7 finding, inference_drift for the Sonnet 4.6 finding) against that primary substrate document.

The same Enumeration Collapse pattern surfaced on a parallel Regulation 1.44 probe testing the rule's cessation triggers, indicating that AI-assisted parameter generation on any enumerated list in this rule, currency lists, cessation triggers, deadline buckets, requires the same verification discipline.

For a hedge fund operations team, the exposure is systemic. System-level parameter errors propagate into transaction records, reconciliation outputs, and audit trails before any review touches them. A margin processing system configured against the compressed two-tier output would generate T+1 deadline expectations for Appendix A currencies and flag T+2 receipts as breaches, surfacing false-positive disputes with the FCM on every Appendix A call. A system configured against the noon cutoff would treat afternoon T+1 receipts as late on non-Appendix-A currencies and document a regulatory basis the CFTC has not provided.

Either error carries through to month-end reconciliation, to the operations review pack circulated to the COO, and to any examination response that references the firm's margin monitoring posture.

The findings carry citation IDs RLB-H-US-CFTC-FCM-MARGIN-ADEQUACY-SEPARATE-ACCOUNTS-REG-1-44-Q001-Opus47 and RLB-H-US-CFTC-FCM-MARGIN-ADEQUACY-SEPARATE-ACCOUNTS-REG-1-44-Q001-Sonnet46. Citation ID RLB-H-...-Q001-Opus47 records the compressed two-tier reconstruction and is classed as outdated against the eCFR-archived primary text. Citation ID RLB-H-...-Q001-Sonnet46 records the fabricated noon cutoff and is classed as inference_drift against the same primary text.

↑ Back to top