AI Hallucination Research › Briefings

Briefings Blog

The running blog from the RLB Specialist Panel delves into real-world scenarios where the compliance, legal, or AI lab team interacts with frontier AI models under specific regulations. The blogs are anonymised to remove client-specific details and include insights from the RLB team analysing the hallucinations experienced in AI models while working on these cases. For example, when a model returns a confident answer that contradicts the regulator's primary text, such as a fabricated staff letter, a wrong appendix, or an inverted scope, these issues are discussed here. Each blog explains one set of findings and what it would have meant for the team that would have acted on it, sans this research initiative. This blog is frequently updated, a few times a day.

263 briefings in the archive · Subscribe via Atom: /briefings/feed.xml (this blog) · /feed.xml (all RegLegBrief publications)
Audience colours: AI Labs Practitioner (profession) Sector × Department
Audience
Jur.
Regulator
Profession
Sector
Dept
Range
Sort
Per page
Showing 5 of 263 · page 24 of 53
Thursday, 09 July 2026
Sector: Investment Banking and Dept: Legal US CFTC

Investment Banking Legal teams: documentation and reporting gaps possible from AI reading of CFTC Regulation 1.44 (Margin Adequacy + Separate Accounts)

For Investment Banking Legal teams working with Regulations to Address Margin Adequacy and to Account for the Treatment of Separate Accounts by Futures Commission Merchants (17 CFR § 1.44): Specialist-Panel-verified...

Investment bank legal teams are increasingly using AI to review FCM customer agreements and margin schedules, validate client-disclosure language against CFTC rules, generate diligence summaries on broker-dealer counterparty risk, prepare transaction-side memos on margin operational requirements, draft 2-page MD/desk briefings on regulatory changes affecting margin processing, and produce comparison tables between the desk's documented procedures and the regulator's text.

CFTC Regulation 1.44 (17 CFR Section 1.44), the rule governing margin adequacy and separate account treatment by Futures Commission Merchants, sits at the centre of that workflow because its three-tier currency deadline schedule defines when each side of a multi-currency margin call is expected to settle.

Two frontier AI models tested by the RLB Specialist Panel produced Regulation 1.44 currency deadline output that contradicts the rule. The RLB Specialist Panel classes the failure pattern as Enumeration Collapse: the models reconstructed the regulation's three-tier currency deadline structure from intuitive priors rather than from the verbatim Section 1.44(f) text. One model compressed three tiers into two, assigning Appendix A currencies a T+1 deadline when the rule sets T+2. The second model added a noon Eastern Time cutoff to the T+1 default tier that does not appear anywhere in the rule.

Both AI subjects answered the operational brief with web search enabled, mirroring how transaction-side legal teams actually use AI assistants under deal-timeline pressure; the failure pattern surfaced regardless of the retrieval pathway. The Specialist Panel binds each finding to the verbatim eCFR text of Section 1.44 and Appendix A held as primary substrate, and records the failure mode classifications (outdated for the Opus 4.7 finding, inference_drift for the Sonnet 4.6 finding) against that primary substrate document.

The same Enumeration Collapse pattern surfaced on a parallel Regulation 1.44 probe testing the rule's cessation triggers, indicating the failure is structural across the regulation's enumerated lists rather than confined to one currency-deadline question.

For an investment bank legal team, the work-product impact runs through the daily flow of transaction-side documentation. A diligence summary on an FCM counterparty built off the compressed two-tier reconstruction would treat Appendix A margin received on T+2 as a late call, mis-stating the counterparty's compliance posture. A client disclosure validated against the noon cutoff would commit the desk to a standard the CFTC did not set.

A 2-page MD briefing repeating either output as guidance for the desk would seed an error into the bank's documented internal view of the rule that travels into examination responses, internal audit findings, and counterparty risk reports.

The findings carry citation IDs RLB-H-US-CFTC-FCM-MARGIN-ADEQUACY-SEPARATE-ACCOUNTS-REG-1-44-Q001-Opus47 and RLB-H-US-CFTC-FCM-MARGIN-ADEQUACY-SEPARATE-ACCOUNTS-REG-1-44-Q001-Sonnet46. Citation ID RLB-H-...-Q001-Opus47 records the compressed two-tier reconstruction and is classed as outdated against the eCFR-archived primary text. Citation ID RLB-H-...-Q001-Sonnet46 records the fabricated noon cutoff and is classed as inference_drift against the same primary text.

Wednesday, 08 July 2026
Sector: Law Firms and Dept: Legal US CFTC

Law Firms Legal teams: documentation and reporting gaps possible from AI reading of CFTC Regulation 1.44 (Margin Adequacy + Separate Accounts)

For Law Firms Legal teams working with Regulations to Address Margin Adequacy and to Account for the Treatment of Separate Accounts by Futures Commission Merchants (17 CFR § 1.44): Specialist-Panel-verified findings...

Lawyers at law firms advising FCMs, hedge fund clients, investment bank counterparty desks, and commodity pool operators on CFTC Regulation 1.44 are increasingly using AI to draft 2-page client memos on margin call timing, generate partner-level briefings on the rule's separate account treatment, prepare board-meeting summaries on FCM counterparty risk, validate threshold language in margin agreements against the published rule, and produce comparison tables between the CFTC text and law-firm interpretive guidance.

Regulation 1.44 (17 CFR Section 1.44) governs how FCMs margin and segregate customer assets in separate accounts, and its three-tier currency deadline schedule sits inside almost every deliverable a firm produces on the rule.

Two frontier AI models tested by the RLB Specialist Panel produced Regulation 1.44 currency deadline guidance that contradicts the rule's text in two distinct ways. The RLB Specialist Panel classes the pattern as Enumeration Collapse: the models reconstructed Section 1.44(f) from intuitive priors and pre-finalisation third-party summaries rather than from the regulation as enacted. One model collapsed the rule's three currency deadline tiers into two, dropping the T+2 Appendix A tier entirely and assigning those currencies T+1. The second model added an intraday Eastern Time cutoff to the T+1 default tier that does not exist in the rule.

Both AI subjects answered with web search enabled, mirroring how associates and counsel at law firms actually use AI assistants on a finalised rule under client time pressure; the failure pattern surfaced regardless of the retrieval pathway. The Specialist Panel binds each finding to the verbatim eCFR text of Section 1.44 and Appendix A held as primary substrate, and records the failure mode classifications (outdated for the Opus 4.7 finding, inference_drift for the Sonnet 4.6 finding) against that primary substrate document.

The same Enumeration Collapse pattern surfaced on a parallel Regulation 1.44 probe testing the rule's cessation triggers, indicating the failure is structural rather than incidental to the currency-deadline question and would surface across any deliverable that asks the model to reconstruct a regulation's enumerated lists.

For a law firm, the work-product exposure runs through every Regulation 1.44 deliverable the firm signs out under its name. A 2-page client memo drafted off the compressed two-tier reconstruction would direct an FCM client to collect Appendix A margin one full business day earlier than the rule requires, creating a documented internal procedure the client can later be examined against.

A partner-level briefing repeating the noon cutoff would cite a specific intraday time with no regulatory basis, an error opposing counsel could surface in a margin dispute or that an examiner could pursue if the client adopted it as policy. A comparison table generated against the AI output would propagate either error across the firm's knowledge base and into future deliverables.

The findings carry citation IDs RLB-H-US-CFTC-FCM-MARGIN-ADEQUACY-SEPARATE-ACCOUNTS-REG-1-44-Q001-Opus47 and RLB-H-US-CFTC-FCM-MARGIN-ADEQUACY-SEPARATE-ACCOUNTS-REG-1-44-Q001-Sonnet46. Citation ID RLB-H-...-Q001-Opus47 records the compressed two-tier reconstruction and is classed as outdated against the eCFR-archived primary text. Citation ID RLB-H-...-Q001-Sonnet46 records the fabricated noon cutoff and is classed as inference_drift against the same primary text.

Practitioner: Stockbrokers / Trading Reps US CFTC

Stockbrokers / Trading Reps: AI summaries of CFTC Regulation 1.44 (Margin Adequacy + Separate Accounts) may understate professional obligations

For Stockbrokers / Trading Reps working with Regulations to Address Margin Adequacy and to Account for the Treatment of Separate Accounts by Futures Commission Merchants (17 CFR § 1.44): where...

Stockbrokers and trading representatives advising clients on accounts cleared through Futures Commission Merchants are increasingly using AI to draft account-opening disclosures, prepare margin-procedure summaries for sophisticated clients, generate monitoring memos on multi-currency exposure, validate threshold language in client agreements, and produce internal training notes on CFTC margin call timing. Regulation 1.44, the CFTC rule governing margin adequacy and the treatment of separate accounts by FCMs (17 CFR Section 1.44), sits at the centre of that workflow because its currency deadline tiers determine when each side of a multi-currency account is expected to settle a margin call.

Two frontier AI models tested by the RLB Specialist Panel produced operational Regulation 1.44 deadline guidance that contradicts the rule. The RLB Specialist Panel classes the failure pattern as Enumeration Collapse: the models reconstructed the regulation's three-tier currency deadline structure from intuitive priors rather than from the verbatim text of Section 1.44(f) and Appendix A. One model compressed three tiers into two, assigning Appendix A currencies a T+1 deadline when the regulation requires T+2. The second model added a noon Eastern Time cutoff to the T+1 tier that does not appear anywhere in the rule.

Both AI subjects answered the brief with web search enabled, mirroring how trading desks actually run finalised-rule queries today; the failure pattern surfaced regardless of the retrieval pathway. The Specialist Panel binds each finding to the verbatim eCFR text of Section 1.44 and Appendix A held as primary substrate, and records the failure mode classifications (outdated for the Opus 4.7 finding, inference_drift for the Sonnet 4.6 finding) against that primary substrate document.

The same Enumeration Collapse pattern surfaced on a parallel Regulation 1.44 probe testing the rule's cessation triggers, suggesting the failure is structural rather than incidental to the currency-deadline question.

For a trading representative working multi-currency client accounts at an FCM, the operational consequence runs through every deliverable that references margin timing. A monitoring memo built from the compressed two-tier output will flag Appendix A margin received on T+2 as a late call when the regulation considers it timely. A client-facing summary incorporating the noon cutoff will assert a regulatory deadline the CFTC never imposed, exposing the representative and the firm to a documented standard that exceeds the rule and that an examiner or opposing counsel could challenge.

A training note that propagates either error into the desk's standard operating procedure compounds the exposure across every multi-currency margin call processed against the wrong parameter.

The findings carry citation IDs RLB-H-US-CFTC-FCM-MARGIN-ADEQUACY-SEPARATE-ACCOUNTS-REG-1-44-Q001-Opus47 and RLB-H-US-CFTC-FCM-MARGIN-ADEQUACY-SEPARATE-ACCOUNTS-REG-1-44-Q001-Sonnet46. Citation ID RLB-H-...-Q001-Opus47 records the compressed two-tier reconstruction and is classed as outdated against the eCFR-archived primary text. Citation ID RLB-H-...-Q001-Sonnet46 records the fabricated noon cutoff and is classed as inference_drift against the same primary text.

Sector: Law Firms and Dept: Legal US CFTC

Law Firms Legal teams: documentation and reporting gaps possible from AI reading of CFTC Digital Asset Collateral & Tokenized Assets Staff Guidance (2025)

For Law Firms Legal teams working with CFTC Digital Asset Collateral No-Action Relief and Tokenized Asset Staff Guidance (Market Participants Division, December 2025): Specialist-Panel-verified findings on where AI...

Law firms advising FCMs, payment stablecoin issuers, and DCOs on the CFTC Digital Asset Collateral Framework are increasingly using AI to draft client memos on payment stablecoin eligibility, generate partner-level briefings on the phased onboarding obligation map, and validate staff-letter citation language before issuing opinions on customer margin collateral acceptance and haircut methodology.

The RLB Specialist Panel put a set of practitioner-grade questions on the CFTC Digital Asset Collateral Framework to two frontier AI models with web search active. Each question is prepared by the Panel based on the workflows that legal teams at law firms firms actually use AI for under the Market Participants Division's December 2025 staff letter, as amended by Staff Letter 26-05. The Panel then binds every AI response to verbatim regulator-issued source text held as primary substrate.

On the CFTC Digital Asset Collateral Framework, the AI subjects returned a single hallucinated answer for legal teams at law firms firms, in the form of Inverted-Position Fabrication.

For legal teams at law firms firms advising on the CFTC Digital Asset Collateral Framework, staff-letter citation accuracy is load-bearing in eligibility opinions, FCM customer-onboarding memos, payment stablecoin issuer due-diligence, and any regulator-facing position paper engaging the framework. A counterparty or examiner who identifies a missing OCC 1183 cross-reference, an inverted weekly reporting characterisation, or a base-floor substitute for the multi-DCO haircut rule on first reading calls the entire piece of advice into question.

The weekly reporting inversion is the most serious failure: a legal opinion structured around a sunset that the regulator explicitly continues produces an ongoing reporting violation for the FCM client and exposes the firm to professional liability when the underlying position is later corrected.

The published Specialist Panel findings carry the following citation identifiers:

Sector: Corporate Banking and Dept: Legal US CFTC

Corporate Banking Legal teams: documentation and reporting gaps possible from AI reading of CFTC Digital Asset Collateral & Tokenized Assets Staff Guidance (2025)

For Corporate Banking Legal teams working with CFTC Digital Asset Collateral No-Action Relief and Tokenized Asset Staff Guidance (Market Participants Division, December 2025): Specialist-Panel-verified findings on...

Legal teams at corporate banks advising FCM clients on the CFTC Digital Asset Collateral Framework are increasingly using AI to draft counsel-facing memos on the post-onboarding obligation set, generate client briefings on the weekly reporting cadence, and validate staff-letter citation language in transactional documents and counterparty correspondence.

The RLB Specialist Panel put a set of practitioner-grade questions on the CFTC Digital Asset Collateral Framework to two frontier AI models with web search active. Each question is prepared by the Panel based on the workflows that legal teams at corporate banking firms actually use AI for under the Market Participants Division's December 2025 staff letter, as amended by Staff Letter 26-05. The Panel then binds every AI response to verbatim regulator-issued source text held as primary substrate.

On the CFTC Digital Asset Collateral Framework, the AI subjects returned a single hallucinated answer for legal teams at corporate banking firms, in the form of Inverted-Position Fabrication.

For legal teams at corporate banking firms advising on the CFTC Digital Asset Collateral Framework, staff-letter citation accuracy is load-bearing in eligibility opinions, FCM customer-onboarding memos, payment stablecoin issuer due-diligence, and any regulator-facing position paper engaging the framework. A counterparty or examiner who identifies a missing OCC 1183 cross-reference, an inverted weekly reporting characterisation, or a base-floor substitute for the multi-DCO haircut rule on first reading calls the entire piece of advice into question.

The weekly reporting inversion is the most serious failure: a legal opinion structured around a sunset that the regulator explicitly continues produces an ongoing reporting violation for the FCM client and exposes the firm to professional liability when the underlying position is later corrected.

The published Specialist Panel findings carry the following citation identifiers:

↑ Back to top