Executive Summary
Across five aggregated findings on the CFTC's 2024 amendments to Regulation 1.25, Claude Opus 4.7 and Claude Sonnet 4.6 (each with web search active) produced confident wrong reconstructions of the rule's three operative pillars: the size-triggered 50 per cent concentration ceiling, the 24-month portfolio dollar-weighted average maturity (DWAM) standard, and the March 31, 2025 SIDR compliance anchor. Every failure is classified as inference_drift against substrate covering 17 CFR 1.25(b)(3)(ii), 17 CFR 1.25(b)(3)(iv), and 7 USC 6d.
For Risk teams at Investment Banking firms in the United States, the failure surface lands directly on the workflows where AI assistance is most attractive: drafting investment policy statements, scoping concentration testing, and building post-amendment SIDR schedules.
How AI gets this regulation wrong
The dominant failure pattern is threshold-trigger elision and carve-out inversion: models surface one axis of a multi-condition rule correctly while dropping the axes that actually govern, substitute adjacent asset classes for narrow exclusion sets, and drift from a date-certain anchor into a generic relative range. Across all five findings, web search did not surface the rule's actual numeric structure; the models defaulted to a plausible-but-wrong schema and returned it as the rule.
| AI's Failure Mode | Count | Affected findings |
|---|---|---|
| 0 | ||
| 0 | ||
| 0 | ||
| 0 | ||
| 0 |
What that means for your team
For Risk teams at Investment Banking firms, every finding in this cell carries regulatory enforcement exposure. The three pillars of the 2024 amendments (concentration ceiling, DWAM standard, SIDR anchor) are the three provisions a Risk team is most likely to be asked to confirm, implement, or test. When the AI output shaping the policy, the testing playbook, or the compliance calendar carries dropped triggers, an inverted carve-out, or a drifted date, the deficiency embeds in the firm's regulated artefacts: investment policy statements, segregation reports, SIDR filings, and customer risk disclosures.
| Risk Impact | Count | Affected findings |
|---|---|---|
| 0 | ||
| 0 | ||
| 0 | ||
| 0 | ||
| 0 |
When this affects your department
Risk teams at Investment Banking firms encounter Regulation 1.25 most often when the firm is structuring an FCM's or DCO's customer-funds investment policy, scoping segregated-fund concentration testing against the amended ceilings, scheduling SIDR and risk-disclosure updates against the compliance calendar, or advising on hedge-fund counterparty due-diligence against the post-amendment framework. AI tools sit naturally in the early framing of each of these workflows: drafting the policy outline, identifying applicable thresholds, building the testing checklist, or summarising the calendar.
Where AI output is used in this way, the risk is that a confident wrong answer becomes the working assumption that frames all subsequent work.
The specific failures documented here show that AI is unreliable precisely where the team needs precision. On the concentration question, both models negated the FCM-size axis (correctly) while dropping the fund-AUM and management-company-AUM axes (incorrectly): a team that reads the answer as adjudicating the size question would set the firm's concentration ceiling against the wrong governing structure. On the DWAM question, one model swapped Treasury repos into the carve-out set in place of the regulator's actual exclusion list; another model returned a no-standard answer for direct Treasuries, where the 24-month standard governs by default.
On the SIDR anchor, the model drifted to a generic relative range when the regulator documents a specific date.
The dollar exposure scales with the firm's segregated customer-funds book. An FCM with several billion in segregated assets that miscalibrates the concentration ceiling, mis-scopes DWAM testing, or misses the SIDR compliance anchor carries a regulatory examination footprint commensurate with the size of the book. For a team using AI as the first-pass research layer, the safeguard is not to abandon AI but to treat every numeric and date-specific output as a hypothesis to be verified against the Federal Register final rule text before it enters any deliverable.
The findings at a glance
The table below summarises each finding: question area, error type, and the citation reference.
Aggregate impact
The five findings cluster tightly on the three pillars of the 2024 amendments, which is what makes this regulation a high-yield failure surface for Risk teams. Pre-amendment Regulation 1.25 had a flatter concentration framework, an embedded carve-out set that secondary commentary often skipped, and SIDR procedural language not anchored to a specific calendar date. An AI tool trained predominantly on pre-amendment commentary reproduces that prior structure under the surface vocabulary of the amended rule, returning answers that look like the current rule but are operationally the previous one.
For Risk teams at Investment Banking firms, the systemic risk is not that one answer is wrong but that the AI's structured, confident presentation closely mimics the format of well-researched compliance content. A concentration table with a uniform percentage reads as a finished compliance artefact, even when the rule actually keys the ceiling to fund AUM and management-company AUM. A DWAM testing checklist that omits the carve-out set reads as complete, even when the omission collapses the testing perimeter.
A SIDR calendar with a six-to-twelve-month window reads as a reasonable compliance runway, even when the actual deadline is March 31, 2025.
The broader pattern is that AI tools perform worst precisely where the department needs accuracy: multi-condition trigger structures, narrow exclusion lists, and date-certain compliance anchors. These are not edge questions; they are the core of what the 2024 amendments changed, and they are exactly what a team is most likely to ask when updating policies or scheduling regulated filings.
What your team should do
For Risk teams at Investment Banking firms, the default workflow on Regulation 1.25 should treat AI output as a starting orientation rather than a source. For every numeric provision (the size-triggered concentration ceiling, the 24-month DWAM, the March 31, 2025 SIDR anchor) the operative position should be that the AI's output is verified against the Federal Register final rule notice and the codified CFR text before it enters a deliverable. The findings here show the AI does not surface uncertainty on these provisions: it returns confident, structured answers that read as adjudicated.
Build the verification step explicitly into the team's workflow. For investment-policy drafting, the checklist for every concentration provision is: what is the governing limit, what triggers activate it, and what is the verified source. For DWAM testing, the checklist for every excluded class is: is this class actually carved out under 17 CFR 1.25(b)(3)(iv) by name, and what is the verified source. For SIDR scheduling, the checklist for every compliance date is: what is the published date certain, and is the scheduling artefact aligned to it.
The DWAM no-standard finding (Sonnet 4.6 on direct Treasuries) is the most operationally dangerous of the five and warrants a specific control: any AI output that returns a no-standard or no-requirement answer on a numeric compliance provision should be treated as a flag for explicit verification, not as resolution of the question. The same applies to the SIDR drift: any compliance date framed as a relative range (six months, a year after the effective date) should be treated as un-anchored until verified against the rule.
How RLB Can Help
RegLeg's published Hallucination Research gives Risk teams a structured pre-flight check before relying on AI tools for Regulation 1.25 work. Before an AI-assisted investment policy, segregation testing playbook, or SIDR scheduling artefact is finalised, the research identifies precisely which areas of the regulatory text (concentration triggers, DWAM carve-outs, date-certain anchors) have historically generated confident but incorrect AI output. That forewarning lets the team apply targeted human scrutiny rather than blanket scepticism, making AI assistance genuinely efficient without importing undetected compliance risk into regulated workflows.
Beyond the published research, RegLeg works with Investment Banking firms on bespoke deep-dives that map AI-supported workflows within the Risk function to the function's actual hallucination exposure. Activities such as drafting concentration policies, scoping DWAM testing, building SIDR calendars, or coordinating cross-jurisdiction segregated-fund frameworks carry different risk profiles, and the deep-dive surfaces which ones warrant additional controls or independent verification steps.
RegLeg also conducts a confidential review of the firm's existing AI-use policy against the RegLeg failure-mode catalogue, delivering a prioritised remediation plan that distinguishes low-risk efficiency gains from higher-risk applications where AI output should be treated as a first draft only.
For teams that want to build durable in-house capability, RegLeg develops training material and content tailored to the Risk context. This covers how to interpret AI-generated regulatory summaries critically, how to structure escalation where AI confidence is high but human verification is essential, and how to document AI-assisted decision-making in a manner consistent with sound regulatory hygiene. The material can be delivered as standalone workshops or integrated into the firm's existing compliance training calendar.