Briefings Blog

Professional engineers scoping projects that may touch areas beyond national jurisdiction are increasingly using AI to draft environmental impact assessment scoping documents, generate technical briefings for design teams on screening thresholds, and validate which provision of the BBNJ Agreement governs a particular obligation before submitting deliverables to clients or regulators.

The RLB Specialist Panel put a set of practitioner-grade questions on the BBNJ Agreement to two frontier AI models with web search active. Each question is prepared by the Panel based on the workflows that professional engineers actually use AI for under this treaty, covering the screening threshold for environmental impact assessments under Part IV, the temporal scope of the marine genetic resources and digital sequence information regime under Part II, the benefit-sharing duty for digital sequence information, and the non-undermining duty constraining Conference of the Parties decisions on area-based management tools under Part III.

The Panel then binds every AI response to verbatim regulator-issued source text held as primary substrate, comparing the AI output line-by-line against the deposited treaty text. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced. On the BBNJ Agreement, the AI subjects returned a single hallucinated answer in the form of Source-Credit Misattribution for professional engineers.

For professional engineers scoping projects that may engage areas beyond national jurisdiction, citation accuracy in environmental impact assessment scoping documents is load-bearing. A scoping document that pins the screening obligation to the wrong article of the BBNJ Agreement will be challenged on first review by a regulator, a peer reviewer, or the client's own legal team. The substantive screening test the AI paraphrased may be the right test, but the citation will need to be reworked, and the engineer issuing the document carries professional responsibility for the accuracy of the regulatory reference framing the work.

The published Specialist Panel findings, with model attribution, carry the following citation identifiers, each hyperlinked to the bound regulator-issued source text on the BBNJ Agreement regulation hub. The audit register surfaces these findings for professional engineers so that any AI-assisted treaty citation, paraphrase, or rule-statement entering a deliverable can be re-validated against the deposited treaty text before the document is issued:

• RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q001-Opus47 (EIA screening threshold misattributed to wrong article)
• RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q001-Sonnet46 (EIA screening threshold misattributed to wrong article)

Lawyers advising on the BBNJ Agreement are increasingly using AI to draft 2-page client memos on benefit-sharing exposure, generate partner-level briefings on environmental impact assessment thresholds for high-seas activities, and validate treaty-citation language against the deposited Agreement text before issuing legal opinions.

The RLB Specialist Panel put a set of practitioner-grade questions on the BBNJ Agreement to two frontier AI models with web search active. Each question is prepared by the Panel based on the workflows that lawyers actually use AI for under this treaty, covering the screening threshold for environmental impact assessments under Part IV, the temporal scope of the marine genetic resources and digital sequence information regime under Part II, the benefit-sharing duty for digital sequence information, and the non-undermining duty constraining Conference of the Parties decisions on area-based management tools under Part III.

The Panel then binds every AI response to verbatim regulator-issued source text held as primary substrate, comparing the AI output line-by-line against the deposited treaty text. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced. On the BBNJ Agreement, the AI subjects returned four hallucinated answers in the form of Inverted-Position Hallucination together with Source-Credit Misattribution for lawyers.

For lawyers issuing legal opinions, memoranda, and transactional documents that engage the BBNJ Agreement, treaty-citation accuracy is load-bearing: a counterparty, opposing counsel, or regulatory reviewer who can identify a citation error on first reading of the document calls the entire piece of advice into question.

An AI-drafted memo that points at the wrong article on a screening threshold, mis-states the direction of a benefit-sharing rule, or mis-locates the constraint on Conference of the Parties authority leaves the lawyer exposed to professional liability, the firm exposed to reputational risk, and the client exposed to commercial loss from a position structured on the wrong rule.

The published Specialist Panel findings, with model attribution, carry the following citation identifiers, each hyperlinked to the bound regulator-issued source text on the BBNJ Agreement regulation hub. The audit register surfaces these findings for lawyers so that any AI-assisted treaty citation, paraphrase, or rule-statement entering a deliverable can be re-validated against the deposited treaty text before the document is issued:

• RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q001-Opus47 (EIA screening threshold misattributed to wrong article)
• RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q001-Sonnet46 (EIA screening threshold misattributed to wrong article)
• RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q003-Opus47 (MGR retroactivity default inverted)
• RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q003-Sonnet46 (MGR retroactivity default inverted)
• RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q004-Sonnet46 (DSI benefit-sharing article misidentified)
• RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q005-Opus47 (Non-undermining clause attributed to wrong article)

Public auditors testing FMI cyber-resilience controls and audit clients exposed to the CPMI-IOSCO 2016 Cyber Guidance are increasingly using AI to draft testing programmes, populate working-paper regulatory criteria sections, and prepare management-letter findings citing the 2016 framework. In practice, AI is used to draft testing programmes for FMI cyber controls, populate the regulatory criteria section of audit working papers, prepare management-letter findings citing CPMI-IOSCO 2016 expectations, and generate cyber-control walkthroughs against the 2016 guidance categories.

That workflow places the regulator-issued text of the 2016 guidance, its 2018-2020 derivative standards, and its current operative status at the centre of every AI-generated deliverable for public auditors.

Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confident, citable reconstructions of the CPMI-IOSCO 2016 Cyber Guidance (June 2016) that the regulator-issued primary text directly contradicts across nine findings spanning four failure classes: Source-Credit Fabrication (an asserted NIST Cybersecurity Framework citation that the 2016 guidance does not contain), Misattribution (the slogan 'secure the periphery, protect the core' located inside CPMI-IOSCO 2016 guidance or its 2018 wholesale-payments paper rather than the actual 2018 speech source), Anachronistic Cross-Reference (the 2016 guidance asserted as definitionally aligned with the November 2018 FSB Cyber Lexicon and the October 2020 FSB Effective Practices that postdate it), and Outdated Standing Claim (the 2016 guidance presented as the unchanged operative standard when CPMI-IOSCO has issued a May 2026 consultative document under active revision).

Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows public auditors use AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For public auditors testing FMI cyber controls, the failure pattern is operationally consequential. A regulatory-criteria section that records an asserted NIST CSF citation in the 2016 guidance documents the audit's reference framework on a wrong reading of the source. A testing programme that records the 2016 guidance as containing operational depth for forensic analysis or a cyber-attack database points the auditor at a specification level the 2016 text does not contain. A management-letter finding that records the 2016 guidance as the unchanged operative standard misstates the regulatory horizon at the reporting date.

The audit's nine findings are documented with immutable RLB Citation IDs. Representative entries include RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q019-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Opus47, and RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Sonnet46. The full audit is documented at the CPMI-IOSCO 2016 Cyber Resilience Guidance hub on RegLegBrief.com.

Lawyers working on the CPMI-IOSCO Principles for Financial Market Infrastructures (PFMI, 2012) are increasingly relying on AI to draft 2-page board memos on FMI governance obligations, generate client-facing summaries of PFMI Principle 2 Key Considerations, prepare partner-level briefings on disclosure-framework responses, and validate committee-mandate language against the regulator's published Key Considerations. The PFMI framework is the global standard for systemically important payment systems, central counterparties, and securities settlement infrastructures, and the document's structure makes it particularly amenable to AI summarisation: numbered Principles, numbered Key Considerations, and lettered annexes that the model can address by number.

That surface structure is also what makes the failure mode the RegLeg Brief Specialist Panel records here invisible at runtime: the document is regularly cited by Key Consideration number in board papers, disclosure-framework returns, and counterparty representations, which means a misattributed citation does not register as a substantive error in the draft, it registers as a competent regulatory paragraph that the reader will not check against the regulator's primary text unless something else prompts the verification.

Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confidently wrong reconstructions of the PFMI's governance and oversight architecture under Principle 2 (governance) and Annex F (oversight expectations for critical service providers). The Panel records one finding in the class the team labels "Source-Credit Misattribution", in which the models stated a substantively plausible governance position and pinned it to a named Key Consideration that the published PFMI text does not support. The finding identifiers are RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q022-Sonnet46.

For Lawyers, the failure shape matters because the work product is client memos on FMI board obligations, legal opinions on PFMI self-assessment responses, drafted board terms of reference, and counterparty representations on governance compliance, all of which travel under the firm's name to a board, supervisor, counterparty, or public reviewer who can locate the cited Key Consideration and check it against the regulator's primary text.

Lawyers who paste AI output into a legal opinion or a client-facing compliance memo are the population most exposed to a fabricated Key Consideration citation that the reader will check against the PFMI primary text the moment a counterparty or supervisor disputes the position.

The Panel documents the finding identifiers RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q022-Sonnet46. The AI subjects under test were Claude Sonnet 4.6, each running with web search enabled, mirroring the workflow most practitioners run when they ask an assistant a Principle 2 or Annex F question. The verbatim regulator text is held as primary substrate (R2-REGULATION-d101a_PFMI_main_text.pdf). Each finding card sets out the exact strings the model produced, the verbatim regulator excerpt the model's output contradicts, and the failure-class label the RegLeg Brief Specialist Panel assigns.

The records are open-access; AI labs named in any finding have an unconditional right of reply, and the Specialist Panel will document any factual correction or contextual response alongside the original finding.

Lawyers advising on cyber resilience for financial market infrastructures and the CPMI-IOSCO 2016 Cyber Guidance are increasingly using AI to draft client memos, validate threshold language, and prepare partner-level briefings on the global guidance and its post-2016 evolution. In practice, AI is used to draft client memos on the CPMI-IOSCO 2016 Cyber Guidance, validate cyber-programme citations against the regulator text, generate partner-level briefings on how the guidance is referenced by national supervisors, and prepare counsel-to-board commentary on FMI cyber-resilience standards.

That workflow places the regulator-issued text of the 2016 guidance, its 2018-2020 derivative standards, and its current operative status at the centre of every AI-generated deliverable for lawyers.

Two frontier AI models tested by the RegLeg Brief Specialist Panel produced confident, citable reconstructions of the CPMI-IOSCO 2016 Cyber Guidance (June 2016) that the regulator-issued primary text directly contradicts across nine findings spanning four failure classes: Source-Credit Fabrication (an asserted NIST Cybersecurity Framework citation that the 2016 guidance does not contain), Misattribution (the slogan 'secure the periphery, protect the core' located inside CPMI-IOSCO 2016 guidance or its 2018 wholesale-payments paper rather than the actual 2018 speech source), Anachronistic Cross-Reference (the 2016 guidance asserted as definitionally aligned with the November 2018 FSB Cyber Lexicon and the October 2020 FSB Effective Practices that postdate it), and Outdated Standing Claim (the 2016 guidance presented as the unchanged operative standard when CPMI-IOSCO has issued a May 2026 consultative document under active revision).

Questions are prepared by the RLB Specialist Panel based on real practical AI usage in the workflows lawyers use AI for. The Panel binds each AI finding to verbatim regulator-issued source text held as primary substrate.

For lawyers advising FMI operators, supervisors, and FMI participant banks, the failure pattern is operationally consequential. A client memorandum that recites an explicit NIST CSF citation that the 2016 guidance does not contain misstates the regulatory foundation. A counsel-to-board briefing that records the 2016 guidance as the unchanged operative standard, when CPMI-IOSCO has issued a May 2026 consultative document under active revision, embeds a falsifiable status claim into a regulated deliverable.

The audit's nine findings are documented with immutable RLB Citation IDs. Representative entries include RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q008-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q019-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Opus47, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q020-Sonnet46, RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Opus47, and RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Sonnet46. The full audit is documented at the CPMI-IOSCO 2016 Cyber Resilience Guidance hub on RegLegBrief.com.