Frontier AI models invert the BBNJ Agreement's retroactivity default and mis-cite the treaty articles.

Two frontier AI subjects tested by the RLB Specialist Panel produced a fundamentally inverted reading of Article 10(1) on marine genetic resources, alongside misattributed citations for the environmental impact assessment screening threshold and the digital sequence information benefit-sharing duty.

The pattern in one line

Frontier AI models tested on the BBNJ Agreement returned answers that read as authoritative, but mis-attributed the screening, benefit-sharing, and non-undermining provisions to the wrong articles, and on marine genetic resources inverted the direction of the underlying rule altogether.

How the RLB Specialist Panel tested this

The questions in this cell were prepared by the RLB Specialist Panel based on real, practical AI usage in the workflows that lawyers actually use AI for under the BBNJ Agreement. Each question targets a specific deliverable type where an AI assistant is plausibly the first draft: a memo, an opinion paragraph, a checklist line, a board-paper bullet, a regulator-facing filing sentence. The Panel issued each question to two frontier AI subjects with web search active.

The Panel then bound every AI response to verbatim regulator-issued source text held as primary substrate, comparing the model output against the deposited treaty text and the regulator-issued source documentation for each provision. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published as findings; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced.

What the models got wrong

The cell carries 4 confirmed findings against the AI subjects on the BBNJ Agreement. Each is published against verbatim regulator-issued treaty text and carries explicit model attribution for audit transparency.

Finding 1: EIA screening threshold pinned to Article 30 instead of Article 27 (Part IV). The Specialist Panel asked, in direct form, which provision of the BBNJ Agreement establishes the qualitative screening threshold that triggers an environmental impact assessment obligation for planned high-seas activities. Claude Opus 4.7 with web search active answered that under Article 30 of the BBNJ Agreement a planned activity in areas beyond national jurisdiction must be screened under Article 31 and, if warranted, undergo a full environmental impact assessment (RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q001-Opus47).

Claude Sonnet 4.6 with web search active produced a closely related answer, stating that under Part IV, Article 30, a planned high-seas activity must undergo an environmental impact assessment when it is likely to have more than a minor or transitory effect (RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q001-Sonnet46). The substrate held by the Panel sets the screening threshold and the qualitative test at Article 27 of Part IV. The qualitative test the models paraphrased is the right test; the article number is wrong.

Finding 2: Marine genetic resources retroactivity flipped from non-retroactive default to retroactive default. The Specialist Panel asked, in application form, whether the BBNJ Agreement applies its marine genetic resources benefit-sharing obligations to specimens and digital sequence information collected from international waters before the Agreement entered into force. Claude Opus 4.7 with web search active answered that under Article 10(1) the marine genetic resources and digital sequence information provisions apply to utilization of resources that were collected or generated before entry into force, not just after (RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q003-Opus47).

Claude Sonnet 4.6 with web search active reached the same substantive conclusion, stating that the benefit-sharing and notification obligations apply to the utilisation of resources collected or generated before the Agreement entered into force on 17 January 2026 (RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q003-Sonnet46). The substrate held by the Panel records the opposite position: Article 10(1) limits the Part II regime to marine genetic resources and digital sequence information collected and generated after entry into force for each Party, with non-retroactivity as the default.

This is the highest-stakes class of error in the cell because the substantive direction of the obligation is inverted, not merely the article number.

Finding 3: Digital sequence information benefit-sharing duty pinned to Article 15(5) instead of Article 14(1). The Specialist Panel asked, in direct form, whether the BBNJ Agreement's benefit-sharing framework extends to digital sequence information derived from marine organisms found in international waters, and which provision governs the obligation. Claude Sonnet 4.6 with web search active answered that Part II, including Article 15(5) and related provisions, applies to both physical marine genetic resource samples and their associated digital sequence information (RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q004-Sonnet46).

The substrate held by the Panel locates the duty at Article 14(1): benefits arising from activities related to marine genetic resources in areas beyond national jurisdiction and their digital sequence information shall be shared in a fair and equitable manner. The substantive conclusion that digital sequence information is in scope is correct; the article number is not.

Finding 4: Non-undermining clause attributed to Article 5 or Article 8 instead of Article 22(2) (Part III). The Specialist Panel asked, in direct form, whether the model correctly identifies Article 22(2) as the provision that constrains the Conference of the Parties from overriding the competences of other international bodies when designating marine protected areas. Claude Opus 4.7 with web search active answered that Article 5 or Article 8 of the Agreement explicitly requires it not to undermine relevant legal instruments, frameworks and competent global, regional, subregional and sectoral bodies (RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q005-Opus47).

The substrate held by the Panel locates the controlling duty at Article 22(2) of Part III: the Conference of the Parties shall respect the competences of, and not undermine, relevant legal instruments and frameworks and relevant global, regional, subregional and sectoral bodies. Article 5 and Article 8 carry general purposes and principles language, not the targeted constraint on Conference of the Parties decisions on area-based management tools.

Why this matters for Lawyers

Lawyers advising on the BBNJ Agreement operate in a high-citation-fidelity environment. Treaty opinions, due-diligence memoranda, contractual representations, and pleadings are all read line-by-line by counterparties, regulatory reviewers, and academic commentators. A misattributed article reference is identifiable on first review, and the wider opinion loses credibility once the reader spots one verifiable error. The Article 10(1) inversion is the more serious failure: a legal opinion that pins benefit-sharing exposure to a retroactive-by-default rule when the treaty actually establishes non-retroactivity as the default produces fundamentally wrong contract terms, wrong due-diligence disclosures, and wrong client guidance.

The misattributions on Articles 27, 14(1), and 22(2) are less directionally wrong but equally fatal to the credibility of the document carrying them.

The regulator's actual position

The verbatim regulator-issued source text held by the RLB Specialist Panel as primary substrate for the BBNJ Agreement sets the position as follows. The references below are drawn from the deposited treaty text and are the controlling reference points against which any AI-assisted citation should be validated.

Article 27, Part IV. The screening threshold and the qualitative more than a minor or transitory effect test sit in Part IV of the BBNJ Agreement at Article 27. Part IV runs the environmental impact assessment regime: screening, scoping, assessment, monitoring, and review. Article 27 is the entry point that determines whether a planned activity in areas beyond national jurisdiction must be assessed at all.

Article 10(1). The marine genetic resources and digital sequence information provisions in Part II apply only to resources collected and generated after the Agreement enters into force for each Party. Non-retroactivity is the default. A number of Parties also recorded formal declarations on this point at deposit. Legacy collections obtained before the Party's date of entry into force are outside the Part II benefit-sharing regime by default; bringing them in requires an affirmative act, not the absence of one.

Article 14(1). Benefits arising from activities related to marine genetic resources in areas beyond national jurisdiction and their digital sequence information shall be shared in a fair and equitable manner. Digital sequence information is on the face of the provision, not parked in a subsidiary clause elsewhere in Part II. Article 15 carries the modalities of monetary and non-monetary benefit-sharing, but the source duty sits at Article 14(1).

Article 22(2), Part III. The Conference of the Parties shall respect the competences of, and not undermine, relevant legal instruments and frameworks and relevant global, regional, subregional and sectoral bodies. This is the targeted constraint on Conference of the Parties decisions on area-based management tools, including marine protected areas. Article 5 and Article 8 of the Agreement set out general purposes and principles; they are not the operative non-undermining duty for Conference of the Parties decisions under Part III.

What this tells us about AI for Lawyers

There are two distinct lessons here for lawyers working with AI on the BBNJ Agreement. The first, on Article 10(1), is the more serious. The AI subjects' answers are not surface citation slips: they invert the direction of an obligation. Where the treaty says non-retroactivity is the default, the AI subjects said retroactivity is the default. A reader who runs an AI-drafted memo or checklist through a citation-check workflow will not catch this; the article number cited is actually right. Only a substantive read against the treaty text catches the inversion.

This is the failure mode AI Labs need to know about, and it is the failure mode practitioners need to defend against most rigorously. The second, on Articles 27, 14(1), and 22(2), is the more familiar pattern: the substantive paraphrase is close enough that the answer reads as authoritative, but the article number is wrong. Citation-checking workflows will catch this if they run against the deposited treaty text rather than against a chained reference.

The defensive posture is the same: always anchor the citation against the deposited Agreement text, not against a secondary commentary, before letting an AI-drafted output go out the door.

What the RLB Specialist Panel is doing about it

The RLB Specialist Panel is engaging with the AI subjects' developers and with practitioner audiences working under the BBNJ Agreement. The Panel maintains an audit register of confirmed hallucinations bound to verbatim regulator-issued source text, surfaces them on the live regulation page and on each audience-specific briefing, and accepts right-of-reply submissions from the AI subjects' developers and from regulator-side reviewers. For lawyers this means the same questions can be re-issued against successor model releases; the bound substrate makes it straightforward to verify whether a specific failure mode has been corrected upstream, or whether the same hallucination is still being produced.

Partnership briefings with AI labs are offered against the audit register, not against synthesised demonstrations, so the corrections that matter are evidenced against treaty text rather than against a paraphrase chain.

The register is structured so that each finding records the question put to the AI subject, the AI subject's verbatim answer, the verbatim regulator-issued source text the answer was bound against, the named model and configuration, and the failure mode. That structure lets practitioner readers see exactly where the AI subject diverged from the treaty text without re-doing the underlying verification, and lets AI lab readers see exactly which provision and which phrasing produced the divergence.

Where a hallucination has been corrected in a successor model release, the register records the rerun and withdraws the finding; where it persists, the finding stays live. This makes the register useful as a continuous-improvement signal for the AI labs and as a defensive checklist for practitioners drawing on AI in regulated workflows.

What Lawyers teams should do

For lawyers drawing on AI in workflows that touch the BBNJ Agreement, the practical action items are direct:

• Anchor every BBNJ Agreement article reference produced by an AI assistant against the deposited treaty text held by the United Nations Treaty Collection before the article number enters a memo, opinion, or contractual representation.
• Treat any AI-generated statement on the temporal scope of Part II marine genetic resources or digital sequence information obligations as a high-risk paragraph: read Article 10(1) directly and confirm the non-retroactive default before relying on the AI characterisation.
• When advising on Conference of the Parties authority to designate marine protected areas, validate that the controlling non-undermining duty is anchored to Article 22(2) of Part III, not to general purposes-and-principles provisions earlier in the Agreement.
• Maintain a firm-level AI assist log that records every AI-drafted treaty citation entering a client deliverable, so that downstream verification and post-hoc audit are traceable.
• Subscribe to the RLB Specialist Panel audit register on the BBNJ Agreement to receive notifications when a confirmed AI hallucination is added, withdrawn, or corrected against successor model releases.

Right of Reply

These findings and associated work have been put up in public with a view of the greater good for the development of a safer AI ecosystem. Any party reading this or any finding on reglegbrief.com may contact us and have an unconditional right of reply; the Specialist Panel will publish any factual correction or contextual response alongside the original finding, with no editorial gatekeeping. Researchers, regulators, and compliance teams with questions on methodology or specific findings can reach the Specialist Panel via the same channel.

Source & Methodology Standards

RegLeg Brief is operated by Verdus Technologies Pte. Ltd. (UEN 201616982R), incorporated in Singapore. The RLB Specialist Panel, with an aggregate of over 60 years of public-policy and industry experience, documents only confirmed hallucination findings, under a methodology that requires a verbatim regulator excerpt for every documented claim. All findings, citation IDs, model outputs, regulator excerpts, and methodology notes are open-access.

Primary source verified: UN BBNJ Agreement (2023), Agreement on the Conservation and Sustainable Use of Marine Biological Diversity of Areas Beyond National Jurisdiction · Substrate documents: p_01_ACT_Part_III___Article_22_2____non_undermini_text-bbnj-agreement.html, p_01_ACT_Part_II___DSI_included_in_BBNJ_vs__exclu_text-bbnj-agreement.html, p_01_ACT_Part_IV_EIA___Article_27___screening_thr_text-bbnj-agreement.html · UN portal: documents.un.org

Citation IDs referenced:

• RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q001-Opus47
• RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q003-Opus47
• RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q004-Sonnet46
• RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q005-Opus47