Lawyers advising on the BBNJ Agreement are increasingly using AI to draft 2-page client memos on benefit-sharing exposure, generate partner-level briefings on environmental impact assessment thresholds for high-seas activities, and validate treaty-citation language against the deposited Agreement text before issuing legal opinions.
The RLB Specialist Panel put a set of practitioner-grade questions on the BBNJ Agreement to two frontier AI models with web search active. Each question is prepared by the Panel based on the workflows that lawyers actually use AI for under this treaty, covering the screening threshold for environmental impact assessments under Part IV, the temporal scope of the marine genetic resources and digital sequence information regime under Part II, the benefit-sharing duty for digital sequence information, and the non-undermining duty constraining Conference of the Parties decisions on area-based management tools under Part III.
The Panel then binds every AI response to verbatim regulator-issued source text held as primary substrate, comparing the AI output line-by-line against the deposited treaty text. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced. On the BBNJ Agreement, the AI subjects returned four hallucinated answers in the form of Inverted-Position Hallucination together with Source-Credit Misattribution for lawyers.
For lawyers issuing legal opinions, memoranda, and transactional documents that engage the BBNJ Agreement, treaty-citation accuracy is load-bearing: a counterparty, opposing counsel, or regulatory reviewer who can identify a citation error on first reading of the document calls the entire piece of advice into question.
An AI-drafted memo that points at the wrong article on a screening threshold, mis-states the direction of a benefit-sharing rule, or mis-locates the constraint on Conference of the Parties authority leaves the lawyer exposed to professional liability, the firm exposed to reputational risk, and the client exposed to commercial loss from a position structured on the wrong rule.
The published Specialist Panel findings, with model attribution, carry the following citation identifiers, each hyperlinked to the bound regulator-issued source text on the BBNJ Agreement regulation hub. The audit register surfaces these findings for lawyers so that any AI-assisted treaty citation, paraphrase, or rule-statement entering a deliverable can be re-validated against the deposited treaty text before the document is issued:
RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q001-Opus47 (EIA screening threshold misattributed to wrong article)RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q001-Sonnet46 (EIA screening threshold misattributed to wrong article)RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q003-Opus47 (MGR retroactivity default inverted)RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q003-Sonnet46 (MGR retroactivity default inverted)RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q004-Sonnet46 (DSI benefit-sharing article misidentified)RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q005-Opus47 (Non-undermining clause attributed to wrong article)This is the consolidated view of findings. Click the Citation IDs or 'see details →' on any item for the full details for each finding.
A lawyer advising a client on whether a planned high-seas activity must undergo an environmental impact assessment under the BBNJ Agreement would, on this AI response, cite the wrong article and could mis-scope the obligation. The screening threshold is set out at Article 27 of Part IV, not Article 30. A legal opinion or transactional disclosure that pins the EIA trigger to Article 30 fails on its face as a treaty citation, and a counterparty or supervisor reviewing the document would identify the error before relying on it.
This is the highest-stakes error in the cell for any lawyer advising on the BBNJ Agreement. Two frontier AI tools asserted that the Agreement's marine genetic resource and digital sequence information benefit-sharing rules apply retroactively by default, with a written opt-out, when Article 10(1) establishes the opposite: non-retroactivity is the default. A lawyer advising a bioprospecting client, a research consortium, or a State Party on the legal status of pre-entry-into-force sample collections would produce a fundamentally wrong compliance analysis. Contracts, access agreements, and benefit-sharing arrangements structured on this error would misallocate obligation and risk.
An AI tool correctly identified that digital sequence information derived from high-seas marine genetic resources is subject to fair and equitable benefit-sharing obligations, but cited Article 15(5) as the source when Article 14(1) is correct. For a lawyer drafting a legal opinion, a compliance memo, or a transactional document that must cite treaty authority precisely, this produces a citation that will not withstand scrutiny by counterparties or a regulatory reviewer. The substantive conclusion that DSI is in scope is right; the article reference is not.
A lawyer advising on the legal limits of the Conference of the Parties' authority to designate marine protected areas, including in international shipping lanes, would receive an answer that places the controlling non-undermining duty at Article 5 or Article 8 of the Agreement. The controlling provision is Article 22(2) of Part III, which specifically bounds Conference of the Parties decisions on area-based management tools.
A legal opinion that engages with Article 5 or Article 8 rather than Article 22(2) misses the targeted constraint and could misrepresent the scope of Conference of the Parties authority to a shipping, energy, or marine infrastructure client.
Every finding on this page compares an AI subject's account of the rule against the regulator's verbatim text from the regulator's own portal. Both are linked. Each delta, its root causes, and impact analysis are documented and published with immutable Citation IDs.