AI Hallucination Research › Briefings

Briefings Blog

The running blog from the RLB Specialist Panel delves into real-world scenarios where the compliance, legal, or AI lab team interacts with frontier AI models under specific regulations. The blogs are anonymised to remove client-specific details and include insights from the RLB team analysing the hallucinations experienced in AI models while working on these cases. For example, when a model returns a confident answer that contradicts the regulator's primary text, such as a fabricated staff letter, a wrong appendix, or an inverted scope, these issues are discussed here. Each blog explains one set of findings and what it would have meant for the team that would have acted on it, sans this research initiative. This blog is frequently updated, a few times a day.

263 briefings in the archive · Subscribe via Atom: /briefings/feed.xml (this blog) · /feed.xml (all RegLegBrief publications)
Audience colours: AI Labs Practitioner (profession) Sector × Department
Audience
Jur.
Regulator
Profession
Sector
Dept
Range
Sort
Per page
Showing 5 of 263 · page 47 of 53
Thursday, 18 June 2026
Sector: Pharmaceuticals and Dept: Legal INT UNTC

Pharmaceuticals Legal teams: documentation and reporting gaps possible from AI reading of BBNJ Agreement

For Pharmaceuticals Legal teams working with BBNJ High Seas Biodiversity Agreement: Specialist-Panel-verified findings on where AI summaries diverge from the regulator's text, and what that means for the sector's...

Legal teams at pharmaceutical firms are increasingly using AI to draft access and licensing agreements for marine-sourced material, generate counsel-facing memos on the BBNJ Agreement's digital sequence information benefit-sharing framework, and validate treaty-citation language in due-diligence disclosures and transactional documents.

The RLB Specialist Panel put a set of practitioner-grade questions on the BBNJ Agreement to two frontier AI models with web search active. Each question is prepared by the Panel based on the workflows that legal teams at pharmaceuticals firms actually use AI for under this treaty, covering the screening threshold for environmental impact assessments under Part IV, the temporal scope of the marine genetic resources and digital sequence information regime under Part II, the benefit-sharing duty for digital sequence information, and the non-undermining duty constraining Conference of the Parties decisions on area-based management tools under Part III.

The Panel then binds every AI response to verbatim regulator-issued source text held as primary substrate, comparing the AI output line-by-line against the deposited treaty text. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced. On the BBNJ Agreement, the AI subjects returned two hallucinated answers in the form of Inverted-Position Hallucination together with Source-Credit Misattribution for legal teams at pharmaceuticals firms.

For legal teams at pharmaceuticals firms advising on the BBNJ Agreement, treaty-citation accuracy is load-bearing in legal opinions, contractual representations, due-diligence disclosures, and any pleading or position paper engaging the Agreement. A counterparty or opposing counsel who identifies a misattributed article on first reading calls the entire piece of advice into question. The marine genetic resources retroactivity inversion is the more serious failure: a legal opinion structured around a retroactive-by-default rule when the treaty establishes the opposite default produces fundamentally wrong contract terms and exposes the firm to professional liability if the underlying position is later corrected.

The published Specialist Panel findings, with model attribution, carry the following citation identifiers, each hyperlinked to the bound regulator-issued source text on the BBNJ Agreement regulation hub. The audit register surfaces these findings for legal teams at pharmaceuticals firms so that any AI-assisted treaty citation, paraphrase, or rule-statement entering a deliverable can be re-validated against the deposited treaty text before the document is issued:

Sector: Pharmaceuticals and Dept: Compliance INT UNTC

Pharmaceuticals Compliance teams: documentation and reporting gaps possible from AI reading of BBNJ Agreement

For Pharmaceuticals Compliance teams working with BBNJ High Seas Biodiversity Agreement: Specialist-Panel-verified findings on where AI summaries diverge from the regulator's text, and what that means for the...

Compliance teams at pharmaceutical firms working with marine-sourced compounds are increasingly using AI to update sample-provenance and access-and-benefit-sharing checklists, generate research-governance bulletins under the BBNJ Agreement, and validate which obligations apply to legacy specimen collections and digital sequence information workflows.

The RLB Specialist Panel put a set of practitioner-grade questions on the BBNJ Agreement to two frontier AI models with web search active. Each question is prepared by the Panel based on the workflows that compliance teams at pharmaceuticals firms actually use AI for under this treaty, covering the screening threshold for environmental impact assessments under Part IV, the temporal scope of the marine genetic resources and digital sequence information regime under Part II, the benefit-sharing duty for digital sequence information, and the non-undermining duty constraining Conference of the Parties decisions on area-based management tools under Part III.

The Panel then binds every AI response to verbatim regulator-issued source text held as primary substrate, comparing the AI output line-by-line against the deposited treaty text. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced. On the BBNJ Agreement, the AI subjects returned two hallucinated answers in the form of Inverted-Position Hallucination together with Source-Credit Misattribution for compliance teams at pharmaceuticals firms.

For compliance teams at pharmaceuticals firms working under the BBNJ Agreement, internal policies, regulator-facing filings, and supervisor-engagement memos turn on citation accuracy. A compliance submission that mis-numbers the source article will be identified by a national Clearing-House Mechanism reviewer or a treaty-body monitoring reviewer on first reading, and the wider compliance narrative loses credibility.

Where the AI subjects inverted the direction of the marine genetic resources retroactivity default, the consequence is more serious: the firm could initiate costly and unnecessary remediation of legacy collections, or misstate its position in due diligence disclosures, licensing negotiations, and regulatory filings - any of which could attract scrutiny from national implementing authorities or treaty-body monitoring mechanisms.

The published Specialist Panel findings, with model attribution, carry the following citation identifiers, each hyperlinked to the bound regulator-issued source text on the BBNJ Agreement regulation hub. The audit register surfaces these findings for compliance teams at pharmaceuticals firms so that any AI-assisted treaty citation, paraphrase, or rule-statement entering a deliverable can be re-validated against the deposited treaty text before the document is issued:

Sector: Oil & Gas and Dept: Legal INT UNTC

Oil & Gas Legal teams: documentation and reporting gaps possible from AI reading of BBNJ Agreement

For Oil & Gas Legal teams working with BBNJ High Seas Biodiversity Agreement: Specialist-Panel-verified findings on where AI summaries diverge from the regulator's text, and what that means for the sector's...

Legal teams at oil and gas firms are increasingly using AI to draft client and board memos on environmental impact assessment scoping, generate partner-level briefings on Conference of the Parties authority over area-based management tools under the BBNJ Agreement, and validate treaty-citation language in contractual representations and regulatory filings.

The RLB Specialist Panel put a set of practitioner-grade questions on the BBNJ Agreement to two frontier AI models with web search active. Each question is prepared by the Panel based on the workflows that legal teams at oil & gas firms actually use AI for under this treaty, covering the screening threshold for environmental impact assessments under Part IV, the temporal scope of the marine genetic resources and digital sequence information regime under Part II, the benefit-sharing duty for digital sequence information, and the non-undermining duty constraining Conference of the Parties decisions on area-based management tools under Part III.

The Panel then binds every AI response to verbatim regulator-issued source text held as primary substrate, comparing the AI output line-by-line against the deposited treaty text. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced. On the BBNJ Agreement, the AI subjects returned two hallucinated answers in the form of Source-Credit Misattribution for legal teams at oil & gas firms.

For legal teams at oil & gas firms advising on the BBNJ Agreement, treaty-citation accuracy is load-bearing in legal opinions, contractual representations, due-diligence disclosures, and any pleading or position paper engaging the Agreement. A counterparty or opposing counsel who identifies a misattributed article on first reading calls the entire piece of advice into question. The marine genetic resources retroactivity inversion is the more serious failure: a legal opinion structured around a retroactive-by-default rule when the treaty establishes the opposite default produces fundamentally wrong contract terms and exposes the firm to professional liability if the underlying position is later corrected.

The published Specialist Panel findings, with model attribution, carry the following citation identifiers, each hyperlinked to the bound regulator-issued source text on the BBNJ Agreement regulation hub. The audit register surfaces these findings for legal teams at oil & gas firms so that any AI-assisted treaty citation, paraphrase, or rule-statement entering a deliverable can be re-validated against the deposited treaty text before the document is issued:

Sector: Maritime & Shipping and Dept: Legal INT UNTC

Maritime & Shipping Legal teams: documentation and reporting gaps possible from AI reading of BBNJ Agreement

For Maritime & Shipping Legal teams working with BBNJ High Seas Biodiversity Agreement: Specialist-Panel-verified findings on where AI summaries diverge from the regulator's text, and what that means for the sector's...

Legal teams at maritime and shipping firms are increasingly using AI to draft client memos on transit-rights exposure under area-based management tools, generate partner-level briefings on Conference of the Parties authority under the BBNJ Agreement, and validate treaty-citation language in contractual and regulatory submissions.

The RLB Specialist Panel put a set of practitioner-grade questions on the BBNJ Agreement to two frontier AI models with web search active. Each question is prepared by the Panel based on the workflows that legal teams at maritime & shipping firms actually use AI for under this treaty, covering the screening threshold for environmental impact assessments under Part IV, the temporal scope of the marine genetic resources and digital sequence information regime under Part II, the benefit-sharing duty for digital sequence information, and the non-undermining duty constraining Conference of the Parties decisions on area-based management tools under Part III.

The Panel then binds every AI response to verbatim regulator-issued source text held as primary substrate, comparing the AI output line-by-line against the deposited treaty text. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced. On the BBNJ Agreement, the AI subjects returned two hallucinated answers in the form of Source-Credit Misattribution for legal teams at maritime & shipping firms.

For legal teams at maritime & shipping firms advising on the BBNJ Agreement, treaty-citation accuracy is load-bearing in legal opinions, contractual representations, due-diligence disclosures, and any pleading or position paper engaging the Agreement. A counterparty or opposing counsel who identifies a misattributed article on first reading calls the entire piece of advice into question. The marine genetic resources retroactivity inversion is the more serious failure: a legal opinion structured around a retroactive-by-default rule when the treaty establishes the opposite default produces fundamentally wrong contract terms and exposes the firm to professional liability if the underlying position is later corrected.

The published Specialist Panel findings, with model attribution, carry the following citation identifiers, each hyperlinked to the bound regulator-issued source text on the BBNJ Agreement regulation hub. The audit register surfaces these findings for legal teams at maritime & shipping firms so that any AI-assisted treaty citation, paraphrase, or rule-statement entering a deliverable can be re-validated against the deposited treaty text before the document is issued:

Wednesday, 17 June 2026
Sector: Maritime & Shipping and Dept: Compliance INT UNTC

Maritime & Shipping Compliance teams: documentation and reporting gaps possible from AI reading of BBNJ Agreement

For Maritime & Shipping Compliance teams working with BBNJ High Seas Biodiversity Agreement: Specialist-Panel-verified findings on where AI summaries diverge from the regulator's text, and what that means for the...

Compliance teams at maritime and shipping firms are increasingly using AI to update route-screening and operational checklists, generate flag-state and treaty-body filing bulletins, and validate which provisions of the BBNJ Agreement govern environmental impact assessment screening and Conference of the Parties area-based decisions.

The RLB Specialist Panel put a set of practitioner-grade questions on the BBNJ Agreement to two frontier AI models with web search active. Each question is prepared by the Panel based on the workflows that compliance teams at maritime & shipping firms actually use AI for under this treaty, covering the screening threshold for environmental impact assessments under Part IV, the temporal scope of the marine genetic resources and digital sequence information regime under Part II, the benefit-sharing duty for digital sequence information, and the non-undermining duty constraining Conference of the Parties decisions on area-based management tools under Part III.

The Panel then binds every AI response to verbatim regulator-issued source text held as primary substrate, comparing the AI output line-by-line against the deposited treaty text. Only responses where the AI subject was demonstrably wrong against the verbatim regulator-issued source text are published; responses that were substantively correct, or that refused on calibration grounds, are retained internally and not surfaced. On the BBNJ Agreement, the AI subjects returned two hallucinated answers in the form of Source-Credit Misattribution for compliance teams at maritime & shipping firms.

For compliance teams at maritime & shipping firms working under the BBNJ Agreement, internal policies, regulator-facing filings, and supervisor-engagement memos turn on citation accuracy. A compliance submission that mis-numbers the source article will be identified by a national Clearing-House Mechanism reviewer or a treaty-body monitoring reviewer on first reading, and the wider compliance narrative loses credibility.

Where the AI subjects inverted the direction of the marine genetic resources retroactivity default, the consequence is more serious: the firm could initiate costly and unnecessary remediation of legacy collections, or misstate its position in due diligence disclosures, licensing negotiations, and regulatory filings - any of which could attract scrutiny from national implementing authorities or treaty-body monitoring mechanisms.

The published Specialist Panel findings, with model attribution, carry the following citation identifiers, each hyperlinked to the bound regulator-issued source text on the BBNJ Agreement regulation hub. The audit register surfaces these findings for compliance teams at maritime & shipping firms so that any AI-assisted treaty citation, paraphrase, or rule-statement entering a deliverable can be re-validated against the deposited treaty text before the document is issued:

↑ Back to top