Failure class on record for Operations teams at Payment Institutions: False-Negative Retrievaland Schema Over-Specification.

Frontier AI models tested by the RLB Specialist Panel on the CPMI Harmonised ISO 20022 Data Requirements (Updated Report) produced 2 discrete hallucinations bound to verbatim regulator-issued source text. Each finding has a direct read-through into operations teams at payment institutions's working deliverables.

The pattern in one line

Leading AI assistants used by operations teams at payment institutions on the CPMI Harmonised ISO 20022 Data Requirements (Updated Report) returned answers that looked sourced and coherent but conflicted, in load-bearing specifics, with the regulator's verbatim text. The errors survive a first-pass review of operations teams at payment institutions's mapper working notes, QA test script, or onboarding checklist and only surface when a counterparty, regulator, or independent reviewer checks the primary record.

The shape of the failure is consistent across the the CPMI Harmonised ISO 20022 Data Requirements (Updated Report) test set: a confident, specific, institutionally plausible answer that maps to general payments-system knowledge rather than to the regulator's actual issued text on the question asked.

How the RLB Specialist Panel tested this

The Panel prepares questions based on real practical AI usage in the workflows payment-institution operations teams use AI for: drafting, validation, benchmarking, and source-citation tasks against the CPMI Harmonised ISO 20022 Data Requirements (Updated Report). Each tested question is bound to verbatim regulator-issued source text held as primary substrate; the Panel does not generate findings against documents whose verbatim text it cannot anchor. For this cell, the Panel ran two Specialist Panel questions against two frontier AI subjects with web search on, and recorded the AI's confident answer alongside the regulator's actual text for direct comparison.

The Specialist Panel questions are framed in the form a working payment-institution operations team would actually pose: not abstract knowledge probes, but deliverable-shaped requests that reflect the mapper working notes, QA test script, or onboarding checklist this audience produces. A Specialist Panel direct question asks the AI to state a regulator-issued fact; a Specialist Panel application-style question asks the AI to draft, validate, or benchmark using that fact. Both formats are run, and the Panel records the failure modes that surface across the two.

What the models got wrong

Finding 1 (RLB-H-INT-BIS-CPMI-ISO-20022-HARMONISATION-UPDATED-2026-Q007-Sonnet46, Claude Sonnet 4.6 (web search on)). On the Specialist Panel question covering the official CPMI/FSB benchmarks for cross-border payment inquiry rates and resolution-time reductions, the AI returned: "no specific official CPMI or FSB statistic was identified." The regulator-issued source text, held by the Panel as primary substrate from Fabio Panetta BIS speech r260316f (March 2026), records: "1 to 3 per cent of cross-border payments generate inquiries, typically requiring 5 to 10 manual touchpoints; ISO 20022 harmonisation can reduce resolution time by up to 80 per cent." The Panel classifies this as False-Negative Retrieval.

The audience-specific impact statement for this finding is recorded in the per-finding analysis card on the cell's detail surface, with the mapper working notes, QA test script, or onboarding checklist read-through spelled out for sign-off use.

Finding 2 (RLB-H-INT-BIS-CPMI-ISO-20022-HARMONISATION-UPDATED-2026-Q010-Opus47, Claude Opus 4.7 (web search on)). On the Specialist Panel question covering when the Fedwire Funds Service implemented the harmonised ISO 20022 data model and what postal address structure it requires, the AI returned: "optional structured fields including Street Name, Building Number, Post Code and Country Sub-Division." The regulator-issued source text, held by the Panel as primary substrate from Federal Reserve Bank Services FAQ on Fedwire ISO 20022 mandating-iso-20022-enhanced-data-in-chaps, records: "country code plus town name as mandatory, with optional free-format lines of 70 characters each for the unstructured component." The Panel classifies this as Schema Over-Specification.

The audience-specific impact statement for this finding is recorded in the per-finding analysis card on the cell's detail surface, with the mapper working notes, QA test script, or onboarding checklist read-through spelled out for sign-off use.

Why this matters for operations teams at payment institutions

Operations teams at Payment Institutions working on the CPMI Harmonised ISO 20022 Data Requirements (Updated Report) carry a direct read-through from the AI's wrong answer to the mapper working notes, QA test script, or onboarding checklist they sign off on. The exposure is STP failures and systematic manual intervention in address-field processing at the transaction volumes the harmonisation programme is designed to reduce. The failure modes recorded here are not edge-case linguistic slips: they touch the load-bearing operational, benchmarking, or governance specifics that operations teams at payment institutions are paid to get right.

Where an AI assistant returns a confident, plausible-looking answer that conflicts with the regulator's verbatim text, the cost of correction rises with every downstream artefact that cites it.

The compounding risk for this audience is structural. A first-pass reviewer sees a specific number, a named institution, or a structured schema and treats it as evidence the AI has retrieved an authoritative source. The Panel's testing surfaces the cases where that assumption does not hold, and where the corrective regulator text is materially different from the AI's confident output. Once an AI-sourced figure or attribution enters a board pack, a regulatory submission, or a vendor specification, the audit trail of correction can extend across multiple deliverables before the error is contained.

The regulator's actual position

On the official CPMI/FSB benchmarks for cross-border payment inquiry rates and resolution-time reductions, Fabio Panetta BIS speech r260316f (March 2026) states:

1 to 3 per cent of payments generate inquiries typically requiring 5 to 10 manual touchpoints; resolution times reducible by up to 80 per cent through harmonised ISO 20022 implementation

On when the Fedwire Funds Service implemented the harmonised ISO 20022 data model and what postal address structure it requires, Federal Reserve Bank Services FAQ on Fedwire ISO 20022 mandating-iso-20022-enhanced-data-in-chaps states:

The July 14, 2025 implementation enabled participants to apply CPMI data model requirements from the Harmonised ISO 20022 data requirements for enhancing cross-border payments

Each verbatim block above is held by the Panel as primary substrate and is the anchor against which the AI subjects' answers were compared. Where a Specialist Panel question crosses more than one regulator-issued document (a BIS speech together with an FRB Services FAQ, for example), the Panel records the source attribution per finding so the corrective text is unambiguous. The verbatim quoted above is the text against which the AI's answer fails.

    ## What this tells us about AI for operations teams at payment institutions

    The pattern recorded against this cell maps to the failure classes the RLB Specialist Panel
    catalogues across regulators:

    - False-Negative Retrieval: the model produced a confident, sourced-looking answer that conflicts with the regulator's actual text on missing inquiry-rate and resolution-time benchmarks.

• Schema Over-Specification: the model produced a confident, sourced-looking answer that conflicts with the regulator's actual text on Fedwire hybrid postal address schema over-specification.

  For operations teams at payment institutions, the practical signal is that AI assistants with web search enabled remain
  prone to producing single composite figures, structured-field over-specification, missed
  retrievals, and source-credit attribution errors on questions where the regulator's primary
  document is the only authoritative anchor. The errors are most dangerous where they look more
  conservative or more specific than the underlying rule, because that is precisely the shape a
  first-pass reviewer is least likely to challenge.
  
  The Panel's broader catalogue records that web-search-enabled frontier models on cross-border
  payments topics tend to default to general payments-industry knowledge (CBPR+ schemas, blended
  adoption statistics, well-known central-bank names) when the underlying regulator-issued text
  contains a specific, less-familiar fact. For operations teams at payment institutions drafting against the CPMI Harmonised ISO 20022 Data Requirements (Updated Report),
  the corrective is straightforward: where the workflow depends on a specific regulator-issued
  fact, the primary substrate has to remain the anchor, and the AI's draft has to be checked
  against it before sign-off.
  

What the RLB Specialist Panel is doing about it

The Panel runs Specialist Panel direct questions and Specialist Panel application-style questions against frontier AI models on every reg-rooted workflow operations teams at payment institutions actually use AI for. Each surfaced hallucination is bound to a verbatim regulator-issued anchor before publication, and each is recorded with a citation ID that traces the question, the AI's response, the verbatim source text, and the audience-specific operational consequence.

The Panel works directly with institutional readers, AI labs, and regulator-facing teams to feed back the patterns the Panel records, so the same failure modes can be addressed at source rather than caught at the reviewer's desk.

For institutional readers in operations teams at payment institutions roles, the Panel's surface is designed to be used in two ways: as a reference index when an AI-drafted deliverable references the CPMI Harmonised ISO 20022 Data Requirements (Updated Report) directly, and as a workflow-level signal of where AI assistance is and is not robust on cross-border payments topics. The Panel's per-finding cards include the regulator's exact text alongside the AI's failed response, so a reviewer can resolve a question without leaving the cell.

Partnership conversations are open to AI labs that want the catalogue's full failure taxonomy fed into their evaluation pipelines.

What Payment-institution operations teams should do

• Verify against primary substrate: where the deliverable depends on a specific number, chair attribution, or schema definition from the CPMI Harmonised ISO 20022 Data Requirements (Updated Report), do not rely on the AI's summary; open the regulator's verbatim text and confirm the load-bearing detail.
• Treat AI-sourced statistics as a starting point, not an anchor: any single composite figure applied uniformly to two distinct regulated populations is a red flag and should be checked against the regulator's separate per-population numbers.
• Build a verification step into the workflow: for the mapper working notes, QA test script, or onboarding checklist this audience produces, introduce a primary-source check on every numeric, schema, or institutional-attribution claim before sign-off.
• Use the Panel's citation IDs as a reference index: when an AI answer matches a known Panel finding, the cited regulator text is already provided as the corrective anchor.
• Treat stricter-looking AI outputs with the same scepticism as looser-looking ones: in the Panel's catalogue, over-specified schemas and over-confident composite figures are precisely the shapes a first-pass reviewer is least likely to challenge, and therefore precisely the shapes most likely to flow into a downstream artefact uncorrected.
• Where the AI workflow is repeated across corridors, jurisdictions, or product lines, run a periodic spot-check against the Panel's catalogue: a single template error can propagate across multiple markets before discovery, and the Panel's per-cell surface is structured to make that template-level audit straightforward.

The Panel records the 2 hallucinations above with citation IDs RLB-H-INT-BIS-CPMI-ISO-20022-HARMONISATION-UPDATED-2026-Q007-Sonnet46, RLB-H-INT-BIS-CPMI-ISO-20022-HARMONISATION-UPDATED-2026-Q010-Opus47 for direct reference in this audience's workflow. The same citation IDs are surfaced on the cell detail page so a reviewer running a sign-off check against an AI-drafted deliverable can pull the corrective regulator text without leaving the briefing.

Right of Reply

These findings and associated work have been put up in public with a view of the greater good for the development of a safer AI ecosystem. Any party reading this or any finding on reglegbrief.com may contact us and have an unconditional right of reply; the Specialist Panel will publish any factual correction or contextual response alongside the original finding, with no editorial gatekeeping. Researchers, regulators, and compliance teams with questions on methodology or specific findings can reach the Specialist Panel via the same channel.

Source & Methodology Standards

RegLeg Brief is operated by Verdus Technologies Pte. Ltd. (UEN 201616982R), incorporated in Singapore. The RLB Specialist Panel, with an aggregate of over 60 years of public-policy and industry experience, documents only confirmed hallucination findings, under a methodology that requires a verbatim regulator excerpt for every documented claim. All findings, citation IDs, model outputs, regulator excerpts, and methodology notes are open-access.

Primary source verified: CPMI Harmonised ISO 20022 Data Requirements for Cross-Border Payments (2026 Update) · Substrate documents: p_11_SPEECH_Cross_border_payment_inquiry_rates_and_e_r260316f.htm, p_17_NOTICE_Fedwire_Funds_Service_implementation_dat_mandating-iso-20022-enhanced-data-in-chaps.html · CPMI portal: bis.org/cpmi

Citation IDs referenced:

• RLB-H-INT-BIS-CPMI-ISO-20022-HARMONISATION-UPDATED-2026-Q007-Sonnet46
• RLB-H-INT-BIS-CPMI-ISO-20022-HARMONISATION-UPDATED-2026-Q010-Opus47