Executive Summary
Risk teams at Statutory Boards & Agencies firms in international jurisdictions routinely reach for AI tools when producing cross-border payment market intelligence — quantitative landscape data on global fast payment systems, cross-border linkage counts, and operator-type breakdowns are standard inputs to risk assessments, board briefings, and regulatory mapping work on the CPMI API harmonisation framework. We tested AI assistants on precisely these quantitative questions drawn from CPMI's own published monitoring programme and authoritative BIS speeches.
Across the question we put to AI tools on this regulation, we found 1 hallucination: AI assistants conflated a CPMI monitoring survey's respondent count with the authoritative global universe figure, and separately denied the existence of operator-breakdown data that is explicitly stated in a published CPMI speech. The compounding risk is that the AI delivers both errors with apparent confidence, retracting only when challenged — by which point the figures may already have been embedded in a deliverable.
How AI gets this regulation wrong
The failure mode on this regulation is confident quantitative misrepresentation: AI tools present a number drawn from a monitoring survey's respondent pool as though it were the authoritative global count of operational fast payment systems, then separately claim that published operator-breakdown data simply does not exist in accessible CPMI sources — when it does. Both errors share the same underlying dynamic: the AI reaches for the nearest available number without distinguishing between what the regulator surveyed and what the regulator counted, and it treats gaps in its own retrieval as gaps in the public record.
| AI's Failure Mode | Count | Affected findings |
|---|---|---|
| Exposed Fabrication | 1 | Finding#1 |
What that means for your team
For Risk teams at Statutory Boards & Agencies firms, the impact concentrates on deliverable quality: a market briefing or regulatory mapping pack that carries a wrong FPS count or an absent operator-breakdown is a wrong deliverable — one that may need to be corrected after it has already reached internal decision-makers or external counterparts. In a sector where Risk functions often produce analysis that flows directly to regulators or board-level stakeholders, the cost of an AI-sourced quantitative error is not abstract.
| Risk Impact | Count | Affected findings |
|---|---|---|
| Wrong deliverable | 1 | Finding#1 |
When this affects your department
Risk teams at Statutory Boards & Agencies firms engage with this regulation when scoping cross-border payment risk assessments, conducting regulatory horizon-scanning, or producing market intelligence to support a business line or governance committee facing a question about API-linked payment infrastructure. A common trigger is an internal request for a landscape briefing — how many domestic fast payment systems exist globally, which already have cross-border capability, how many are building it, who operates them — to anchor a strategic or risk-governance decision.
AI tools are an obvious accelerant for this kind of synthesis task, and the specific data points in question here (operational count, cross-border linkage count, operator type) are exactly the kind of headline statistics a junior analyst would pull from an AI query to populate a briefing deck.
The failure point is precise. The AI presents 57 — the number of systems that responded to a CPMI monitoring survey — as the global count of operational domestic fast payment systems. The authoritative figure, from a CPMI speech, is 70+. For a Risk team, 57 versus 70+ is not a rounding difference: it misrepresents the scale of the ecosystem, affects the scope of any regulatory mapping exercise, and can flow into board papers or regulatory submissions without a second glance if the analyst trusts the AI's apparent confidence.
The companion error — the AI's assertion that the central bank / private operator breakdown is not available in accessible public CPMI sources — is equally consequential. That split (40% central bank, 35% private entity) is a governance-structure data point; omitting it from a cross-border payment risk analysis removes a key dimension of systemic-risk context.
The acute exposure for the firm arises when these deliverables travel beyond the Risk team. Statutory Boards & Agencies firms in international jurisdictions often work alongside or are supervised by the bodies that publish the source data. A document carrying a demonstrably incorrect CPMI figure — one the regulator itself can check against its own published speech — undermines the firm's credibility in an ongoing regulatory relationship and may require a formal correction, with the reputational and relationship costs that entails.
The findings at a glance
The table below sets out the finding tested against AI tools for this regulation, identifying the question area, how the AI failed, and the resulting risk impact for Risk teams at Statutory Boards & Agencies firms in international jurisdictions.
| # | Finding title | Type | Citation ID |
|---|---|---|---|
| 1 | Global FPS count and operator breakdown data | Hallucination | RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010 |
Aggregate impact
The finding in this cell concentrates on a specific and consequential vulnerability: quantitative data from CPMI's monitoring programme on the global fast payment system landscape. This is not peripheral material for Risk teams — FPS landscape statistics are standard inputs to cross-border payment risk assessments, regulatory mapping exercises, and executive briefings on payment system infrastructure. The AI's error is not a vague mischaracterisation of a policy principle; it is a precise wrong number presented as authoritative CPMI data, compounded by a false assertion that published supporting data is unavailable.
The failure has two layers that reinforce each other. The first is a category error: the AI presents the count of survey respondents (57 systems) as the count of globally operational domestic fast payment systems (70+), treating sample size as universe size. A Risk professional reviewing primary sources would catch this immediately, but a junior analyst relying on the AI's answer at face value would not.
The second layer is a false denial: the AI asserts that the central bank / private operator breakdown is not enumerated in accessible public CPMI sources, when a CPMI speech explicitly states the split as 40% central bank and 35% private entity. The AI's retrieval failure is presented as an absence of data — which is a materially different claim, and one that the analyst has no prompt to verify unless they independently know the data exists.
Together, the two errors produce a market briefing that appears complete but is quantitatively wrong in two distinct ways: wrong on the size of the ecosystem and silent on a key governance-structure dimension. For a Statutory Boards & Agencies firm whose Risk function feeds this analysis into board-level or regulator-facing deliverables, both errors carry real exposure — and the AI's pattern of confident assertion followed by retraction only under challenge means the error is most likely to survive into the final document precisely when time pressure is highest and the temptation to skip a primary-source check is greatest.
What your team should do
The default position for this regulation is clear: AI tools are not a reliable source for CPMI quantitative monitoring data. The specific figures that matter for cross-border payment risk analysis — global FPS operational count, cross-border linkage counts, planning-horizon projections, operator-type breakdowns — are drawn from a combination of CPMI monitoring reports, BIS speeches, and the API harmonisation toolkit itself. All of those sources are publicly accessible at bis.org. The authoritative numbers should be pulled directly from primary CPMI publications, not sourced via an AI query.
This is a one-time overhead, not an ongoing burden: the relevant figures are stable across a monitoring cycle, and a Risk team maintaining a live regulatory reference document can update them when new CPMI survey results are published.
Where AI tools remain genuinely useful for Risk teams on this regulation is in structural and textual work: mapping the CPMI recommendations to specific API components, summarising the toolkit's data elements and messaging standards requirements, drafting initial frameworks for internal policy alignment against the harmonisation objectives, or producing a first-pass gap analysis between a firm's current API architecture and the CPMI recommendations. These tasks work from well-established published text and do not require the AI to synthesise monitoring-programme statistics.
The hallucination risk on textual and structural work is substantially lower, and AI tools can meaningfully compress the time it takes to produce first-draft materials that a Risk professional then reviews and refines.
The practical control is straightforward: any deliverable that cites a CPMI quantitative figure must trace that figure to a specific identified publication or BIS speech, not to an AI summary. For teams that use AI to accelerate first-draft production, this check should sit at the review stage — a named reviewer verifies every statistic against a primary CPMI source before the document leaves the team. An additional discipline worth building in: confirm the publication date of the source.
CPMI monitoring survey results are updated periodically, and an AI working from older training data may surface counts from a prior survey cycle as though they were the most recent — a risk that compounds the sample-versus-universe confusion identified in this cell.
How RLB Can Help
RegLeg's published Hallucination Research gives your Risk team a ready-made pre-flight check before trusting AI output on any regulatory question — particularly cross-border obligations, multi-jurisdictional capital standards, and the kind of nuanced statutory-mandate interpretation where a confidently wrong answer causes the most damage. The research is regulation-specific and failure-mode specific: you can look up the reg your team is working against, see exactly where AI tools have misfired, and calibrate your reliance accordingly before it becomes a sign-off problem.
Beyond the published findings, we run bespoke regulator deep-dives scoped to Statutory Boards & Agencies Risk functions — mapping which workflows carry the highest hallucination exposure given your regulatory perimeter. That typically covers areas like cross-jurisdictional regulatory horizon scanning, policy-gap analysis against international standards bodies, and AI-assisted regulatory correspondence drafting, where the failure modes cluster around jurisdiction-specific carve-outs and mandate-boundary questions that AI tools systematically flatten. The output is a prioritised risk map your team can act on, not a generic AI-risk taxonomy.
We also work directly with Risk leads on confidential review of existing AI-use policies — comparing your current controls against our failure-mode catalogue and identifying where policy language creates unexamined exposure. Where the Risk team wants to build internal capability, we can develop training material and CPD-aligned content that is grounded in real regulatory hallucination data rather than vendor case studies, giving your team something they can defend to compliance, legal, and the board.