Executive Summary
The CPMI-IOSCO Level 3 assessment on general business risk evaluates how FMIs have implemented PFMI Principle 15, and for Governance & Company Secretarial teams at Payment Institutions operating internationally, it sits squarely in scope for board-level reporting, regulatory engagement calendars, and internal audit frameworks. Across the single finding tested in this cell, AI assistants produced a materially incorrect answer, getting the assessment's timeline wrong in a way that would corrupt any methodology note, regulatory submission, or senior management briefing built from it.
The failure was a temporal truncation: AI tools described the assessment as running through 2023–24, omitting the April 2025 findings-sharing and validation phase that is explicitly documented in the published report. A Governance & Company Secretarial team relying on that answer would produce an internal record that misrepresents when the assessment concluded and how FMIs were engaged, a factual error the regulator can check against its own published timeline.
How AI gets this regulation wrong
The AI failure on this regulation is temporal: AI assistants presented outdated information about the assessment's lifecycle as if it were the complete and current picture. The table below shows where that pattern manifested, and how it maps to the kind of work-product a Governance & Company Secretarial team would be producing when they ask AI to help frame an engagement with CPMI-level oversight outputs.
| AI's Failure Mode | Count | Affected findings |
|---|---|---|
| Outdated | 1 | Finding#1 |
What that means for your team
For Governance & Company Secretarial teams at Payment Institutions, the dominant risk here is the wrong deliverable: a methodology note, board paper, or regulatory engagement summary built on AI output that cuts the assessment's timeline short. The table below maps how that materialises across specific internal processes, and why catching it after the fact is considerably more costly than verifying the source before the document leaves the function.
| Risk Impact | Count | Affected findings |
|---|---|---|
| Wrong deliverable | 1 | Finding#1 |
When this affects your department
Governance & Company Secretarial teams at Payment Institutions reach for AI assistance on this assessment in at least two recurring contexts. The first is internal documentation: methodology notes and regulatory engagement summaries drafted to inform senior management or the board on where CPMI-level oversight stands, what assessments are live, when they ran, which FMIs were in scope, and what obligations flow from the outputs. The second is external-facing work: formal responses to regulators, due-diligence packs for counterparties, and audit evidence files where the firm needs to characterise its understanding of the regulatory landscape it operates in.
Both use cases treat AI output as a research shortcut, and both would inherit any factual error in that output.
The specific failure mode here, AI truncating the assessment's timeline to 2023–24 rather than the published 2023–25, is exactly the kind of error that looks authoritative in a first draft. The AI's answer is partially correct (data collection did run in that window) but incomplete: it omits the findings-sharing and follow-up engagement phase that ran to April 2025. A methodology note that states the assessment concluded in 2024 is factually wrong, and if that note goes to a regulator or into a board pack under the firm's letterhead, the error is the firm's, not the AI's.
For a Payment Institution operating internationally, the stakes are compounded by the J1 nature of CPMI oversight, these are global standards that domestic regulators in multiple jurisdictions will reference in their own supervisory engagement. A Governance & Company Secretarial team that mischaracterises the assessment's scope or timeline risks providing inconsistent information across jurisdictions, undermining the firm's credibility in regulatory dialogue at precisely the point where the firm should be demonstrating command of the frameworks it is subject to.
The findings at a glance
The table below summarises the finding tested in this cell, the question asked, how AI tools responded, and the label applied to the failure.
| # | Finding title | Type | Citation ID |
|---|---|---|---|
| 1 | Assessment timeline truncated, 2023–24 stated, 2023–25 correct | Hallucination | RLB-F-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q005 |
Aggregate impact
The single finding in this cell is a clean example of AI confident on the broad shape of a regulatory process but wrong on a specific factual detail, the endpoint of the assessment timeline. The AI described the CPMI-IOSCO Level 3 assessment as running during 2023–24; the published document states it ran through 2023–25, with further rounds of FMI engagement and follow-up questions extending to April 2025. The AI's answer was not a random fabrication, it appears to have drawn on secondary sources or earlier coverage that described only the data-collection phase.
That makes it harder to catch: the answer looks well-grounded and internally consistent, and a junior team member with no prior familiarity with the published report would have no obvious reason to question it.
For a Governance & Company Secretarial function at a Payment Institution, the practical impact clusters on methodology notes and regulatory engagement summaries, exactly the documents that characterise the firm's regulatory awareness to external audiences. A methodology note that states "the assessment was carried out during 2023–24" is directly contradicted by the BIS publication, and any regulator or counterparty with access to that document can identify the discrepancy immediately. The finding is labelled Contradictory because the AI's claim is falsifiable against the primary source.
The systemic risk here is not that a single paper gets a date wrong. It is that teams using AI to accelerate the production of regulatory documentation tend to do so precisely when they are under time pressure, quarterly board packs, regulatory response windows, audit cycles. Under time pressure, the verification step that would catch this error is the one most likely to be skipped. A Governance & Company Secretarial function that has not built source-verification into its AI-assisted drafting workflow for BIS-level assessments is operating with a structural gap in its quality controls for international regulatory documentation.
What your team should do
The default position for Governance & Company Secretarial teams at Payment Institutions should be: do not use AI output as the primary source for any factual characterisation of a CPMI-IOSCO assessment, its timeline, scope, methodology, or participating FMI count, in a document that will be reviewed by a regulator or senior governance body. For this assessment specifically, the BIS publication is publicly available and the relevant details (assessment period, data collection windows, FMI engagement phases) are stated plainly in the document.
The correct process is to draft from the primary source, not to draft from AI and then try to verify retrospectively.
Where AI tools are appropriate in this workflow is in the earlier, lower-stakes stages: summarising the general background on Principle 15 obligations, drafting a first-pass structure for a methodology note, or generating a list of questions the note should answer. Those are tasks where a factual error in AI output does not immediately corrupt a deliverable, the team is still in drafting mode and the substance will be filled in from primary sources.
The risk concentrates when AI is used to answer specific factual questions ("when did the assessment run?", "how many FMIs responded?") and the answer is taken directly into a final document without independent verification.
Practically, the safeguard is a single-step check: for any factual claim about a named CPMI-IOSCO assessment in a governance document, require a direct citation to the BIS publication page or report section before the document exits the function. This is not a heavy lift, the BIS portal is well-indexed and the Level 3 report is publicly accessible.
The cost of the check is minutes; the cost of an unchecked error in a board paper or regulatory submission is considerably higher, particularly for a Payment Institution operating across multiple jurisdictions where the same document may be reviewed by different regulatory authorities with independent access to the primary source.
How RLB Can Help
RegLeg's published Hallucination Research is available free of charge and serves as a practical pre-flight check for Governance and Company Secretarial teams at payment institutions before placing reliance on AI-assisted output on regulatory questions. The research identifies specific failure modes, including confidently stated but inaccurate procedural rules, misattributed board obligations, and outdated licensing thresholds, that arise when AI tools are applied to payment regulation. Reviewing the relevant findings before drafting board papers, statutory filings, or regulatory correspondence takes minutes and materially reduces the risk of importing an AI error into a document that carries the firm's name.
Beyond the published research, RegLeg works directly with payment institution governance teams on bespoke regulator deep-dives that map AI-supported workflows to their actual hallucination exposure. This typically covers the workflows where Governance and Company Secretarial functions most commonly turn to AI tools: drafting board minutes and committee terms of reference, tracking regulatory change for director briefings, preparing licensing and authorisation submissions, and maintaining the corporate record against evolving jurisdictional requirements. The output is a prioritised exposure map specific to the firm's operational footprint and the regulators it faces, not a generic framework.
Where a firm already has an AI-use policy in place, RegLeg offers a confidential review against our failure-mode catalogue, with prioritised remediation recommendations the Governance and Company Secretarial team can act on without external disclosure. We also develop training material and CPD-aligned content tailored to the governance function, giving the team a shared reference point for where AI tools can be used with confidence, where additional verification steps are warranted, and how to document that judgement in a way that satisfies regulatory expectations around AI governance.