Finding#1, SARB pre-validation partnership downgraded to speculation

The model retrieved the SARB-CPMI pre-validation partnership context — it had enough signal to know the question was about a named central-bank collaboration on a specific recommendation — but downgraded a regulator-confirmed fact to speculative hedge phrasing ('plausible but unverified'). This is calibration drift in a direction that looks like safety: the model hedged where it should have committed. The training and retrieval pipeline appears to penalise commitment without verifying retrieval, producing under-confidence on facts the regulator has itself published.

For evaluators, the high-value probe is the asymmetry: when the AI has retrieved enough to identify the entity at issue (SARB, payment pre-validation, CPMI Brief No. 9), commitment behaviour should match retrieval depth rather than defaulting to hedge. A confidence-calibration eval that scores against regulator-published source text would surface this drift before it ships to production users.

Sonnet 4.6 with web search returned a confident negative — 'available sources do not identify SARB as a named pilot partner' — when CPMI Brief No. 9 (November 2025) explicitly does name SARB. The failure mode is a false-negative retrieval gap presented as a positive knowledge claim. The model's web-search loop either did not surface CPMI Brief No. 9 or surfaced it and did not extract the SARB identification from it; in either case the model treated the absence-of-retrieval as evidence-of-absence rather than as a retrieval-coverage limitation.

For an AI lab, this is a high-value alignment probe: confident negatives on entity-level regulatory questions — 'no named partner exists', 'no specific date is published' — should be evaluated against a corpus of regulator-published material to determine the false-negative rate. The same retrieval pattern almost certainly produces similar false-negatives across other regulator briefs.