This is the consolidated view of findings. Click 'see details →' on any item for the full details for each finding.
This finding implicates the model's source-attribution logic at the intra-ecosystem level: when the correct source and the asserted source are thematically adjacent outputs from the same organisation in the same year, the model's retrieval or generation step does not reliably distinguish between them. For labs with RAG or web-search integrations, this suggests the citation grounding layer needs finer-grained document-level anchoring, not just organisation- or topic-level matching — two 2018 CPMI outputs on related subjects should not be interchangeable in a citation.
see details →This finding points to a gap in how the model handles comparative questions spanning documents with a known temporal gap. The model's training data likely contains substantial commentary treating the 2016 guidance and 2018 FSB Cyber Lexicon as a coherent regulatory pair, which may have caused the model to elide the two-year gap. Post-training reward signals for regulatory comparison tasks should penalise responses that imply contemporaneous co-development between documents with materially different publication dates.
see details →This finding directly implicates the web-search tool's failure to surface a publicly announced BIS press release from May 2026. The model's training-era prior — that the 2016 guidance is the operative standard — was not overridden by retrieval, suggesting either that the search query did not return the relevant press release or that the model did not weight recent retrieval results against a strong training prior. This is a measurable failure of the retrieval-augmentation loop on a query type (current regulatory status of a named document) that is both common and high-stakes in financial-services deployments.
see details →This finding implicates the model's tendency to convert structural similarity into an explicit citation claim — a specific failure mode that is likely to recur on any regulatory document whose architecture mirrors a widely known framework. For labs building compliance or legal-research products, this pattern represents a systematic false-positive risk: the model will tell users that a regulation explicitly cites a framework when the evidence is structural resemblance only. Evals targeting explicit-citation claims, with ground-truth derived from the document text, would surface this class of error systematically.
see details →This finding points to a gap in the model's ability to distinguish the level of operational detail characteristic of a 2016 principles-based guidance document versus a 2020 operational-practices publication. The model populated the 2016 document's supposed content with material more consistent with the later FSB guidance, suggesting that its internal representation of the 2016 document is contaminated by subsequent regulatory outputs on the same topic. Synthetic training pairs that contrast high-level principles text with operational-detail text from a later document — with correct attribution — could help calibrate this boundary.
see details →This finding reveals that the model not only collapsed a temporal gap but asserted a specific causal relationship (that the FSB Lexicon drew on the CPMI-IOSCO definition) for which no evidential basis was found. This is a more advanced failure than simple conflation: the model constructed a plausible-sounding provenance claim that goes beyond what the documents support. This class of error — inferred causation stated as documented fact — is particularly hazardous in legal and compliance contexts and is likely to evade generic hallucination red-teaming that focuses on factual accuracy rather than provenance accuracy.
see details →Like the analogous Opus 4.7 finding, this result implicates the web-search integration's failure to surface recent regulatory announcements that would override a training-era prior. The phrase 'as of the date of this response' in the model's output is particularly significant: it signals that the model is attempting to hedge on currency but does so without actually checking — suggesting the hedging behaviour is a learned linguistic pattern rather than an operationally grounded check. A retrieval step that actively queries for amendment or consultation activity on named regulatory documents before answering status questions would address this gap directly.
see details →