When Two Independent Checks Fail the Same Way: A Case Study

👁 114 reads

A paper by Fabio Rovai, accepted at the AI for Science workshop at NeurIPS 2026, examines a specific error in a data pipeline comparing two open registries of objects orbiting Earth. The author describes a failure mechanism he calls “common-mode failure.”

What is dual verification and why it failed this time

A standard safeguard in data pipelines is redundant computation: you calculate each published number using two different methods based on different technologies, and if the results differ, the pipeline stops. In the case described, one path used Python and set operations, the other used SPARQL queries to an RDF graph. The gate comparing both results printed the message ALL CROSS-CHECKS AGREE for seven checked values. Three of them were wrong. One of the incorrect values was inflated more than fourfold: the pipeline returned 932 instead of the correct 220.

Why didn’t verification catch this? Both paths imported the same constants, which encoded an incorrect reading of the status dictionary from the source document. Since the error was identical in both paths, the gate comparing results had no chance of noticing it - both results were equally wrong, so they mutually “confirmed” correctness.

The correction was also wrong

The author then checked the corrected code against the history of phases for each object, stored in a source file the pipeline never read. It turned out the correction also contained errors: 42 of 261 discrepancies were artifacts. None of the three control tests Rovai described detected these artifacts.

Experiment with language models

The author conducted a controlled replication with three language models with additional tools disabled. When asked to generate independent verification paths: 72 of 75 generated paths computed the same incorrect value as the original defect. Importantly, 29 of 30 paths computed the incorrect value even when the prompt contained status code definitions from the source’s own documentation. Rovai does not draw broader conclusions beyond the described case - he explicitly notes that the evidence concerns one pipeline and one family of defects.

Conclusion from the paper

The author puts it this way: redundancy verified the implementation, and the errors that reached publication were errors of meaning. The pipeline checked whether both paths gave the same result - and they did. It did not check whether the common assumptions both paths relied on were correct. Rovai provides the failure mechanism, a detailed object-level record reconciling each number, and three tests returning to source documentation, measured on both the faulty code and its correction. Code, prompts, raw model outputs, and records of individual runs are publicly available at the address indicated in the paper.

What this means

This case study challenges an assumption many engineers treat as obvious: that two different technical implementations of the same logic provide verification independence. Rovai shows that technical independence does not mean semantic independence - if the error lies in understanding the input data rather than in the code, no number of parallel paths will detect it. The result of the experiment with language models - 72 of 75 paths repeated the same error, even with definitions in the prompt - suggests that models may amplify this type of common-mode failure instead of catching it. The author himself cautions, however, that this is one case and one family of defects, so extrapolation should not go too far.

Source: arXiv cs.AI: Independent Verification Paths Are Not Independent: A Case Study of Common-Mode Failure in a Satellite Catalogue Pipeline (paper from 2026-09-30). This discussion is based on the full text of the paper, not reports from other media.

// read moreReal AI Case Studies: Incidents, Data, and LessonsState of AIBrain.fm Use Cases: Deep Work, Study, Open Office & SleepGuidesGoogle Earth AI Generator: 24 Hours That Cost Two DecadesMagazine

Kinetyra