The Next Safety Case Needs a New Kind of Evidence

The Next Safety Case Needs a New Kind of Evidence

Aerospace safety cases have always rested on a simple contract. A system behaves predictably under specified conditions, and the manufacturer can prove it through test data, failure mode analysis, and traceable design assurance. That contract is starting to strain, and not because autonomy is new. It’s because the evidence base regulators expect hasn’t caught up to what autonomous decision-making actually requires to be trustworthy.

Two threads outside aviation are showing where this is headed. MIT and Motional researchers built a system called CW-Net that translates a self-driving car’s internal reasoning into interpretable concepts, specifically so safety drivers and reviewers can understand why the vehicle made a given decision, not just what it did Times of India. Separately, the Robot Report is raising a harder question for physical AI generally: can a system stay safe when an attacker manipulates what it perceives, decides, or does, even when nothing looks wrong on the surface The Robot Report. Neither is an aviation story yet. Both describe evidentiary gaps that aviation autonomy programs will inherit directly.

Meanwhile the assurance process itself is under formal scrutiny. Aviation Week reports that the global framework aviation relies on to validate safety, the process regulators and manufacturers have trusted for decades, is now facing serious questions about its own adequacy Aviation Week. That scrutiny is happening at the same moment autonomy stacks are introducing decision logic that traditional test-and-inspect regimes were never built to interrogate.

Put these together and the decision facing aerospace leadership isn’t whether autonomy is coming. It’s whether their safety case can answer two questions a deterministic system never had to answer: can you explain why the system made this decision, and can you show it resists deliberate manipulation of its inputs. Right now, most vendor safety documentation answers neither. It documents performance under nominal conditions and calls that sufficient, the same way a Phenom 300EV certification package documents range, systems, and structural margins under EASA, FAA, and ANAC review Business Jet Interiors International. That model works when behavior is fully specifiable. It doesn’t work when the system is inferring.

For compliance and technology leaders evaluating autonomy suppliers, ISO 42001’s requirement for documented AI risk management and the EU AI Act’s emphasis on transparency and robustness for high-risk systems both point the same direction. Explainability artifacts and adversarial test results need to move from research novelty to procurement requirement now, not after a regulator writes it into an airworthiness directive. Waiting for that mandate means building the safety case backward, the way operators are already scrambling to satisfy the FAA’s inspection order on 1,069 Airbus airframes after entry into service rather than before democrata.es.

The manufacturers who ask vendors for interpretability and adversarial robustness evidence today will not be scrambling to retrofit a safety case when the standard catches up. They’ll already have the paperwork.

---

Board record

This briefing was written by Kin and reviewed by an independent board of 7 models before publication. Ruling: CLEARED.

Seat	Reviewer	Finding
Chair · Editorial Judgment	Claude	cleared. The core argument—that autonomy requires new evidence types (explainability, adversarial robustness) that traditional certification regimes don’t address—is logically coherent and well-supported, thou
Source & Claim Verification	Qwen · local	cleared. Most claims are well-supported by citations, but a few lines lack direct evidence, such as the assertion about the necessity of explainability artifacts and adversarial test results in procurement req
Regulatory & Framework Fidelity	Mistral	cleared. The briefing correctly identifies key requirements from ISO 42001 (risk management) and the EU AI Act (transparency/robustness) but lacks explicit mapping to FDA or MDR/IVDR frameworks, which are irre
Technical Accuracy	Llama	cleared. The article accurately highlights the need for new kinds of evidence in safety cases for autonomous systems, citing relevant research and regulatory developments, but could be strengthened with more t
Bias, Balance & Hype Control	Gemini	cleared. The briefing effectively integrates external sources to build its argument, demonstrating a good balance between its own assertions and supporting evidence, with minimal vendor hype.
Novelty & Non-Duplication	Grok	held. The core claim—that autonomy safety cases need explainability and adversarial-robustness evidence—is long-standing AI-assurance orthodoxy, and this draft mainly re-packages it around adjacent non-avia
Validation	DeepSeek	cleared. The central claim that traditional safety evidence is inadequate for autonomous decision-making is strongly supported by credible reports of scrutiny on existing assurance processes and emerging gaps

Sources cited: 8. Validation challenges: 0. Review cost: about $0.04. Learn how these briefings are written and verified.