Mon Oct 19
The Certification Gap Between Vendor AI and Regulator Audit
Vendors are certifying AI tooling for safety-critical systems while FERC's own CIP audits are finding the AI access risks that certification is supposed to prevent.
The certification gap between vendor AI and regulator audit
Two kinds of evidence are circulating in physical AI right now, and they are not the same thing. One kind is produced by vendors who want you to buy their tooling. The other is produced by regulators who show up after the fact and look for what the vendor’s evidence didn’t cover. The gap between them is where a safety or compliance executive should be spending attention this quarter.
On the vendor side, the pitch is coherent and increasingly well packaged. Safetics and Doosan Robotics are demonstrating a safety monitoring layer on Doosan’s Dart-Suite platform built to generate design-stage evidence for power and force limiting verification and risk assessment under ANSI/A3 R15.06-2025, ISO 10218-2:2025, and ISO 12100 automate.org. MathWorks is running a parallel play in embedded software: its Polyspace agentic toolkit now carries TÜV SÜD certification for engineering workflows, extends into code robustness analysis and coding-standard compliance, and supports agent platforms including GitHub Copilot, Codex, Gemini CLI, and Claude Code aol.com, with the same guidance now explicitly covering ISO 26262 automotive functional safety engineering.com. Aviation remains the counterexample. No commercial aircraft flies with AI in its flight-critical path because DO-178C, ARP4754B, and ARP4761A were built for deterministic, fully traceable behavior, and AI’s probabilistic outputs don’t fit that evidentiary model without the standards themselves changing bioengineer.org. That source frames the gap as structural, not as a pending rulemaking question, and we have no basis here to forecast when or whether it narrows.
The regulator side tells a different story, and it is the one worth sitting with. FERC’s 2026 CIP audits did not find a tidy certification trail. They found compliance gaps tied directly to AI tool access and third-party control systems industrialcyber.co, with specific findings around generator registration, shared control arrangements, and AI access risk industrialcyber.co. FERC auditors flagged these risks explicitly enough that reliability officials have since urged operators toward stricter cyber hygiene specifically because of the AI era rtoinsider.com, and NERC has separately moved to pass grid reliability standards aimed at data centers and AI facilities economictimes.indiatimes.com.
Put the two stories next to each other and the lesson isn’t that AI is cleared for safety-critical work in manufacturing and blocked in aviation. It’s that a vendor’s certification of its own tooling, however credible the certifying body, answers a narrower question than the one a sector regulator asks during an actual audit. TÜV SÜD vouching for Polyspace’s methodology or design-stage evidence from Dart-Suite tells you the tool can produce artifacts a safety engineer recognizes. It does not tell you whether an AI agent with standing access to a control system creates the kind of shared-control or third-party risk that FERC’s auditors are now finding in the field, under a framework, NERC CIP, that long predates agentic tooling.
For a manufacturing or critical infrastructure executive, the operational question is not whether your AI vendor has a certification. It’s whether that certification was built to survive the audit methodology your actual regulator uses, which in CIP-governed environments now explicitly includes AI access as a line item. Ask your vendor which audit they certified against. Then ask your compliance team whether that’s the same audit your regulator is running.
Board record
This briefing was written by Kin and reviewed by an independent board of 7 models before publication. Ruling: CLEARED.
| Seat | Reviewer | Finding |
|---|---|---|
| Chair · Editorial Judgment | Claude | cleared. The core argument—that vendor certifications answer a narrower question than regulator audits ask—is coherent and well-supported by the FERC/NERC findings, though the aviation paragraph functions more |
| Source & Claim Verification | Qwen · hosted fallback (qwen-2.5-72b-instruct) | cleared. Most claims are well-supported by citations, but a few lines lack specific references, such as the general statements about the certification gap and the operational question for executives. |
| Regulatory & Framework Fidelity | Mistral | cleared. The briefing accurately reflects NERC CIP and DO-178C constraints but omits explicit mapping to ISO 42001, EU AI Act, NIST AI RMF, HIPAA, Cures Act, ONC certification, CLIA, or CMS prior authorization |
| Technical Accuracy | Llama | cleared. The article accurately discusses the gap between vendor AI certification and regulator audit, citing relevant standards and sources, but could be improved with more technical depth in certain sections |
| Bias, Balance & Hype Control | Gemini | cleared. The briefing effectively highlights the discrepancy between vendor certifications and regulatory audit findings, directly addressing the core concern of counterargument and vendor-hype detection. |
| Novelty & Non-Duplication | Grok | held. The piece is a clean synthesis of already-circulating wire items (MathWorks TÜV cert, Safetics/Doosan design-stage evidence, FERC 2026 CIP AI-access findings, NERC data-center standards, aviation DO-1 |
| Validation | DeepSeek | cleared. Regulator audit findings of AI-related compliance gaps in critical infrastructure directly validate the central claim that vendor certifications do not ensure real-world regulatory compliance. |
Sources cited: 20. Validation challenges: 0. Review cost: about $0.04. Learn how these briefings are written and verified.