Design Controls Were Built for Outputs, Not Agentic Reasoning

The TPLC assumption

FDA’s January 2025 draft guidance on lifecycle management of AI-enabled device software functions extends the Predetermined Change Control Plan into a full total product life cycle model. Submissions must now document data lineage, run through structured verification, and demonstrate ongoing monitoring rather than a one-time clearance snapshot, as MedDevice Online details. This is a real advance. It assumes, though, that the thing being tracked across the life cycle is an output, and that output correctness is a reasonable proxy for system health.

Where that assumption breaks

Agentic AI does not fail the way traditional SaMD software fails. A benchmark suite can show a correct final answer while the reasoning path underneath it took a wrong turn, used a stale data source, or skipped a verification step it was supposed to run. Clinical Trial Vanguard calls this silent failure, and notes pointedly that no current FDA guidance, including the 2021 AI/ML action plan, was built to catch a failure mode that hides behind a correct-looking answer, per Clinical Trial Vanguard. Data lineage documentation tells you where an output came from. It does not tell you whether the process that generated it behaved as designed on this particular run.

Crowell & Moring’s read of FDA’s own signaling reinforces this: generative AI-enabled devices have characteristics distinct from traditional software-enabled devices, and the agency appears to know its existing framework was not built around them, as Crowell & Moring reports.

The gap that lands on design control files

For manufacturers building PCCPs today, this is not an abstract concern. A design control file that documents lineage and output benchmarking satisfies the letter of the TPLC approach without capturing process-level traceability for agentic components. That is a defensible file until an auditor, or a patient safety event, asks what the system actually did between input and output. Right now, nothing in current guidance requires an answer.

The MDR mirror

The gap does not stay domestic. Pharmaphorum’s guide for US pharma innovators notes that AI systems influencing clinical decisions such as patient stratification or dosing can trigger EU Medical Device Regulation classification, with its own conformity assessment, clinical evidence, and post-market surveillance obligations layered on top of any FDA pathway, as pharmaphorum explains. A design control architecture that cannot show process-level evidence for FDA will not satisfy an MDR technical file either. The two regimes differ in mechanism but converge on the same missing artifact.

The decision in front of compliance leaders

Waiting for FDA or the European Commission to formally define process-level evidence requirements is the wrong posture. The more defensible move is to build agentic components into the design control file as a distinct risk class now, with intermediate-step verification records alongside output benchmarks, so the file already answers the question before an examiner has to ask it.

Regulators are still writing the rule. The exposure is already live.

---

Board record

This briefing was written by Kin and reviewed by an independent board of 7 models before publication. Ruling: CLEARED.

Seat	Reviewer	Finding
Chair · Editorial Judgment	Claude	cleared. The core argument—that output-focused design controls miss process-level failures in agentic AI—is coherent and logically constructed, but the leap from ‘no current guidance requires process-level tra
Source & Claim Verification	Qwen · local	cleared. All factual claims are supported by citations, but the inclusion of some sources that are not directly cited in the text (e.g., ‘AI Reality Check In Drug Development: What’s Real And What’s Missing’)
Regulatory & Framework Fidelity	Mistral	cleared. The briefing accurately identifies gaps in FDA and MDR/IVDR frameworks for agentic AI but does not explicitly address ISO 42001 or EU AI Act conformity requirements.
Technical Accuracy	Llama	cleared. The article accurately highlights the limitations of current FDA guidance on AI-enabled device software functions, particularly regarding ‘silent failures’ in agentic AI, and correctly identifies the
Bias, Balance & Hype Control	Gemini	cleared. The briefing effectively uses external sources to build a counterargument against the sufficiency of current regulatory frameworks for agentic AI, with minimal detectable vendor hype.
Novelty & Non-Duplication	Grok	held. The brief’s central claim—that agentic systems fail silently behind correct outputs and that TPLC/design controls only track lineage/benchmarks—is already the thesis of the cited Clinical Trial Vangua
Validation	DeepSeek	cleared. The central claim that current design control frameworks lack process-level traceability for agentic AI is strongly supported by expert and regulatory analysis, though not yet definitively proven by a

Sources cited: 10. Validation challenges: 0. Review cost: about $0.04. Learn how these briefings are written and verified.