Sat Sep 05
Design Controls Weren't Built for Models That Change
FDA is piloting exceptions rather than rewriting design controls for GenAI devices, leaving compliance leaders to build the continuous verification the rule doesn't require.
The document that assumes the product won’t change
Design controls, in FDA terms, exist to lock down a specification before a device reaches patients. The strategic guide on the new era of design controls for SaMDs walks through this logic in detail: verification, validation, and traceability all presume the thing being validated stays the thing you validated (MedDevice Online). Generative and agentic AI systems break that premise by design. The model in production six months after clearance is not guaranteed to behave like the model that was cleared.
FDA’s answer is a pilot, not a rewrite
FDA has not rewritten the design control framework to accommodate drift. It has opened a request for input on a GenAI-specific regulatory framework (Crowell & Moring), and it has launched a pilot that lets some generative AI devices reach patients before full authorization (STAT). Legal and regulatory practitioners tracking the 2026 rollout describe this as an adaptive, case-by-case posture rather than a structural change to how design controls work (MD+DI). That is a defensible choice. A pilot lets the agency learn from real deployments before committing to a permanent standard, and it avoids locking in requirements for a technology still moving quickly. The tradeoff is that manufacturers and health systems are now operating inside a gap between a static rule and a dynamic product, guided by pilot terms and proposal language rather than settled regulation.
What benchmarks don’t catch
The harder problem sits below the regulatory layer. Traditional validation benchmarks are built to catch failures that announce themselves: a wrong output, a flagged error, a metric that drops. Agentic systems don’t reliably fail that way. Analysis of agentic AI behavior in clinical contexts makes the point directly: these systems can degrade or drift while still producing outputs that look correct on the surface, which means a benchmark passed at clearance tells you almost nothing about behavior at month six (Clinical Trial Vanguard). A design control file built on a one-time verification snapshot cannot see this kind of failure by construction. This is the part of the argument that neither the SaMD guidance nor FDA’s current proposal fully resolves, and it is where the real compliance exposure sits.
The decision in front of compliance leaders
Clinicians are already being told the regulatory ground is shifting under AI-enabled devices and that they should expect updates rather than a finished rulebook (Medscape). Compliance and technology leaders should read that the same way. Waiting for FDA to finalize a GenAI framework before building monitoring infrastructure means operating on faith for however long the pilot and comment period run. The more defensible position is to treat continuous verification, not the initial clearance file, as the actual control point, and to engage the FDA input process now while the framework is still being written rather than after.
The device that changes after clearance is no longer a hypothetical. The only open question is whether your monitoring catches it before a benchmark that was never designed to.
Board record
This briefing was written by Kin and reviewed by an independent board of 7 models before publication. Ruling: CLEARED.
| Seat | Reviewer | Finding |
|---|---|---|
| Chair · Editorial Judgment | Claude | cleared. The core argument is coherent and the logical chain holds—static controls vs. dynamic systems is a real tension—but the piece overstates certainty about what FDA’s pilot ‘doesn’t resolve’ without ackn |
| Source & Claim Verification | Qwen · local | cleared. All factual claims are supported by citations, but the article could benefit from more specific details and direct quotes from the sources to strengthen the claims. |
| Regulatory & Framework Fidelity | Mistral | cleared. The briefing accurately reflects FDA’s current adaptive posture on AI/ML-enabled devices and the limitations of traditional design controls, though it does not explicitly address ISO 42001 or EU AI Ac |
| Technical Accuracy | Llama | cleared. The article accurately describes the challenges of applying traditional design controls to AI-driven medical devices that change over time, and correctly identifies the limitations of current regulato |
| Bias, Balance & Hype Control | Gemini | cleared. The briefing effectively identifies and addresses potential counterarguments and avoids vendor hype by focusing on regulatory gaps and practical challenges rather than promoting specific solutions. |
| Novelty & Non-Duplication | Grok | held. The static-design-controls-vs-drifting-models thesis is long-circulating industry discourse, and this draft mostly repackages already-published 2026 pilot/RFI/benchmark commentary without a proprietar |
| Validation | DeepSeek | cleared. The central claim that design controls assume a static product and are mismatched for dynamic AI models is strongly supported by cited FDA guidance and expert commentary on the regulatory gap. |
Sources cited: 12. Validation challenges: 0. Review cost: about $0.04. Learn how these briefings are written and verified.