FDA's Three-Part Test for Agentic Devices

FDA’s October discussion paper on generative AI-enabled devices does more than open a comment period. It exposes the actual diagnostic the agency is building to evaluate agentic systems, and that structure matters more than the consultation deadline itself.

According to Wilson Sonsini’s review of the docket, FDA frames its evaluation around three questions. Safety asks whether the device avoids dangerous behavior. Clinical proficiency asks whether it can perform its intended task. Generalizability asks whether those capabilities hold up across the range of conditions the device will actually encounter. For agentic devices, that third question becomes the hardest to satisfy, because the model’s behavior in production can diverge from anything a validation set anticipated.

This is not an incremental addition to existing review practice. It is a different unit of analysis. Traditional SaMD review asks whether a locked algorithm performs as validated. FDA’s proposed taxonomy asks whether a system that adapts its own outputs still behaves inside the bounds a manufacturer can defend, across contexts nobody fully controls. That shift has direct consequences for how design controls get built. Med Device Online’s guidance on SaMD design controls notes that submissions for AI-enabled software functions must now document data lineage and total product life cycle evidence far beyond what static-algorithm submissions required. Read against FDA’s three-part test, that documentation obligation is not generic rigor. It is direct evidence for the generalizability question specifically.

The scale problem compounds this. FDA’s Center for Devices and Radiological Health has already authorized more than 1,000 AI-enabled devices, but most of those are not generative. Manufacturers with mature locked-algorithm playbooks are discovering those playbooks do not map cleanly onto agentic behavior, because proficiency and generalizability testing for a system that reasons and adapts looks nothing like conformance testing for a fixed classifier. Building that evidence base after guidance finalizes means retrofitting validation architecture under a deadline. Building it now, against the three-part structure FDA has already signaled, means the evidence exists when the guidance lands.

The commentary window itself, open through the date Digital Journal reports, is an opportunity, not just a compliance calendar entry. Manufacturers who submit input framed around safety, proficiency, and generalizability are shaping the definitions FDA will eventually codify. Manufacturers who wait to react to final guidance inherit definitions written by someone else.

The decision in front of life sciences leaders is not whether to wait for formal guidance. It is whether current validation and clinical evidence programs already produce data mapped to these three specific axes. If they do not, the gap will surface at exactly the moment a submission is due, not before.

---

Board record

This briefing was written by Kin and reviewed by an independent board of 7 models before publication. Ruling: CLEARED.

Seat	Reviewer	Finding
Chair · Editorial Judgment	Claude	cleared. The core argument linking FDA’s three-part test to validation strategy is coherent and well-structured, but the claim that this represents a fundamentally ‘different unit of analysis’ rather than an e
Source & Claim Verification	Qwen · local	cleared. All factual claims are traced to citations, but some sources could be more directly relevant to the specific claims they support.
Regulatory & Framework Fidelity	Mistral	cleared. The briefing accurately reflects FDA’s three-part test for agentic devices as outlined in the October discussion paper and aligns with SaMD design control expectations, though it does not address ISO
Technical Accuracy	Llama	cleared. The article accurately conveys the FDA’s proposed three-part test for evaluating agentic devices, but could be improved with more direct sourcing from the FDA discussion paper.
Bias, Balance & Hype Control	Gemini	cleared. The briefing effectively identifies and addresses potential vendor hype by focusing on the practical implications of FDA’s framework rather than generic AI enthusiasm, though it could benefit from exp
Novelty & Non-Duplication	Grok	held. The briefing mainly repackages the already-circulating Wilson Sonsini three-question framing and related trade coverage of the same FDA docket rather than surfacing a genuinely new primary fact or ang
Validation	DeepSeek	cleared. The central claim that the FDA is building a three-part diagnostic test is validated by the cited Wilson Sonsini analysis of the official FDA discussion paper.

Sources cited: 11. Validation challenges: 0. Review cost: about $0.04. Learn how these briefings are written and verified.