The Governance Gap FDA's Rulemaking Won't Reach

The Governance Gap FDA’s Rulemaking Won’t Reach

FDA’s discussion paper on generative AI in medicine is explicitly not guidance. It poses open questions to physicians, manufacturers, and patients and invites comment before any rule takes shape FDA wants feedback on how to regulate generative AI in medicine. That sequencing matters more than it looks. It means the agency is still at the question-asking stage for a category of tools that hospitals are already running in production.

Sheba Medical Center’s partnership with OpenAI is the clearest evidence of that gap in practice. Sheba’s team describes building its own guardrails, monitoring, and policy in real time because established rules don’t yet reach what they’re deploying: “There is some regulation, of course, but we are in a way building the regulation, the guardrails, the monitoring, the policy, the governance,” said Sheba’s Akselrod-Ballin, “we’re building it together” How Sheba Medical Center became OpenAI’s first international hospital partner. That is a health system writing its own compliance architecture ahead of any regulator, not because it wants to, but because nothing else exists yet.

The exposure this creates isn’t primarily a device-classification problem. It’s an identity and inventory problem. Coverage of embedded, shadow AI in health systems points to generative tools operating inside clinical and administrative workflows without ever entering a formal AI or software inventory, which means no one can say with confidence what models touch patient data, where, or under whose account The Privacy Risks of Embedded, Shadow AI in Health. Renown Health’s move to rebuild its digital identity strategy is a direct response to that reality, treating identity and access governance as the control point when the AI layer itself is too diffuse to inventory tool by tool How Renown Health Is Reshaping Its Digital ID Strategy. And the broader security teams flagging this problem are candid that “emerging threats” from AI are becoming harder to prioritize precisely because they don’t map cleanly onto existing risk registers built for conventional software Why ‘Emerging Threats’ Are Harder to Prioritize in the AI Era.

For compliance leaders, the decision this raises is not whether to wait for FDA’s eventual generative AI framework. It’s whether to wait at all. Shadow AI already sits inside PHI-adjacent workflows today, governed by HIPAA and internal privacy policy regardless of whether any tool ever gets classified as a device. An ISO 42001-aligned inventory of every generative AI system touching clinical or patient data, bound to identity and access controls, is the only control that functions now, independent of where FDA’s rulemaking eventually lands.

Sheba’s candor about building the rules as they go is instructive precisely because most institutions won’t say it out loud. They’re doing it anyway.

---

Board record

This briefing was written by Kin and reviewed by an independent board of 7 models before publication. Ruling: CLEARED.

Seat	Reviewer	Finding
Chair · Editorial Judgment	Claude	cleared. The core argument—that a regulatory gap exists and institutions must self-govern now—is coherent and supported, but the piece conflates three distinct problems (FDA’s pre-rulemaking posture, shadow AI
Source & Claim Verification	Qwen · local	cleared. Most factual claims are supported by citations, but a few lines lack direct references, such as the discussion on Renown Health’s digital identity strategy and the broader security teams’ challenges w
Regulatory & Framework Fidelity	Mistral	cleared. The briefing accurately reflects the current regulatory gap for generative AI in healthcare but does not fully address specific ISO 42001, EU AI Act, or MDR/IVDR requirements beyond high-level alignme
Technical Accuracy	Llama	cleared. The article accurately describes the current regulatory gap and challenges related to generative AI in medicine, citing relevant sources and examples, with only minor technical inaccuracies or oversim
Bias, Balance & Hype Control	Gemini	cleared. The briefing effectively identifies and critiques the gap between regulatory pace and real-world AI adoption, with minimal vendor hype and a clear counterargument to waiting for FDA guidance.
Novelty & Non-Duplication	Grok	held. The piece mostly restates circulating wire items (FDA discussion-paper stage, Sheba building its own guardrails, shadow-AI inventory/identity gaps) into a familiar ‘don’t wait for the regulator’ frame
Validation	DeepSeek	cleared. The central claim that hospitals are deploying generative AI ahead of formal FDA rules is validated by the cited, real-world example of Sheba Medical Center building its own governance.

Sources cited: 14. Validation challenges: 0. Review cost: about $0.04. Learn how these briefings are written and verified.