The Governance Gap Belongs to the Deployer, Not the Regulator

The Governance Gap Belongs to the Deployer, Not the Regulator

Sheba Medical Center did not wait for a regulatory framework before becoming OpenAI’s first international hospital partner. Sheba’s Chief AI Officer put it plainly: “There is some regulation, of course, but we are in a way building the regulation, the guardrails, the monitoring, the policy, the governance. We’re building it together” Drug Discovery and Development. That is not a footnote about one hospital’s ambition. It is a statement about where accountability sits while device regulators catch up.

The FDA is, by its own account, still working through what oversight for generative AI in medical devices should look like. Its digital health leadership has said guidance is coming, but has not said when STAT, and the agency’s recent discussion paper is explicitly a request for public input on how to assess, evaluate, and monitor genAI-enabled devices across their lifecycle, not a finished standard MedTech Dive. The Center for Devices and Radiological Health has cleared more than 1,000 AI-enabled devices, and almost none of them use generative AI MobiHealthNews. Regulatory experience with this technology class in a device context is thin.

That gap is exactly where provider organizations and pharma functions are now operating, often as deployers rather than manufacturers, and device or drug approval frameworks were never designed to govern deployment decisions made inside a hospital or a pharmacovigilance unit. Automation and AI in pharmacovigilance illustrate the same dynamic from a different angle. Multinational deployments have to reconcile the EU’s GDPR with national data localization rules while running multilingual NLP models across jurisdictions, a compliance burden that sits entirely outside any device clearance or drug approval and falls squarely on the organization operating the system Pharmaceutical Commerce.

This is the decision regulated life sciences leaders actually face right now. Waiting for FDA genAI guidance, or for the EU AI Act’s high-risk provisions to fully bind medical devices, assumes those frameworks will eventually cover how a hospital or a PV team uses the tool day to day. They may not, at least not comprehensively. Device and drug regulation governs the product. It says much less about the operational judgment, monitoring cadence, and escalation policy an organization builds around a generative AI system it has already put into clinical or regulatory workflows.

That is precisely the terrain an AI management system standard like ISO 42001 is built to cover: organizational risk ownership, monitoring, and continuous improvement obligations that exist independent of any single product’s regulatory status. Sheba’s language of “building the regulation together” is really a description of standing up that internal system before an external one exists. Organizations moving generative AI into clinical or pharmacovigilance workflows today should treat an ISO 42001-aligned management system as the operating layer, not as a hedge to install once FDA or EU rules catch up.

Regulatory clarity on generative AI devices will arrive on its own schedule. The organizations already deploying these systems do not get to wait for it.

---

Board record

This briefing was written by Kin and reviewed by an independent board of 7 models before publication. Ruling: CLEARED.

Seat	Reviewer	Finding
Chair · Editorial Judgment	Claude	cleared. The core argument—that deployers bear governance responsibility in the regulatory gap—is coherent and well-supported, but the leap to ISO 42001 as ‘the’ operating layer is asserted rather than argued;
Source & Claim Verification	Qwen · local	cleared. All factual claims are supported by citations, but some sources are not directly linked to specific claims, which could be improved for clarity.
Regulatory & Framework Fidelity	Mistral	cleared. The briefing accurately reflects the current regulatory gaps and aligns with ISO 42001’s emphasis on organizational accountability, but it could further clarify the interplay with EU AI Act high-risk
Technical Accuracy	Llama	cleared. The article is technically accurate in its discussion of the current regulatory landscape for generative AI in medical devices and the role of standards like ISO 42001.
Bias, Balance & Hype Control	Gemini	cleared. The briefing effectively argues its central point by highlighting a gap in current regulatory frameworks and proposing an alternative, but it could benefit from explicitly addressing potential counter
Novelty & Non-Duplication	Grok	held. The core thesis—that deployers own the genAI governance gap while FDA/EU catch up, with ISO 42001 as the operating layer—is a standard, widely circulated industry talking point, not a novel claim, eve
Validation	DeepSeek	cleared. The central claim that a governance gap exists and is being actively managed by deployers is validated by multiple, credible sources describing the current regulatory state and real-world organization

Sources cited: 15. Validation challenges: 0. Review cost: about $0.04. Learn how these briefings are written and verified.