●New: The Enterprise AI Evaluation Playbook (2026 edition) is out — download it free.
Qapitol
← All research

Qapitol Research · First edition

The State of AI Assurance in Healthcare 2026

A data-driven briefing for regulated healthcare enterprises on where AI governance, regulatory compliance, and assurance infrastructure stand today — and what budget-holders must do before the next enforcement cycle closes.

June 2026·Designed PDF · 31 pages·Free with email
Download the report →

Executive summary

  • Healthcare AI has crossed from experimentation into operational dependency — yet the assurance infrastructure needed to govern it safely remains critically underdeveloped. Only 22% of hospitals report high confidence they could produce a complete, auditable AI explanation within 30 days to a regulator or payer, while 88% have incomplete or missing centralized AI inventories and 84% do not capture human overrides of AI outputs. These are not aspirational gaps; they are audit failures waiting to be triggered by a regulatory cycle that is already in motion.
  • The policy environment has hardened materially since 2024. FDA's PCCP guidance (August 2025) now requires AI-enabled device manufacturers to pre-specify what will change and how it will be validated. CMS has confirmed that Medicare Advantage organizations using AI in coverage determinations must satisfy individualized review and physician sign-off requirements. The proposed HIPAA Security Rule NPRM would mandate a comprehensive AI-technology inventory touching ePHI. And EU AI Act literacy and governance obligations are already live. Budget-holders who treat AI assurance as a future problem are accumulating regulatory liability in the present.

Headline prediction

By year-end 2026, converging FDA, EU AI Act, CMS, and HIPAA pressures will force the majority of health systems to treat AI assurance as a non-discretionary operating cost rather than a discretionary IT project — yet fewer than one in three has the foundational controls to comply today.

What this report covers

  • The current state of hospital AI governance maturity and the specific control gaps that create regulatory exposure
  • FDA's evolving framework for AI-enabled Software as a Medical Device (SaMD), including PCCP requirements and post-market surveillance weaknesses
  • EU AI Act compliance timelines and the integrated MDR/IVDR conformity pathway for MedTech manufacturers
  • CMS and payer-sector obligations governing AI in prior authorization and coverage determinations
  • HIPAA Security Rule modernization and its proposed AI-specific requirements for ePHI risk management
  • Explainability, fairness, and drift detection: the technical assurance standards emerging from FUTURE-AI, NIST AI RMF, and EU AI Act Article 15
  • AI assurance market sizing, investment trends, and the financial incentives now linking governance quality to insurance premiums

Download the report

Get “The State of AI Assurance in Healthcare 2026” — designed PDF, 31 pages

Free with your details. We’ll send the PDF to your inbox and tailor what we share next to your role.

No spam. We’ll only send research relevant to your role. Unsubscribe anytime.

Sources

  • Artificial Intelligence in Software as a Medical Device | FDA — https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-software-medical-device
  • Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions | FDA — https://www.fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predetermined-change-control-plan-artificial-intelligence
  • Predetermined Change Control Plans for Machine Learning-Enabled Medical Devices: Guiding Principles | FDA — https://www.fda.gov/medical-devices/software-medical-device-samd/predetermined-change-control-plans-machine-learning-enabled-medical-devices-guiding-principles
  • FDA Regulation of AI-Enabled Devices - EveryCRSReport.com — https://www.everycrsreport.com/reports/IF13245.html
  • AI Meets HIPAA Security: Understanding HHS’s Risk Strategies and Proposed Changes - Lexology — https://www.lexology.com/library/detail.aspx?g=c0c12d96-21d0-4a2c-99f1-60bf5ce415a5
  • The Proposed HIPAA Security Rule Update: What It Would Change and How to Prepare — https://compliancy-group.com/proposed-hipaa-security-rule-update-2026/
  • Hospital Trends in the Use, Evaluation, and Governance of Predictive AI, 2023-2024 - ONC Health IT Research & Analysis — https://healthit.gov/data/data-briefs/hospital-trends-use-evaluation-and-governance-predictive-ai-2023-2024/
  • Advancing healthcare AI governance through a comprehensive maturity model based on systematic review | npj Digital Medicine — https://preview-www.nature.com/articles/s41746-026-02418-7
  • New Healthcare AI "Operational Control-Plane" Benchmark Finds Governance Readiness Lagging Behind Deployment Velocity | Morningstar — https://www.morningstar.com/news/accesswire/1130933msn/new-healthcare-ai-operational-control-plane-benchmark-finds-governance-readiness-lagging-behind-deployment-velocity
  • U.S. Hospitals Underfund AI Governance as Adoption Accelerates — https://www.accessnewswire.com/newsroom/en/healthcare-and-pharmaceutical/u.s.-hospitals-underfund-ai-governance-as-adoption-accelerates-1100555
  • Regulation of AI in Prior Authorization and Claims Review: A Look at Federal and State Consumer Protections | KFF — https://www.kff.org/patient-consumer-protections/regulation-of-ai-in-prior-authorization-and-claims-review-a-look-at-federal-and-state-consumer-protections/
  • MEMORANDUM — https://content.naic.org/sites/default/files/inline-files/Health%20Survey%20Memo%20to%20BDAIWG%2005092025%20-%20Final_1.pdf
  • SECTION 1: INTRODUCTION, BACKGROUND, AND LEGISLATIVE AUTHORITY — https://content.naic.org/sites/default/files/national_meeting/Materials-Att-3A-2023-11-22-Model-Bulletin-NM-Draft_Tracked-Changes.pdf
  • NAIC Intensifies AI Regulatory Focus: What Health Insurance Payors Need to Know | Crowell & Moring LLP — https://www.crowell.com/en/insights/client-alerts/naic-intensifies-ai-regulatory-focus-what-health-insurance-payors-need-to-know
  • Medicare advantage becoming a disadvantage with use of artificial intelligence in prior authorization review | npj Digital Medicine — https://www.nature.com/articles/s41746-026-02387-x
  • CMS confirms Medicare Advantage organizations may use AI in… — https://www.reedsmith.com/articles/cms-confirms-medicare-advantage-organizations-may-use-ai-in-making-coverage/
  • Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions | FDA — https://www.fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predetermined-change-control-plan-artificial-intelligence
  • Article 82: Compliant AI systems which present a risk | AI Act Service Desk — https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-82
  • FDA AI Medical Devices: 2026 Guidance, PCCP, and EU AI Act Comparison - Reg Intel — https://reg-intel.com/fda-ai-medical-devices-2026-guidance-pccp-and-eu-ai-act-comparison/
  • EU AI Act for Medical Devices: SaMD Compliance Deadlines & Requirements — https://mdxcro.com/eu-ai-act-medical-devices-samd/
  • What Is EU AI Act? Medical Device Compliance Guide — https://www.complizen.ai/post/what-is-eu-ai-act-medical-device-compliance-guide
  • Regulatory Insights From 27 Years of Artificial Intelligence/Machine Learning–Enabled Medical Device Recalls in the United States: Implications for Future Governance - PMC — https://pmc.ncbi.nlm.nih.gov/articles/PMC12274014/
  • Early Recalls and Clinical Validation Gaps in... : JAMA Health Forum — https://www.ovid.com/journals/jahf/fulltext/10.1001/jamahealthforum.2025.3172~early-recalls-and-clinical-validation-gaps-in-artificial
  • More than algorithms: an analysis of safety events involving ML-enabled medical devices reported to the FDA - PMC — https://pmc.ncbi.nlm.nih.gov/articles/PMC10280342/
  • Artificial intelligence related safety issues associated with FDA medical device reports - PMC — https://pmc.ncbi.nlm.nih.gov/articles/PMC11615200/
  • Class 2 Device Recall Dexcom G7 Continuous Glucose Monitoring (CGM) System — https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfRes/res.cfm?id=213398
  • Class 2 Device Recall Philips IntelliSpace — https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfRes/res.cfm?id=212082
  • People process technology and operations framework for establishing AI governance in healthcare organizations | npj Digital Medicine — https://www.nature.com/articles/s41746-026-02419-6
  • Advancing healthcare AI governance through a comprehensive maturity model based on systematic review | npj Digital Medicine — https://preview-www.nature.com/articles/s41746-026-02418-7
  • Managing AI governance in healthcare: What boards need to know | Global law firm | Norton Rose Fulbright — https://www.nortonrosefulbright.com/en/knowledge/publications/61a8a7bf/managing-ai-governance-in-healthcare
  • Artificial Intelligence Governance in Health Systems: Systematic Review of Frameworks and Integrative Model Proposal — https://www.jmir.org/2026/1/e87448/PDF
  • Health Industry Cybersecurity — https://healthsectorcouncil.org/wp-content/uploads/2026/05/AI-Cyber-Governance-Framework-Implementation-Guide.pdf
  • The Responsible Use of — https://digitalassets.jointcommission.org/api/public/content/dcfcf4f1a0cc45cdb526b3cb034c68c2
  • FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare — https://media.tghn.org/medialibrary/2025/03/FUTURE-AI-_international_consensus_guideline_for_trustworthy_and_deployable_artificial_intelligence_in_healthcare.pdf
  • Frontiers | Secure healthcare data management using federated learning, blockchain, and explainable artificial intelligence: a systematic review — https://www.frontiersin.org/journals/digital-health/articles/10.3389/fdgth.2026.1871960/full
  • An ethics-informed computable audit framework for monitoring misdiagnosis risk in AI-assisted diagnosis | Scientific Reports — https://www.nature.com/articles/s41598-026-46652-1
  • Frontiers | Auditing fairness in clinical AI systems using provenance-based simulation: a comparative and regulatory perspective — https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2026.1756023/full
  • Frontiers | Explainable AI in healthcare: a systematic review of XAI use cases in imaging, diagnostics, and rehabilitation — https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2026.1749527/full
  • Artificial Intelligence Risk Management Framework (AI RMF 1.0) — https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf
  • Article 15: Accuracy, robustness and cybersecurity | AI Act Service Desk — https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-15
  • Federal Register, Volume 90 Issue 3 (Monday, January 6, 2025) — https://www.govinfo.gov/content/pkg/FR-2025-01-06/html/2024-30983.htm
  • Implementing the Health Insurance Portability and Accountability Act (HIPAA) Security Rule: A Cybersecurity Resource Guide — https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-66r2.pdf
  • Federal Register, Volume 88 Issue 186 (Wednesday, September 27, 2023) — https://www.govinfo.gov/content/pkg/FR-2023-09-27/html/2023-20955.htm
  • Cybersecurity in Medical Devices: Quality Management System Considerations and Content of Premarket Submissions | FDA — https://www.fda.gov/regulatory-information/search-fda-guidance-documents/cybersecurity-medical-devices-quality-management-system-considerations-and-content-premarket
  • AI In Healthcare Governance And Safety Market Size, Share & 2031 Growth Trends Report — https://www.mordorintelligence.com/industry-reports/ai-in-healthcare-governance-and-safety-market
  • The MedTech snapshot: Why “assurance” became the growth strategy in 2025 - vamstar — https://vamstar.io/newsroom/the-medtech-snapshot-why-assurance-became-the-growth-strategy-in-2025/
  • https://blackbookmarketresearch.com/uploads/pdf/Healthcare-IT-Capital-Signals-Q2-2026.pdf
  • Clinical GenAI Safety Monitoring Market Size, Share & Forecast to 2036 | FMI — https://www.futuremarketinsights.com/reports/clinical-genai-safety-monitoring-market
  • AI In Healthcare Governance and Safety Market Size to Touch USD 19.65 Billion by 2035 — https://www.marketstatsinsight.com/ai-in-healthcare-governance-and-safety-market-2/
  • 2026 Healthcare Industry Trends Report — https://www.svb.com/trends-insights/reports/healthcare-investments-and-exits/
  • Predetermined Change Control Plans (PCCPs) for Medical Devices: FDA Issues Draft Guidance — https://www.kslaw.com/attachments/000/012/033/original/ca082324.pdf?1724431749=
  • Good machine learning practice for medical device development: Guiding principles - Canada.ca — https://www.canada.ca/en/health-canada/services/drugs-health-products/medical-devices/good-machine-learning-practice-medical-device-development.html
  • FDA Proposes Regulatory Framework for Artificial Intelligence/Machine Learning Software as a Medical Device — https://www.kslaw.com/attachments/000/006/842/original/ca040919.pdf?1554821839=
  • Hospital Trends in the Use, Evaluation, and Governance of Predictive AI, 2023-2024 - ASTP Health IT Data Brief - NCBI Bookshelf — https://www.ncbi.nlm.nih.gov/books/NBK618497/
  • Federal Register, Volume 90 Issue 3 (Monday, January 6, 2025) — https://www.govinfo.gov/content/pkg/FR-2025-01-06/html/2024-30983.htm
  • Hospital Trends in the Use, Evaluation, and Governance of Predictive AI, 2023-2024 - ASTP Health IT Data Brief - NCBI Bookshelf — https://www.ncbi.nlm.nih.gov/books/NBK618497/
  • OCR director defends HIPAA updates: "The cost of doing nothing is very high" | TechTarget — https://www.techtarget.com/healthtechsecurity/feature/OCR-director-defends-HIPAA-updates-The-cost-of-doing-nothing-is-very-high
  • Hospital Trends in the Use, Evaluation, and Governance of Predictive AI, 2023-2024 - ASTP Health IT Data Brief - NCBI Bookshelf — https://www.ncbi.nlm.nih.gov/books/NBK618497/
  • New Healthcare AI "Operational Control-Plane" Benchmark Finds Governance Readiness Lagging Behind Deployment Velocity — https://www.accessnewswire.com/newsroom/en/healthcare-and-pharmaceutical/new-healthcare-ai-%22operational-control-plane%22-benchmark-finds-governa-1130933
  • Hospitals face AI governance gaps heading into 2026, report finds — https://www.beckershospitalreview.com/healthcare-information-technology/ai/hospitals-face-ai-governance-gaps-heading-into-2026-report-finds/
  • New Healthcare AI "Operational Control-Plane" Benchmark Finds Governance Readiness Lagging Behind Deployment Velocity — https://www.accessnewswire.com/newsroom/en/healthcare-and-pharmaceutical/new-healthcare-ai-%22operational-control-plane%22-benchmark-finds-governa-1130933
  • Hospitals face AI governance gaps heading into 2026, report finds — https://www.beckershospitalreview.com/healthcare-information-technology/ai/hospitals-face-ai-governance-gaps-heading-into-2026-report-finds/
  • Hospitals face AI governance gaps heading into 2026, report finds — https://www.beckershospitalreview.com/healthcare-information-technology/ai/hospitals-face-ai-governance-gaps-heading-into-2026-report-finds/
  • Hospitals face AI governance gaps heading into 2026, report finds — https://www.beckershospitalreview.com/healthcare-information-technology/ai/hospitals-face-ai-governance-gaps-heading-into-2026-report-finds/
  • Model - Innovation, Cybersecurity, and Technology (H) Working Group — https://content.naic.org/sites/default/files/inline-files/2023-12-4%20Model%20Bulletin_Adopted_0.pdf
  • https://calhospital.org/wp-content/uploads/2024/02/HPMS-Memo-FAQ-on-CC-and-UM-020624.pdf
  • Federal Register, Volume 88 Issue 70 (Wednesday, April 12, 2023) — https://www.govinfo.gov/content/pkg/FR-2023-04-12/html/2023-07115.htm
  • Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions | FDA — https://www.fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predetermined-change-control-plan-artificial-intelligence
  • The Implications and Scope of the NAIC Model Bulletin on the Use of AI by Insurers | Holland & Knight LLP - JDSupra — https://www.jdsupra.com/legalnews/the-implications-and-scope-of-the-naic-6389491/
  • Predetermined Change Control Plans for Medical Devices | FDA — https://www.fda.gov/regulatory-information/search-fda-guidance-documents/predetermined-change-control-plans-medical-devices
  • Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations — https://www.hhs.gov/guidance/sites/default/files/hhs-guidance-documents/FDA/guidance-ai-enabled-device-software-functions.pdf