Integrating AI in Healthcare: Bridging the Gap in Validation

Recent research reveals that while the majority of health systems have integrated third-party AI tools into their operations, less than half possess the necessary infrastructure to rigorously test and validate these technologies prior to their implementation in patient care settings.

The adoption of artificial intelligence (AI) in healthcare is rapidly increasing, with hospitals across the United States deploying AI tools to enhance clinical and administrative tasks. However, this swift integration often outpaces the infrastructure and governance needed to effectively manage these technologies. A report by the Center for Connected Medicine at UPMC and KLAS Research highlights a concerning gap: fewer than 50% of health systems have established environments for testing AI tools before integrating them into patient care.

AI in Healthcare: Development Status and Challenges

The report finds that approximately 63% of health systems describe their AI programs as either under development or managed on an ad hoc basis. This piecemeal approach is often a result of limited resources, time, and expertise required to develop a structured testing system. Ken Howard, Vice President of Technology Services Engineering at UPMC Enterprises, explains that hospitals frequently lack the necessary infrastructure to thoroughly validate AI solutions, leading them to rely on conventional IT implementation processes.

"Without having a dedicated or consistent test environment strategy, they’re going to go down that path just to learn that all that work potentially wasn’t justified."

Ken Howard, UPMC Enterprises

Governance and Validation Frameworks: A Case for Structured Testing

To address these issues, UPMC has taken proactive steps by developing Ahavi, a platform that allows health systems to validate third-party AI tools with non-identifiable patient data. Rob Bart, UPMC's Chief Medical Information Officer, stresses the need for continuous monitoring and governance, extending beyond the initial AI deployment. UPMC's formal AI governance framework, maintained for over two years, ensures ongoing evaluation of AI tools, addressing potential issues such as bias and model drift.

Industry Trends and Future Implications

The need for rigorous evaluation methods grows as AI adoption accelerates. Kate Eisenberg, Senior Medical Director at DynaMed, points out that clinician use of AI tools nearly doubled between 2023 and 2026, complicating the establishment of uniform evaluation standards. Eisenberg emphasizes the importance of considering equity in AI systems and training clinical teams to identify and mitigate bias in AI responses.

As health systems forge ahead with AI integration, the need for robust governance and standardized evaluation cannot be overstated. While individual systems currently shoulder the responsibility for AI regulation, collaboration across the industry will be crucial to develop consistent standards and ensure the responsible implementation of AI technologies in healthcare.