The FDA is approving an average of 69 medical AI devices per year, but about half of those approvals lack clinical validation data according to an analysis published in Nature. The various uses for Artificial Intelligence in medical settings are rapidly growing, and the FDA only approved an average of two AI devices per year before 2016. The study authors wrote, “Devices that lack adequate clinical validation pose risks for patient care. A new validation standard is proposed to evaluate FDA authorization as an indication of clinical effectiveness in medical AI.”

Researchers Sammy Chouffani El Fassi and Gail E. Henderson led an analysis of over 500 AI medical devices to find that about half lacked clinical scientific data to prove the devices are effective for human use. The authors stated, “Although AI device manufacturers boast of the credibility of their technology with FDA authorization, clearance does not mean that the devices have been properly evaluated for clinical effectiveness using real patient data. With these findings, we hope to encourage the FDA and industry to boost the credibility of device authorization by conducting clinical validation studies on these technologies and making the results of such studies publicly available.”

144 of the devices were “retrospectively validated,” which means the AI is given image data from the past. A better scientific approach is prospective validation, which provides real-time patient data. 148 of the devices in the analysis were approved with prospective validation. 22 of the devices were approved with randomized control trials, which is the gold standard.

The study’s authors said there is a need for the FDA to clearly define and distinguish between the three types of validation. Chouffani El Fassi added, “We shared our findings with directors at the FDA who oversee medical device regulation, and we expect our work will inform their regulatory decision-making. We also hope that our publication will inspire researchers and universities globally to conduct clinical validation studies on medical AI to improve the safety and effectiveness of these technologies. We’re looking forward to the positive impact this project will have on patient care at a large scale.”

The AI medical device market will grow from $15 billion in 2023 to $22 billion in 2024. Estimations say it will reach $97 billion by the year 2028. An executive order from October 2023 lays out concerns and hopes for the development of AI technology for a wide variety of industries.

226 out of 521 authorizations by the FDA lacked validation data from real patients. Some of them contained created images rather than data from real patients.

That executive order states, “Artificial Intelligence must be safe and secure.  Meeting this goal requires robust, reliable, repeatable, and standardized evaluations of AI systems, as well as policies, institutions, and, as appropriate, other mechanisms to test, understand, and mitigate risks from these systems before they are put to use.”

In response to that executive order, the FDA Center for Drug Evaluation and Research (CDER) is forming a new council that will oversee all AI-related technologies within the FDA. This will replace the current AI-related steering committee and additional AI-related working groups.

The council is co-led by three individuals within the FDA. Tala Fakhour, the associate director for data sciences and AI policy, has previously worked for the CDC and ICF International, a consulting firm. Qi Liu, the associate director for innovation and partnership in the office of Clinical Pharmacology, formerly worked for Merck. Sri Mantha, the director of the Office of Strategic Programs, has previously worked for multiple pharmaceutical companies including Pfizer and AstraZeneca.

Chouffani El Fassi said, “We shared our findings with directors at the FDA who oversee medical device regulation, and we expect our work will inform their regulatory decision-making. We also hope that our publication will inspire researchers and universities globally to conduct clinical validation studies on medical AI to improve the safety and effectiveness of these technologies. We’re looking forward to the positive impact this project will have on patient care at a large scale.”

A 2023 research paper outlines several drawbacks for using AI in the medical field, including concerns related to ethics, society, and privacy. AI devices in healthcare require algorithmic models that use a significant amount of private patient data. Researchers have warned that this is a privacy risk because this data can potentially be hacked.

Last year, the World Health Organization (WHO) warned that the rapid adoption of AI technology for medical purposes can cause harm to patients. The WHO said, “Precipitous adoption of untested systems could lead to errors by healthcare workers, cause harm to patients, erode trust in AI and thereby undermine (or delay) the potential long-term benefits and uses of such technologies around the world.”

FDA Director Robert Califf warned last year about the healthcare industry being “swept up quickly by something that we hardly understand.” In March, Dr. Califf spoke at the Coalition for Health AI and raised similar concerns. Califf said, “My concern is that our health systems do not have the infrastructure and tools to make the most important determinations about whether an AI application is “effective” for health outcomes. The ability of algorithms to provide accurate assessments will drift if left untended, often in unpredictable, and sometimes dangerous ways.”

Califf added that it is the responsibility of the manufacturer to ensure products are safe with oversight from the FDA. He said it is unclear how this system can work effectively for AI models that change and adapt over time.

“Not surprisingly, I’m hearing that the “effectiveness” metric being used by health systems to make decisions about incorporating an AI implementation is a financial metric,” Califf said. “Will the algorithm improve the bottom line of the part of the health system making the purchase?  I worry that the main use of AI algorithms will be decisions that optimize the bottom line rather than optimizing the longevity and well-being of patients.  This is counter to the mission of the FDA, where effectiveness means an improvement in a health outcome.”

The HighWire has been reporting about the ongoing concerns with AI, including a conversation between Jeffery Jaxen and Host Del Bigtree about whether AI is an existential threat. In an earlier episode, the Jaxen Report included a warning by Professor Stephen Hawking that AI could destroy mankind. In June, Jaxen discussed AI technology that utilizes human brain tissue.

Steven Middendorp

Steven Middendorp is an investigative journalist, musician, and teacher. He has been a freelance writer and journalist for over 20 years. More recently, he has focused on issues dealing with corruption and negligence in the judicial system. He is a homesteading hobby farmer who encourages people to grow their own food, eat locally, and care for the land that provides sustenance to the community.

Other Headlines

Coronavirus

ICAN Releases Recordings of FDA’s Peter Marks; Vaccine Official Dismissed Vaccine Injuries

The Informed Consent Action Network (ICAN) and REACT 19 held a joint press conference today to reveal a trove of secret recordings and FOIA documents of Peter Marks, the former Director of the Center for Biologics Evaluation and Research. The press conference included ICAN Founder Del Bigtree, Attorney Aaron Siri, and REACT 19’s Brianne Dressen.Continue reading ICAN Releases Recordings of FDA’s Peter Marks; Vaccine Official Dismissed Vaccine Injuries

More news about Coronavirus

Health & Nutrition

New Fluoride Study Ignores Neurotoxicity and Sugar Intake in Cost Analysis

A new fluoride study has been criticized for being a tool of the industry to promote continued water fluoridation following the banning of the practice in Utah and Florida. The authors conducted a simulation model and cost-effectiveness analysis with data from 8484 children with a mean age of 9.6 years. The authors concluded eliminating fluorideContinue reading New Fluoride Study Ignores Neurotoxicity and Sugar Intake in Cost Analysis

More news about Health & Nutrition

Vaccines

HHS Terminates Moderna Contract For Bird Flu Vaccine; Calls mRNA Technology “Under-Tested”

HHS has terminated a $766 million contract with Moderna to develop an mRNA bird flu vaccine. HHS spokesman Andrew Nixon said the decision is about “safety, integrity, and trust.” He added that “mRNA technology remains under-tested, and we are not going to spend taxpayer dollars repeating the mistakes of the last administration, which concealed legitimateContinue reading HHS Terminates Moderna Contract For Bird Flu Vaccine; Calls mRNA Technology “Under-Tested”

More news about Vaccines

Science & Tech

Humanoid Robots Malfunction and Attack; Sparks Debate Over AI Safety and Liability

A viral video released from an undisclosed Chinese factory shows a humanoid robot malfunctioning and attacking the factory workers. The Unitree H1, a product of the Chinese company Unitree Robotics, can be seen flailing its arms as workers move away to protect themselves. The robot was hung on a crane, and the workers were notContinue reading Humanoid Robots Malfunction and Attack; Sparks Debate Over AI Safety and Liability

More news about Science & Tech

Environment

CDC Advised Health Care Providers Not to Test East Palestine Residents for Chemical Illnesses, Lawsuit Alleges

Newsnation reported this week that the CDC advised local health care providers in East Palestine, OH, not to test local residents for chemical illnesses after the town was exposed to a toxic plume from a “controlled burn” after the Norfolk Southern train derailment in 2023. A lawsuit has been filed against Norfolk Southern and moreContinue reading CDC Advised Health Care Providers Not to Test East Palestine Residents for Chemical Illnesses, Lawsuit Alleges

More news about Environment

Policy

HHS and CMS Urge Changes to Gender-Affirming Care for Minors; Clinics Face Potential Loss of Federal Funding

HHS Secretary Robert F. Kennedy Jr. wrote a letter addressed to healthcare providers saying he expects them to update their “treatment protocols and training to ensure that care for children with gender dysphoria is evidence-based.” The letter was also addressed to healthcare risk managers and state medical boards. The letter refers to the report publishedContinue reading HHS and CMS Urge Changes to Gender-Affirming Care for Minors; Clinics Face Potential Loss of Federal Funding

More news about Policy