Checklist Outlines Six Characteristics of Ideal Machine-Learning Algorithms in Healthcare

A newly proposed checklist outlines six objectives to strive for in the development of machine-learning algorithms that help clinicians make health care decisions and recommendations for patients. Tyler Loftus of University of Florida Health, Gainesville, and colleagues present this framework in an opinion piece for the open-access journal PLOS Digital Health on January 18, 2022.

Clinicians increasingly make important healthcare decisions with the aid of algorithmic tools that draw on large amounts of complex data to a degree that far exceeds the reasoning capacity of the human mind. For instance, researchers have developed algorithms that learn from large amounts of data from previous patients to predict a new patient's risk of complications after surgery. 

However, while existing guidelines are in place to ensure that such algorithms are reported to the healthcare community in a clear, standardized manner, a framework for evaluating these algorithms has been lacking.

Now, Loftus and colleagues propose six characteristics of ideal healthcare algorithms. To maximize its benefits, according to the researchers, an algorithm must be:

1. Explainable: able to clarify the relative importance of various characteristics of a patient and their medical condition in determining outcomes, without confusing association for causation
2. Dynamic: able to adjust predictions according to real-time changes in patient characteristics
3. Precise: able to take full advantage of data collected at a high enough resolution to robustly capture a patient's changing condition
4. Autonomous: able to learn and return results with minimal human input
5. Fair: able to account for any implicit bias and social inequity
6. Reproducible: able to be shared widely with the research community for validation

To demonstrate this new framework, the researchers used it to evaluate eight algorithms that have been highly cited in academic literature. None fulfilled all six of the proposed characteristics, highlighting opportunities for improvement-;especially in autonomy, fairness, explainability, and reproducibility.

The authors suggest possible avenues for improving healthcare algorithms so that they achieve the new objectives and, ideally, maximize benefits for patients, clinicians, and other researchers.

Dr Bihorac summarizes: "We propose a framework for trustworthy medical AI by defining the set of minimal key attributes that algorithms intended for the use in clinical care need to fulfill."