AI for prognoses in cancer care: Integrating physician expertise with deep learning

Known in artificial intelligence circles as the ‘Godfather of AI’, Geoffrey Hinton was recently awarded the Nobel Prize in physics for his contributions to the advancement of deep learning and neural networks that power many of today’s most popular AI applications, including large language models like ChatGPT.

In the field of radiology, however, Hinton isn’t quite as revered and is perhaps best known for his prediction in 2016 that radiologists would soon be replaced with AI systems, declaring a at conference in Toronto that “people should stop training radiologists”.

While the use of AI in medicine has increased in the eight years since Hinton’s pronouncement, the brains of radiologists have yet to be made superfluous by artificial neural networks. In fact, radiologists are in high demand in countries like the U.S. due to a shortage, and some physicians believe that Hinton’s statement may have dissuaded aspiring radiologists from joining the discipline altogether.

When envisioning the future, Muhammad Ridzuan, a doctoral student at the Mohamed bin Zayed University of Artificial Intelligence, believes that the best care for patients will be made possible not by replacing doctors with AI, but by building technologies that foster deep collaboration between physicians and machines. “Instead of looking at AI as the competition, we want clinicians to have access to tools that can help them perform better,” he says.

Ridzuan and colleagues from MBZUAI recently developed an innovative new system called Human-in-the-Loop for Prognosis (HuLP) that is designed to help physicians assess how cancer may progress in patients. The model provides users information about how it makes its predictions and allows users to intervene and correct them if they are inaccurate. “We hope to foster a more synergistic relationship between the physician and AI,” Ridzuan explains.

The work was presented in October at the 27th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), which was held in Marrakesh, Morocco.

Predicting prognosis

While diagnosis looks at the present state of a disease, prognosis predicts how a disease will progress in the future. Accurate prognoses help physicians prioritize patients, delivering care to the most critical cases first. Accurate prognoses also help patients make decisions about what treatments to pursue, as they often must consider the side effects that come with many cancer treatments.

Like any problem related to predicting the future, developing an accurate prognosis is a significant challenge, as there are many factors that affect the course of disease. Today doctors review the latest scientific literature, analyze patient data (including medical history and images of tumors) and use statistical tools that process data to aid in developing predictions.

This approach, however, requires physicians to synthesize a huge amount of information and takes time. “AI models, on the other hand, have the advantage of being able to ingest a lot of information,” Ridzuan says. What’s more, physicians in some parts of the world lack access to the most recent medical literature and other resources. Predictive AI models for prognosis have the potential to help physicians in large hospitals and remote clinics alike.

Humans in the loop

Deep learning has been used to develop prognoses for cancer before, but these so-called “black box” systems produce prognoses that aren’t interpretable by humans, meaning the models don’t provide explanations about how they arrive at their decisions. With a typical deep learning approach, “the user would just have to take the prognosis as is,” Ridzuan says. “We want to understand the reasoning behind a prognosis made by a model.”

HuLP works by first predicting patients’ diagnostic features, including status according to the TNM cancer staging system, a global standard for assessing the spread of disease. HuLP uses these features to perform prognosis and gives physicians an opportunity to intervene in the model’s predictions and correct them if they are wrong, improving the model’s prognostic outcomes.

Ridzuan and his colleagues tested the performance of HuLP on a lung cancer dataset called ChAImelon with and without humans in the loop. When physicians were given the opportunity to collaborate with the model, its ability to determine an accurate prognosis improved by 11%.

Minding the gap

While huge amounts of data are collected from patients in today’s clinics, records still often lack important information, posing a challenge for analysis by machine-learning systems. One approach to filling these gaps is by using naïve imputation, where a chosen method is used to estimate and fill in missing values in a dataset. But this is not how doctors solve the problem, Ridzuan explains.

When important data are missing from a record, physicians typically turn to radiological images to gather more information about a patient’s disease and use insights gleaned from these images to develop a prognosis. HuLP mirrors this approach.

Instead of filling gaps in data through naïve imputation, HuLP uses a neural network to analyze radiology images and generates values to fill in the gaps. Once this process is performed, HuLP processes the newly completed records to form a prediction. “We’re trying to emulate the workflow of a clinician, which we think can increase trust and lower resistance to AI tools” in clinics, Ridzuan says.

AI with a purpose

Ridzuan adds that the goal of his work is to develop ways of using technology that can improve human life. And while plugging away on the figurative nuts and bolts of AI can make it difficult to see the direct impact that innovations can have for people, “projects like these can be a lot more meaningful,” he says.

“There is a lot of hype today in AI, but there is also a lot of fear at the same time. Instead of saying that AI is all good or AI is all bad, why don’t we try to understand why some people are resistant to it and develop systems that are useful for everybody.”