Accelerating echocardiogram analysis with AI: a new deep learning method presented at MICCAI

The echocardiogram is perhaps the most performed heart test. These exams, which have held a place in the cardiologist’s toolkit for decades, are conducted frequently because they provide a non-invasive method for evaluating heart function. Yet while echocardiograms are easy to perform, their results take time to analyze — and time is a precious resource in busy hospitals and clinics across the world.

Recently, scientists at the Mohamed bin Zayed University of Artificial Intelligence have developed a new method driven by deep learning that can quickly and accurately estimate important clinical measurements from echocardiograms. The work, led by Fadillah Adamsyah Maani, a research assistant at MBZUAI, was nominated for the Best Paper Award and Best Young Scientist Paper Award at the 27th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), which took place in October in Marrakesh, Morocco.

Challenges in analysis

Echocardiograms send ultrasound waves through the chest; the ultrasound waves are converted to images, allowing a clinician to see the size and structure of the heart and watch it beat in real time. A key measurement made from echocardiograms is the ejection fraction of the left ventricle, which is representative of the heart’s ability to circulate blood throughout the body.

To calculate the ejection fraction, doctors measure the size of the left ventricle at two points — when it’s full of blood and when it’s contracted — and apply a formula that determines how much blood is pumped with each beat. This value gives doctors an idea of overall heart health and can also be used to determine a patient’s risk for heart failure.

Though cardiologists are skilled at measuring ejection fraction, it takes time to perform these analyses. “If we can solve these types of problems with machine learning, we can help doctors with their work and improve care for patients as well,” Maani said. “These systems could be particularly helpful in places like my home country of Indonesia, where patients often have to wait a long time to receive care.”

Researchers have previously built systems to estimate ejection fraction based on echocardiograms, but Maani and his colleagues saw room for improvement. For example, a previous deep-learning approach could identify cases of poor ejection fraction, but the system arrived at these results in a way that limited its performance and transferability to other datasets.

When building a machine-learning application, the data that are analyzed are translated into representations that are plotted in what’s known as a representation space. This is a kind of multidimensional, virtual space where similar data points are clustered close together, while dissimilar ones are far apart. That’s the goal, anyways.

Previous echocardiogram models, however, did not organize the data in this way. “The representations were fragmented, and our assumption is that during training, the models looked at clinically irrelevant features in the data,” Maani said. “The models could have been looking at demographic information from the patients and the types of machines that were used to create the echocardiograms, which can lead to biases in the results.”

To solve this problem, Maani and his colleagues reduced their model’s reliance on features in the data that weren’t clinically relevant. They also used a technique called contrastive learning that forms relationships between pairs of data, strengthening associations between like representations and weakening associations between dissimilar representations. In this case, echocardiograms with similar ejection fractions were grouped together, strengthening their relationship.

Maani and his colleagues call their deep-learning approach CoReEcho (continuous representation echocardiogram) and it processes echocardiogram videos. Previous approaches analyzed select, single frames from videos, but the results were not ideal. “Echocardiogram imaging is very noisy,” Maani explained. Analyzing video provides better results as “giving a model only still images can confuse it as sections of the heart can be occluded in individual frames.”

Training on limited data

Machine-learning applications hold the potential to play a major role in patient care in the coming years, but data that are needed to train these applications are often hard to come by in a field where patient information is often restricted due to privacy concerns, or simply have not been aggregated into large datasets. “Limited data is a major challenge in pretty much all medical imaging problems,” Maani said. “When we are developing applications for these kinds of tasks, we need to develop deep learning methods that work on limited data.”

Because there is often little data to train models used in medicine, scientists must choose the right architecture to accommodate this scarcity. CoReEcho employs a deep-learning feature extractor that consists of a convolutional neural network (CNN) combined with a vision transformer, taking advantage of the strengths of these two different types of architectures.

CNNs excel at identifying local relationships in data, which are important in echocardiograms. Maani and his colleagues also take advantage of CNNs’ characteristic of generalizing from these local relationships, a phenomenon known as inductive bias, which is a benefit when data are scarce. Transformers are good at making sense of global relationships in data — known as long-range dependencies — which make them suitable for processing video.

The feature extractor identifies key parts of the echocardiogram, while a multilayer perceptron head estimates the ejection fraction.

Results and future directions

The researchers used a dataset called EchoNet-Dynamic and split the dataset into training, validation and testing subsets. Once their model was trained, the scientists compared its performance to the current state-of-the-art approach called EchoCoTr, which was also developed by MBZUAI researchers. CoReEcho performed better than EchoCoTr with a mean absolute error (MAE) of 3.90 and coefficient of determination (R2) of 0.82.

It took four hours to train the model, and it returned results during testing in approximately six seconds, which makes it fitting for use in the fast-paced environment of the clinic.

Maani says that he and his colleagues are planning to continue with this work and to modify their model so that it can process different kinds of data, for example, electronic health records, in addition to video.

More generally, Maani said that he gained valuable insight about how to approach designing deep learning models in the future. “By building a model that can learn meaningful representations, we can enhance its performance and transferability. If a model has meaningful representations, we can apply the model to another dataset, say from a smaller hospital, and it would likely perform well on that smaller dataset also.”