Safeguarding AI medical imaging

Part of the promise of AI is that it will tap into vast repositories of data and deliver significant, untapped value. In the healthcare realm, AI promises to make patients, treatment teams, hospitals, and insurance companies healthier, better informed, and more efficient. But there’s a catch.

All of the data needed to power health and efficiency gains must be protected. Legislators the world over know this, which is why we have laws such as HIPAA and GDPR. And it is in adhering to strict privacy and security requirements that many AI for health projects fall flat as AI systems can be opaque, and their robustness in the face of hacking or attack can be questionable.

An MBZUAI team – comprising Associate Professors of Computer Vision Karthik Nandakumar and Mohammad Yaqub and research assistant Faris Almalik – aims to address security concerns around computer vision systems and the way they read, interpret, and report on medical images.

Their research: Self-Ensembling Vision Transformers for Robust Medical Image Classification (published at MICCAI), is helping to demonstrate the security and robustness of AI imaging systems. Their intention is to help safeguard the anonymity of the patients, as well as the validity of results, so that the revolution in AI for health can move forward.

A case of stop or go

Imagine, for a moment, that you are designing a computer vision system for a self-driving car. In research published recently, a team demonstrated that they could fool vision systems into “seeing” a stop sign, and through the introduction of visual noise, cause the system to instead interpret the sign as a yield, or as a sign that communicates the speed limit.

This work demonstrates how highly sophisticated systems can be disrupted cheaply and easily, with potentially deadly consequences. And it begs the question: who would want to disrupt self-driving cars from stopping at stop signs? The risk, while important, is probably quite low. When we move into the healthcare realm, however, the stakes get a lot higher — patients could suffer, doctors could act on bad information, and insurance fraud could skyrocket.

This is why the team from MBZUAI set out to test and resolve challenges with attacks in medical imaging systems. A hack that causes a medical imaging system to misinterpret a chest X-ray could have deadly and costly consequences. And holding such systems for ransom might likewise be profitable for capable and immoral hackers.

Protecting against such a scenario is one of the many reasons that mountains of healthcare regulations have been signed into law around the world. And also, as mentioned earlier, why developing and implementing such systems, while vital, is also so challenging.

Understanding computer vision

Funduscopy is the only way to directly inspect arteries, veins, and the central nervous system in an intact, living patient. Reading the visual outcomes of funduscopy accurately can help reveal a range of important things about a patient’s status related to endocarditis, anemia, diabetes, leukemia, hemorrhaging and more. The importance of funduscopy then, makes using AI to support treatment teams and patients a highly valuable undertaking and one that the research team were interested in investigating.

In the paper, the team propose a novel self-ensembling method to enhance the robustness of Vision Transformers (ViT) and Convolutional Neural Networks (CNN) — competitor computer vision technologies — for various computer vision tasks in medical imaging such as classification and segmentation. CNNs and ViTs have both been shown to have vulnerabilities to adversarial attack, which raises serious concerns about safety in clinical settings.

In response, the team propose Self-Ensembling Vision Transformers (SEViT) that make use of the fact that feature representations learned by initial blocks of a ViT are relatively unaffected by adversarial perturbations. Learning multiple classifiers based on these intermediate feature representations and combining these predictions with that of the final ViT classifier can provide robustness against adversarial attacks.

Measuring the consistency between the various predictions can also help detect adversarial samples. Experiments on two modalities (chest x-ray and funduscopy) demonstrate the efficacy of SEViT architecture to defend against various adversarial attacks in the gray-box (attacker has full knowledge of the target model, but not the defense mechanism) setting. Code: https://github.com/faresmalik/SEViT