To train their algorithm, Yeom worked with Park and Christopher Chute, a graduate student in computer science, and outlined clinically significant aneurysms detectable on 611 computed tomography (CT) angiogram head scans.

“We labelled, by hand, every voxel – the 3D equivalent to a pixel – with whether or not it was part of an aneurysm,” said Chute, who is also co-lead author of the paper. “Building the training data was a pretty grueling task and there were a lot of data.”

Following the training, the algorithm decides for each voxel of a scan whether there is an aneurysm present. The end result of the HeadXNet tool is the algorithm’s conclusions overlaid as a semi-transparent highlight on top of the scan. This representation of the algorithm’s decision makes it easy for the clinicians to still see what the scans look like without HeadXNet’s input.

“We were interested how these scans with AI-added overlays would improve the performance of clinicians,” said Pranav Rajpurkar, a graduate student in computer science and co-lead author of the paper. “Rather than just having the algorithm say that a scan contained an aneurysm, we were able to bring the exact locations of the aneurysms to the clinician’s attention.”

Eight clinicians tested HeadXNet by evaluating a set of 115 brain scans for aneurysm, once with the help of HeadXNet and once without. With the tool, the clinicians correctly identified more aneurysms, and therefore reduced the “miss” rate, and the clinicians were more likely to agree with one another. HeadXNet did not influence how long it took the clinicians to decide on a diagnosis or their ability to correctly identify scans without aneurysms – a guard against telling someone they have an aneurysm when they don’t.

To other tasks and institutions

The machine learning methods at the heart of HeadXNet could likely be trained to identify other diseases inside and outside the brain. For example, Yeom imagines a future version could focus on speeding up identifying aneurysms after they have burst, saving precious time in an urgent situation. But a considerable hurdle remains in integrating any artificial intelligence medical tools with daily clinical workflow in radiology across hospitals.

HeadXNet team members (from left to right, Andrew Ng, Kristen Yeom, Christopher Chute, Pranav Rajpurkar and Allison Park) looking at a brain scan. Scans like this were used to train and test their artificial intelligence tool, which helps identify brain aneurysms. (Image credit: L.A. Cicero)

Current scan viewers aren’t designed to work with deep learning assistance, so the researchers had to custom-build tools to integrate HeadXNet within scan viewers. Similarly, variations in real-world data – as opposed to the data on which the algorithm is tested and trained – could reduce model performance. If the algorithm processes data from different kinds of scanners or imaging protocols, or a patient population that wasn’t part of its original training, it might not work as expected.

“Because of these issues, I think deployment will come faster not with pure AI automation, but instead with AI and radiologists collaborating,” said Ng. “We still have technical and non-technical work to do, but we as a community will get there and AI-radiologist collaboration is the most promising path.”

Additional Stanford co-authors are Joe Lou, undergraduate in computer science; Robyn Ball, senior biostatistician at the Quantitative Sciences Unit (also affiliated with Roam Analytics); graduate students Katie Shpanskaya, Rashad Jabarkheel, Lily H. Kim and Emily McKenna; radiology residents Joe Tseng and Jason Ni; Fidaa Wishah, clinical instructor of radiology; Fred Wittber, diagnostic radiology fellow; David S. Hong, assistant professor of psychiatry and behavioral sciences; Thomas J. Wilson, clinical assistant professor of neurosurgery; Safwan Halabi, clinical associate professor of radiology; Sanjay Basu, assistant professor of medicine; Bhavik N. Patel, assistant professor of radiology; and Matthew P. Lungren, assistant professor of radiology.

Hong and Yeom are also members of Stanford Bio-X, the Stanford Maternal and Child Health Research Institute and the Wu Tsai Neurosciences Institute. Patel is also a member of Stanford Bio-X and the Stanford Cancer Institute. Lungren is a member of Stanford Bio-X, the Stanford Maternal and Child Health Research Institute and the Stanford Cancer Institute.

Pages: 1 · 2

More Articles

• National Institutes of Health: Common Misconceptions About Vitamins and Minerals
• A Yale Medicine Doctor Explains How Naloxone, a Medication That Reverses an Opioid Overdose, Works
• Kaiser Health News Research Roundup: Pan-Coronavirus Vaccine; Long Covid; Supplemental Vitamin D; Cell Movement
• How They Did It: Tampa Bay Times Reporters Expose High Airborne Lead Levels at Florida Recycling Factory
• A Scout Report Selection: Science-Based Medicine
• Journalist's Resource: Religious Exemptions and Required Vaccines; Examining the Research
• Government of Canada Renews Investment in Largest Canadian Study on Aging
• Kaiser Health News: Paying Billions for Controversial Alzheimer’s Drug? How About Funding This Instead?
• Medicare Covers FDA-approved COVID-19 Vaccines; You Pay Nothing For the COVID-19 Vaccine
• Envision Color: Activity Patterns in the Brain are Specific to the Color You See; NIH Research Findings Reveal New Aspects of Visual Processing