Augmented Reality Guidance for Structural Heart Disease Interventions Using Deep Learning-Based Catheter Segmentation


Sun-Joo Jang
Research Associate, Dalio Institute of Cardiovascular Imagining, Department of Radiology at Weill Cornell Medicine
New York-Presbyterian

Overview

At the Dalio Institute of Cardiovascular Imaging, Sun-Joo Jang and his team have undertaken the task of using a mix of augmented and virtual reality to aid in cardiology procedures.

In this presentation, Jang describes the process that he and his team used to build a deep learning algorithm to create accurate and real-time three-dimensional models for use during operations.

Watch Sun-Joo Jang’s full presentation here

Technological advances continue to aid the medical community in their capacity to treat an increasing array of afflictions. Now, medical imaging has begun to take the next revolutionary step with the help of algorithms that allow medical professionals the ability to virtually step inside procedures being performed.

As Jang describes, virtual reality can be thought of as a reality in which you need to view through some sort of optics (usually glasses) while augmented reality is able to create a digital display that is separate from the real physical environment. At Dalio Institute of Cardiovascular Imaging, Jang and team are using a combination of these two to create a mixed reality where generated holograms can be interacted with in a three-dimensional virtual environment.

The benefits to using this combination of technology and machine learning can lend to substantial improvements in the ability for cardiologists to perform procedures effectively and efficiently. Often times when procedures are difficult or have a risk of serious complications additional preparation and staff are needed during the procedure. Being able to create these three-dimensional models to study prior to the procedure and then use them in real time during the procedure can greatly increase the success of procedures while at the same time decreasing the supporting cast cardiologists require.

“Education wise, understanding the anatomy has higher potential and mixed reality will play a much better role teaching future generations.”

Additionally, this mixed reality imaging is an excellent tool for teaching. Where more experienced cardiologists can tell where they are exactly positioned, less experienced or new cardiologist may have a difficult time translating the traditional two-dimensional CAT scans and X-rays into the real three-dimensional world. The mixed reality imaging aids in bridging this gap between the two-dimensional and three-dimensional worlds that often separates text book learning and real-world application. 

Deep learning algorithms are similar to neural networks in the way that they are structured where data gets passed back and forth within an algorithm that attempts to optimize results. Where deep learning differs from neural networks is the amount of structure that is provided to the data set fed in to the modeling process and the expectations of the results that the model is told to look for. The data sets for deep learning models are unstructured and the targeted outputs are not defined. In this manner, the algorithm is allowed to come up with trends and expected outcomes free of human bias or persuasion.

Like neural networks, the data still needs to be trained, labelled, and validated. The challenge that Jang and team faced was being able to find consistent points of reference within the data set for which to build reliable models off of. First, external markers were placed along the spine to help define spaces. Next, X-ray images from many different angles were then used to determine proper marker locations and aid in filtering out shadows and other extraneous aspects in images.

Using this method an algorithm and model was created with over 99% accuracy that could place a catheter within 1mm of true position in three-dimensional space. As hearts are all shaped differently a flexible algorithm is required in order for catheters to be accurately guided into different spaces of the heart.

Sun-Joo Jang and his team now have the ability to bring a three-dimensional model of cardiovascular procedures in to any environment to the benefit of students or working professionals. For the purposes of education or being able to expertly navigate the intricacies of procedures, these models can be shared and utilized with precision and ease. Putting effort into the development and preparations of these models continue to advance the abilities that these medical professionals are able to provide and enhance the knowledge-base of all parties involved.


Tags   •   Healthcare

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Posts

Recent Posts

Does Healthcare AI Meet Basic Ethics Principles? - Ingrid Vasiliu-Feltes Chief Quality and Innovation Officer MEDNAX, Health Solutions Partner Over the past decade we have noticed an exponential…
“Ask Me Anything” with Zappos’s Head of AI/ML Research & Platforms, Ameen Kazerouni - Ameen Kazerouni Head of AI/ML Research & Platforms Zappos Family of Companies Ai4 recently hosted an "Ask Me Anything" session…
Top AI Conferences - Interested in learning the latest in AI this year? We’ve compiled a list of the top artificial intelligence conferences in…
Securing Personal Data for AI/ML Computing in the Cloud - Serge Vilvovsky Cybersecurity & Big Data Engineer MIT Startup Exchange Overview In this presentation, Serge Vilvovsky, a Cyber Security and…
Secure Your Code Via AI - Eliezer Kanal Technical Manager, Cyber Security Foundations, CERT Division Carnegie Mellon University Software Engineering Institute Introduction In this presentation Eliezer…
Leveraging AI in Cybersecurity Risk Modeling & Mitigation - Christopher Novak Director, Threat Advisory Research Verizon Wireless Introduction Originally, there was a poor understanding of why cyber breaches were…
Using AI to Build More Secure Software - Mark Sherman Technical Director, Cyber Security Foundations, CERT Division Carnegie Mellon University Software Engineering Institute Introduction MITRE's Common Vulnerabilities and…
Artificial Intelligence & Cybersecurity: Math Not Magic - Wayne Chung CTO FBI Introduction The field of cybersecurity has slowly progressed from an art to a science. It has…
Life Hacks for Thrilling the Customer with Your Data Science Technical Summary Products - Anne Lifton Manager of Data Science Nordstrom Overview Anne Lifton is a Manager of Data Science at Nordstrom and in…
Machine Learning for Pricing and Inventory Optimization @ Macy’s - Jolene Mork Senior Data Scientist Macy's Iain Stitt Data Scientist Macy's Bhagyesh Phanse VP, Data Science Macy's Overview In this…

Popular Posts

Leveraging AI in Cybersecurity Risk Modeling & Mitigation - Christopher Novak Director, Threat Advisory Research Verizon Wireless Introduction Originally, there was a poor understanding of why cyber breaches were…
Machine Learning for Pricing and Inventory Optimization @ Macy’s - Jolene Mork Senior Data Scientist Macy's Iain Stitt Data Scientist Macy's Bhagyesh Phanse VP, Data Science Macy's Overview In this…
Machine Learning and Artificial Intelligence in Banking - Artit "Art" Wangperawong Distinguished Engineer US Bank Introduction Every company’s AI journey is different. We’re all trying to figure out…
Convergent AI in Reducing Overdiagnosis, Overtreatment, and Misdiagnosis - Stephen Wong Chief Research Information Officer & Chair Professor Houston Methodist The current healthcare situation:  High-cost/low-quality healthcare is now a…
Figuring Out Applied ML: Building Frameworks and Teams to Operationalize ML at Scale - Dr. Nels Lindahl Director of Clinical Systems CVS Health How do you operationalize machine learning at scale? How do you…
The Autonomous Pharmacy: Applying AI and ML to Medication Management Across the Care Continuum - Ken Perez VP of Healthcare Policy Omnicell, Inc. Ken applies artificial intelligence (AI) and machine learning (ML) solutions to medication…
Unleashing the Power of AI on R&D: The Potential, The Pitfalls, The Progress - Dr. Jim Weatherall VP, Data Science & AI Astrazeneca R&D Intro to AstraZeneca R&D Here we see an end-to-end case…
Advancements at Siemens Healthineers in AI for Medical Imaging - Bimba Rao Head of Global Artificial Intelligence Engineering Siemens Healthineers Ultrasound Siemens Healthineers background  Siemens Healthineers builds healthcare products and…
What AI Will Bring to Medicine and Why Human Experts Are Here to Stay - Hakima Ibaroudene Group Leader - Research & Development Southwest Research Institute Overview Hakima Ibaroudene discusses how she and her team…
Does Healthcare AI Meet Basic Ethics Principles? - Ingrid Vasiliu-Feltes Chief Quality and Innovation Officer MEDNAX, Health Solutions Partner Over the past decade we have noticed an exponential…