In the original Star Trek TV series, the physician on the USS Enterprise, Bones McCoy, would diagnose his patients with just a few waves of his hand-held “tricorder”. Well, medical science has not progressed that far, at least not yet.
But talk with Dr. Simon Kos, Microsoft’s Chief Medical Officer, and you get a real sense of how fast new digital technologies are changing what’s happening at, and away from, the bedside – and with better clinical outcomes.
As with so many other professions, the work of doctors, nurses, and other healthcare professionals is being digitally disrupted and transformed. “It’s exciting and it is happening faster than we ever thought,” Dr. Kos said on a recent visit to Singapore. “The future is already here. Clinicians are embracing technology, and we are learning a lot.”
Many of these changes – from diagnosis to scanning, to chronic disease management, to home nursing, and even surgery – are being conceived and pioneered by some of the world’s top medical institutions. Perhaps the most exciting gamechanger is mixed reality, which blends the real and digital worlds so that physical and digital objects co-exist before the eyes of a user. It’s now being applied in medical education, in surgical planning and, increasingly, in operating theaters.
Let’s start at the beginning. The internal workings of the body have fascinated us since before the time of Leonardo da Vinci. And up until today, most medical schools have relied on books, models and, just like Leonardo, the dissection of cadavers to teach anatomy and surgical procedures to tomorrow’s doctors. But that has suddenly changed.
“Our way of visualizing the body in medicine has historically been as a two-dimensional abstraction. Now through mixed reality, we can view it in three dimensions and that is pretty transformative,” Dr. Kos says. Mixed reality is so transformative that clinicians – who are normally conservative in the uptake of new technologies – are “leaping at it”.
Institutions like Case Western Reserve University in Cleveland in the United States and the University of Sydney in Australia are using Microsoft HoloLens to instruct their students. Donning headsets, they can “see” and study the anatomical complexities as if from the inside of the body. Three-dimensional tutorials, projected across a visor, graphically show how the heart pumps, how the nervous system functions, how bones and muscles interact, as well as the layout of vital organs. Students can walk around and even through the projected images to get intimately acquainted with the subject.
Similarly, global training and education company, CAE Healthcare, is aiming to improve patient safety by getting its students to use HoloLens to simulate and practice medical procedures. Its President Dr. Robert Amyot says healthcare providers are at their “most dangerous” when they are still on a “learning curve”. But now that has completely changed due to HoloLens. Through repeated and 3-D simulations, students can practice varied scenarios, make mistakes, and try things before treating real patients.
Meanwhile, the Hololens – which is an advanced computer in its own right – has also become an unrivaled tool for surgeons planning intricate operations, which have little or no room for error. This is how it works: A patient is scanned by a CT or MRI machine. The resulting two-dimensional images are then presented in 3-D within a HoloLens headset. With these a doctor can determine exactly what needs to be done and how.
This is happening right now in Norway’s capital, Oslo, by a team of specialists renowned for correcting congenital defects in the tiny hearts of infants. Until recently, they had their own hi-tech way of planning such intricate surgeries: They used scans and a 3-D printer to create physical models of each heart. But now, the precise interactive 3-D imagery within the Hololens has made that once cutting-edge solution redundant after just a few short years.
Dr. Kos believes mixed reality will revolutionize how surgeons tackle a wide range of delicate operations, resulting in better success rates and speedier patient recovery times. “Using HoloLens means you can do surgery with no surprises. You can plan carefully ahead of time. No one’s anatomy is the same as another’s. So, you have got to know what you are in for.”
Today HoloLens is not approved for use in the United States for live surgical scenarios. But it is starting to be used in operating theaters elsewhere. In France, mixed reality imagery from actual operations is recorded for student instruction. Also in Europe, surgeons relying on the preciseness of HoloLens not just to plan ahead, but also to guide them through actual operations. For instance, Scopis, a company that specializes in surgical navigation as well as medical augmented and mixed reality technologies, this year launched its Holographic Navigation Platform. It incorporates HoloLens, which is being used by surgeons during multiple vertebrae fixation operations. The planned positioning of screws is projected onto the surgeon’s field of view and overlaid exactly onto the patient, creating the mixed reality experience, the company says. This allows a surgeon to find the screws’ planned positions faster and to align surgical instruments interactively with the holographic visualization.
“Scopis’ holographic solution has the potential to make spine surgery more effective, safe, and precise,” said Professor Christian Woiciechowsky, Chief of the Spinal Surgery Clinic at Vivantes Humboldt Hospital in Berlin, Germany. “Integrating mixed-reality tools into surgery is a huge technological advancement toward enhancing a surgeon’s vision and may provide greater benefits to patients.” Scopis sees this mixed reality technology being applied in other areas, such brain surgery where tumors could be located faster and with more accuracy.
Dr. Kos also sees huge potential for mixed reality in improving medical home care for patients, such as the aged and those with disabilities. One pioneer in this field in Silverchain, a healthcare provider in Australia. It has reimagined its client care record system with Microsoft Dynamics 365, which provides a versatile informational interface both for standard mobile devices and HoloLens.
“A visiting nurse can come to a patient’s home and access this information at the bedside,” he says.
“Wearing a HoloLens, they can do hands-free documentation while a doctor back at his clinic can see the same via video remotely. So, a full consultation can be conducted in real time in the comfort of the patient’s home.”
By Geoff Spencer Digital Content Editor, Microsoft Asia
*Source: Microsoft