Augmented Reality assists surgeons in the operating theater

Artificial intelligence is taking on more and more tasks in our modern world. For example, we use it every day when we use online search engines. Translation programs are unimaginable without AI, as are speech recognition, face recognition, computer games and, in the future, autonomous driving. In medicine, AI is also becoming more widespread and has already found its way into the operating theater. Just a few days ago, Innovation Origins wrote about operating with live 3D image navigation inside the body.

The Karlsruhe Institute of Technology (KIT) has now gone one step further and has even been awarded the NEO 2019 Innovation Prize (worth €20,000) by the Karlsruhe TechnologyRegion for their ‘HoloMed’ system. The new system assists surgeons in the operating room via Artificial Intelligence (AI) and Augmented Reality (AR). It does this by creating a model from computer tomographic images of the patient. These reveal the hidden structures deep inside the body.

GPS for the brain

HoloMed’s main focus is on cranial punctures. This is a procedure whereby accumulated fluid is removed from the brain in order to reduce pressure. Frequently used for e.g. brain hemorrhages, craniocerebral trauma and strokes. In order to determine the optimal point of insertion and alignment for the puncture, the surgeon must measure and glean data from “various anatomical landmarks” from computer tomography (CT) and/or magnetic resonance imaging (MRI) scans.

“The difficulty lies in the fact that determining the angle of insertion only allows for a very small margin of error and the doctor isn’t able to see the target straightaway,” notes Professor Björn Hein. He oversees the project together with Professor Franziska Mathis-Ullrich at KIT. Determining this exact point is complicated as these images are only two-dimensional and the human head is three-dimensional. That’s why only about 60 percent of all free-hand incisions are able to pinpoint the best position.

Surgeons use HoloMed augmented reality glasses to assist them in determining this optimal insertion point and angle for the puncture needle. An AI developed at the AI by science staff member Christian Kunz uses the data from the patient’s digital file and their latest CT and/or MRI scans for creating a model that accurately depicts the structures deep inside the body that cannot be seen externally. This information is superimposed onto the surgeon’s AR glasses and shows the surgeon precisely where and how to guide the needle, much like a navigation system.

Easy to use and cost-efficient

Professor Hein states that machine learning methods are used in the automated generation of this information. “First of all, a segmented 3D model of the head is generated, which is used to determine the target position. However, the doctor is always able to make their own adjustments if appropriate,” Hein adds. The aim of the system is to provide an “innovative, novel and cost-effective solution that has a direct influence on the quality of these procedures”.

After its puncture method is successfully rolled out, HoloMed will also be used for other operations in the future. Since the system is, firstly, easy to use, and secondly, cost-efficient, the inventors say it is ideal for lowering healthcare costs. Plus it would also benefit poorly financed hospitals in emerging countries.

Cover photo: Dr. Michal Hlavac from the University Clinic for Neurosurgery Ulm and Christian Kunz from the “Health Robotics and Automation” (HERA) KIT team evaluating the HoloMed system during the initial surgery simulation with a dummy. (Photo: KIT-HERA).

Start-up of the day: Swifter surgeries thanks to new discovery

Medical science in the Netherlands is known for its pioneering discoveries. Sometimes these solutions may seem simple, but they can speed up processes considerably. Such as the invention made by SLAM Ortho, for instance. We spoke to the founders Bart Kölling and Just Schornagel.

What kind of innovation has SLAM Ortho come up with?

We have a new product that is able to speed up orthopedic surgery. This type of procedure often involves the use of metal plates and screws for fixating broken bones. To do this, you need to drill into the bone and then measure the depth of the hole with a metal hook. This step is time-consuming and prone to mistakes. We have developed a drill bit that automatically measures this hole depth and can subsequently send this information wirelessly directly to the surgeon to read. This drill bit works with all drilling equipment and no other steps need to be taken during the operation. It really is a plug-and-play solution that can speed these procedures up by 10%, which can save valuable time in the operating room (OR). Less time spent in the OR is also better for the patients, and they also have a lower chance of having the wrong length of screw inserted, which could potentially be harmful.

How did you come up with the idea of setting up this company?

The surgeon in our team has often seen this problem and wanted to do away with using the metal hook. He then sought out the other founders and together we looked for potential solutions. After some experimentation we saw that there had to be a solution that would work well for surgeons. Consequently, the company was launched and a patent was applied for.

Bart Kölling and Just Schornagel from Slam Ortho BV

Is there much competition in this area?

Several products from independent parties have come onto the market which are trying to solve this same problem. What we see is that these products often tend to add extra steps or inconveniences, which is why they have not been put into practice in many places yet. At the moment we do not see any other competing ideas from the major players within the field of trauma/orthopedic surgery.

What are the biggest obstacles that SLAM Ortho faces?

Entering the market for equipment used in the OR is not that simple. There is a lot of regulation in the field of medical equipment and it is not so easy to just start discussing your idea with a hospital. That is why we are looking for parties to work with and help resolve these problems.

What are you proud of?

We are proud of the performance level that the prototypes have achieved so far. They have already proven their worth on real human tissue. We are also proud of the patent that has been granted.

What are your plans for this year?

This year we are working with the EIT Health Validation Lab at YES!Delft. We are currently in the process of getting our commercial plans validated. After this, we will look for investment that can help us on the path to certification of our medical equipment which can actually be brought to market.

What is your goal in the next five years?

In the longer term, we want to further develop our product portfolio and use our ‘smart’ tools with the technology we have developed in more places. The technology we can offer and the data we are able to collect will be very useful in improving the quality and efficiency of surgeries.

Read moreStart-up of the day: Swifter surgeries thanks to new discovery

Five magical medical channels on YouTube

Innovation is taking place in all kinds of sectors, some of which affect more people than others. Yet no one can avoid this sector as we all have a body that should ideally be kept in tip-top condition. Healthcare is developing at a rapid pace.  Whereas the need for hand-washing in medical procedures was disputed until the end of the 19th century, nowadays we can perform complete operations using nanorobots.

Pretty remarkable how we are now able to stay healthier longer and in improved ways. Are you interested in the latest developments in the field of healthcare and do you want quick and easy access to this information? Check out these five channels on YouTube, where medical professionals are passionate about their own very special and important field of expertise.

1. Mayo Clinic

Mayo Clinic‘s history dates back all the way to 1855, and this American non-profit organization employs around 63,000 people who work in hospitals, research labs and universities. Everyone can follow their work on YouTube where their channel explains the function of organs and reports on extraordinary surgical operations.

Perhaps the most well-known is their series about a facial transplant that they performed. The treatment is first explained using motion graphics and then they demonstrate how the patient was operated on. There is also room for follow-up care so you can see how the family has responded and what the result looks like after two years.


2. Armando Hasudungan

The Australian physician Armando Hasudungan has been making films about his profession since 2012. No moving images from the operating room for him, but a whiteboard and a pen. Hasudungan explains in an accessible way the stories behind diseases, the impact of medicines and the workings of the human body.

His own consistent visual style and accessible language guarantee that both medical professionals and people with an interest in healthcare can understand him easily. Not only is he doing well on the internet, he is doing well in universities too, where his films are often recommended to students.


3. Osmosis

The Osmosis motto states that: ‘Everyone who cares for someone will learn by Osmosis!’ What is particularly striking about this channel is that it takes the viewer’s level of knowledge into account. Their videos on YouTube are aimed at mainstream audiences who want to know more about the medical world, psychology and public health.

They have a website for the real die-hards and professionals where subject matter is discussed in more detail. However, you will be able to find a lot of useful information on YouTube as well; every disorder, both physical and mental, has its own explanatory video. It may not be the best channel to watch if you have the inclination to be hypochondriac …. although they do have a film about that too.


4. Medcram

The Medcram channel offers medical lectures and is based on the principle that the viewer should learn as much as possible in as little time as possible. Complex topics are explained in a visual way and sufficient frameworks are provided so as to ensure that the information is complete. Medcram is primarily a tutor who puts the emphasis on what is most crucial from a clinical point of view.

The lectures are very diverse; from simple first aid to symptoms of common injuries. The channel is specifically intended for people working within the healthcare sector and who are in the process of receiving medical training. Nevertheless, because of its public character, everyone can join this virtual lecture hall.


5. The Medical Futurist

34-year-old Dr. Bertalan Mesko is a welcome guest at international conventions on healthcare. In fact, there are few channels in which a title so clearly discloses the content. Whereas the preceding channels mainly explain the practical element of healthcare, Mesko focuses on the future and casts his expert eye on new developments in the field where he gained his PhD.

The Medical Futurist takes the viewer into … the future. The future of medication, healthcare, diseases, treatments and technologies that are await us in the upcoming years. These developments are made concrete with clear examples and personal stories. Mesko believes that progress in healthcare lies predominately in novel and far-reaching applications of technology. That’s when it’s quite OK to think big.


Not yet done with learning? In that case, we have even more educational YouTube channels on offer. In the previous episodes of this series, technological innovation, entrepreneurship and mobility were key.

VIVATOP: How Virtual Reality and 3D-printed models make surgery easier

Bildaufnahme OP-Saal mit Displaytechnink

Surgical procedures always  pose a risk. Moreover, x-ray imagery or data from CT or MRT do not reveal everything. A research project led by the Technology Centre for Computer Science and Information Technology (Technologie-Zentrum Informatik und Informationstechnik or TZI) at the University of Bremen aims to change this through new technologies such as 3D printing, virtual reality and augmented reality. These should help surgical teams to better plan their procedures and reduce the risks for the patient.

VIVATOP devises VR solutions for surgeons

Modern surgery has long been a high-tech domain. This applies not only to the operations themselves, but also to their preparation. Surgeons make use of X-ray images, computer tomography (CT) or magnetic resonance imaging (MRI) results and much more. Computer-generated 3D models are relatively new. But they still have to be viewed on 2D screens.

Surgeons still have to employ models made by 3D printers, such as those emerging from VIVATOP. However, there are already experiments with mixed reality applications. For instance, in the field of oral and maxillofacial surgery. VIVATOP stands for “Versatile Immersive Virtual and Augmented Tangible OP”. The project combines model organs from the 3D printer with applications from VR and augmented reality.

VR-glasses and computer simulations help preparing surgical procedures.
Foto: TZI, Universität Bremen

3D printed organ models have many advantages. You are able to touch them and look at them closely, or pass them around as instructional objects. VR glasses allow for an even more detailed display. Participants in meetings are able to hold the printed organ model in their hands and see additional information displayed on their glasses. Communication between doctors and patients is also easier. With the help of the model, doctors can explain more precisely how the upcoming surgical procedure should proceed.

VIVATOP technology facilitates surgery

VR technology and organ models support the actual operation. Because every human body is different, surgeons have to reorient themselves again and again. VIVATOP’s new technology facilitates both decisions and orientation during the operation. The operating theatre is peppered with sensor and recording technology. Depth cameras and other sensors record every movement of the surgical team and the operation on the organ itself.

Printed models such as this liver help to plan and explain a surgical procedure.
Foto: cirp GmbH

The surgical team wears VR glasses during the procedure. These VR glasses constantly display all vital information. The surgeons control their glasses via voice control. Their glasses also give them access to all planning data used in the procedure’s preparation. In addition, operations can be shown live in virtual reality.

According to project coordinator Professor Dr. Rainer Malaka, it would also be conceivable to use the system in telemedicine, for example, by involving experts from a different location. However, there would always be a doctor on site. Yet the possibility of simultaneously following and recording complex surgical procedures live, opens up new possibilities for worldwide cooperation. Experts could follow an operation and support the team without having to be on site themselves.

market introduction in the making

However, it will take some time before the solutions developed within VIVATOP reach operating theatres. “The project will run for three years, after which the industry will have to continue with it,” says Professor Rainer Malaka. In his experience, certifications for new medical devices are costly and time-consuming. Although applications for training purposes and patient education might reach the market sooner.

The Federal Ministry of Research and Education is funding VIVATOP with 2.2 million euros. In addition to Malaka’s research group, other partners are involved in TZI. Another research group is working on virtual reality and computer graphics. The Oldenburg Pius Hospital contributes medical expertise, the 3D organ models are products of the Fraunhofer Institute for Digital Medicine. On top of that, there are industrial partners for augmented reality and 3D printing. Malaka estimates that the first commercial AR applications could be available in 2020.

Microsure ready for next steps with Microsurgery Robot

Microsurgical Robot prototype from TU/e

Microsure, a joint spin-off by Eindhoven University of Technology and Maastricht UMC+, this week becomes fully independent. The company’s development of a surgical robot able to perform operations with the utmost precision is almost complete. The robot, which is always controlled by a surgeon, is suited to performing reconstructive surgery that human hands are hardly capable of doing. The device, the first of its kind, should be ready for use in operating theaters within two years.

Read moreMicrosure ready for next steps with Microsurgery Robot