MATD: a hologram that you can feel and hear

Holograms are part of everyday life in science fiction films and series such as Star Trek. On Captain Picard’s ‘Enterprise’ or on the ‘USS Voyager’, there are complete ‘holodecks’ where you can create any environment and any hologram you want. You are even able to physically communicate with them. On Voyager there is even a “Doctor” in sickbay – the Emergency Medical Holographic program (EHM).  Just like a person, this hologram is capable of learning from his experiences. He has many other human qualities as well.

As far as technology has advanced in the 24th century, we are not quite that far yet in the 21st century. But scientists from the University of Sussex have come a step closer to holograms like those in Star Trek. For the first time, they have developed holograms that can be seen with the naked eye, yet can be felt and heard too. The Multimodal Acoustic Trap Display (MATD) is not yet able to respond to emergency calls and treat patients. Though this system can show a colorful butterfly, emojis and other images without the need for a ‘headset’ for Virtual Reality (VR) or Augmented Reality (AR).

© University of Sussex

Using sound waves to manipulate physical objects

“Our new technology takes inspiration from old TVs which use a single color beam scanning along the screen so quickly that your brain registers it as a single image. Our prototype does the same using a colored particle that can move so quickly anywhere in 3D space that the naked eye sees a volumetric image in mid-air.” says Dr. Ryuji Hirayama, a JSPS Fellow and Rutherford Fellow at the University of Sussex. He is also the lead author of the study published in Nature.

The holograms in Star Trek are comprised of photons and force fields. According to Hirayama, the researchers’ holograms in Great Britain are produced in a similar way. “To my understanding, the holograms in these fictional mediums are generated by bending and shaping light using force fields. Similarly, our technology uses sound waves to manipulate physical objects. In the case of generating holograms, these sound waves are able to move a bead fast enough to generate solid images, and are given color through an external light source.”

The MATD is modeled on the method that made free floating projections possible as early as 2018. Small particles were illuminated by laser beams which in turn produced a hologram. The MATD uses ultrasonic sound to hold a ball in a room, move it at breakneck speed. Then light it up with red, green and blue light to create a colorful image. With up to 100 position changes per second, the rapid movement through the room creates a 3D illusion.

© University of Sussex

Room for improvement

The MATD prototype is approximately the size and shape of a microwave oven. It is made up of 512 ultrasonic speakers arranged around a clear space. The resolution of the generated holograms is not very good at the moment. “The MATD was created using low-cost and commercially available components. We believe there is plenty of room to increase its capacity and potential,” Hirayama states.

Unlike the holograms that are currently being made, the hologram created by the MATD can also produce sound. It can even be felt in real life. “Even if not audible to us, ultrasound is still a mechanical wave and it carries energy through the air. Our prototype directs and focuses this energy, which can then stimulate your ears for audio, or stimulate your skin to feel content,” Dr Diego Martinez Plasencia, co-creator of the MATD and a researcher on 3D User Interfaces at the University of Sussex. “Our prototype sends and bundles this energy, which can then generate sound or touch our skin so that we are able to feel something. For example, an infrared sensor detects when a hand is approaching the hologram. The loudspeakers are then adjusted to concentrate sound pressure of more than 150 decibels onto that hand, allowing you to feel the hologram.

Revolutionary 3D display

“Our MATD system revolutionizes the concept of 3D display. It is not just that the content is visible to the naked eye and in all ways perceptually similar to a real object while still allowing the viewer to reach inside and interact with the display. “It is also the fact that it relies on a principle that can also stimulate other senses, putting it above any other display approaches and getting us closer than ever to Ivan Sutherland’s vision of the Ultimate Display.” Project leader Sri Subramanian, Professor of Informatics at the University of Sussex and a Royal Academy of Engineering Chair in Emerging Technologies) (Sutherland is the pioneer of VR techniques – ed.)

University of Sussex

Practical applications

In order for the hologram to also be able to produce sound, the scientists had to adjust the ultrasonic waves. They did this in such a way that the relevant resonance effects were created in the hologram. Yet these sounds are still very basic. They are a long way from being anywhere close to speech. Nevertheless, Hirayama believes that this can certainly be improved upon. “Operating at frequencies higher than 40KHz will allow the use of smaller particles, increasing the resolution and precision of the visual content, while frequencies above 80KHz will result in optimum audio quality. More powerful ultrasound speakers, more advanced control techniques or even the use of several particles, could allow for more complex, stronger tactile feedback and louder audio.”

But these new holograms don’t just have an advantage when it comes to science and entertainment. The authors believe that the technology can offer interesting opportunities. Such as mixing chemicals without contaminating them. Or performing ultrasounds in tissues in order to administer life-saving medicines. As well as numerous ‘lab-in-a-chip’ applications.

And what are the odds for holograms like the HoloDoc in Star Trek? “Right now, our display manipulates a single particle to create holograms. Using multiple particles instead will allow us to create more complex holograms. Also, by modelling the dynamics of the moving particle more accurately, the particle would be able to move faster and more precisely, allowing the display to project more realistic holograms. We are going to keep working on such challenges one by one because I personally want to see holograms like the EMH one day!”

Start-up of the week: clean your house with acorns

”Your sneak preview of the future” is the slogan of Innovation Origins, and that’s just what we will highlight with our Start-up of the Week column. Over the past few days, five start-ups of the day have been featured and on Saturday we will choose the week’s winner.

Innovation Origins presents a Start-up of the Day each weekday

We shall consider various issues such as sustainability, developmental phase, practical application, simplicity, originality and to what extent they are in line with the Sustainable Development Goals  of UNESCO. They will all pass by here and at the end of the week, the Start-Up of the Week will be announced.

 

PlaygroundVR – Playing outdoors while stuck in a hospital bed

A long-term stay in a hospital is no fun at all for a child. Outdoor play is often sorely missed. A Dutch team developed PlaygroundVR and this is exactly what you would expect from it. By using VR-glasses, children are able to take a trip to a colorful online playground where there is plenty to do. Fantasy and creativity play a prominent role and physical games are also a possibility thanks to the virtual reality feature. Gaming numbs your brain?!? – What a 20th century mindset!

The game is connected to the internet so that anyone, anywhere in the world, is able to join in the fun. It can also be a solution for healthcare providers because it offers a bit of a distraction when it is time for those unpleasant medical procedures. Lastly, it offers a good alternative to children who need some distraction but are no fans of Clini Clowns.

Axiles Bionics – prosthetic feet with joints

The technology for feet prosthetics has been standing still for fifty years according to the founders of the Brussels start-up Axiles Bionics. The wooden peg prosthetic as used by a stereotypical pirate is more accurate than you may think – even in the second decade of the 21st century. Of course it does have the shape of a foot, but the way in which it works has not changed over time.

Nevertheless, a human foot is a lot more complex than a piece of solid wood. As a result, wearers of these prosthetics will never be able to move as effortlessly as they used to and will experience back pain due to unnatural body posture. Yet developments are not standing still, robotics might very well prove to be the answer. A complex mechanism of springs and motors is able to accurately simulate the functioning of muscles and joints. The team is working towards an affordable prototype that should last for a good number of years.

Skoon Energy – A marketplace for large-scale batteries

The benefits of the Sharing Economy have become increasingly evident during the second decade of this century. For instance, a drill is used merely a few hours each year while it is kept in storage for the rest of the time. It’s a shame and extremely inefficient. Surely it would be much more practical if you shared your sunbed with a large group of people who could each use it when they wanted to?

Those at Skoon Energy think that the same thing is happening when it comes to large batteries. Festivals, construction sites or ships all need a lot of power, although not all year round. To be able to meet this peak demand for electricity, this Dutch team makes it possible to have a mobile battery delivered in a container as conveniently as a home-delivered meal. And when it is no longer needed? Then it can just go on to the next destination.

Felyx Sharing – Thousands of share-scooters

In an average big city when you look all around you, it’s impossible to imagine the streets without all those brightly colored shared cars and shared bikes. This trend has actually evolved quite fast, but what struck the founders of Felyx Sharing was that there was at the time hardly anything like this available for electric scooters.

Two years on, Felyx Sharing now has a fleet of 1250 flexible rental scooters in Amsterdam, Brussels, Rotterdam and The Hague. Simplicity is essential here. Users install an app and are able to drive off with just a few clicks of a button. The parking spaces are unattended and located at several locations. The team is ambitious and has big plans ahead for scaling up!

Not convinced? Try driving into the center of Amsterdam with a car.

Herbi Clean – Cleaning with acorns

Cleaning products typically contain a lot of harmful chemicals. It is not without reason that those orange warning labels appear on the packaging. Yet Mother Nature also has a cleaning lady hiding in her, as the Polish company Herbi Clean has proven. They came up with cleaning agents made of acorns which do not need any ominous orange warning labels.

How do they do that? This is due to tannin, a substance that plants produce in order to protect their seeds from greedy herbivores. Acorns contain 7% of this substance which has an anti-bacterial effect. And it is precisely these microorganisms that tannin fights that are the main cause of a lot of grime. There are already six products on the Polish supermarket shelves, including a toilet cleaner and a detergent.

It is actually quite odd that not more research is being done into cleaning products made from plant-based material. Why should we spray our homes with dangerous substances or artificial chemicals if there is a substance in nature that does exactly the same without the disadvantages? As there seems to be a lot more to be gained from this, Innovation Origins Herbi Clean has been awarded the title of Start-up of the Week.

Start-up of the day: Virtual playground supports recovery of children in hospitals

A virtual playground for children in hospitals. PlaygroundVR is putting the finishing touches to the technical development of their virtual reality environment. “Playing outdoors is invaluable for children’s development,” says Freek Teunen, co-founder of PlaygroundVR. “Outside, there is plenty of scope for their imagination. They do a thousand things with just a ball.” Former students of Eindhoven University of Technology also want to offer this experience to younger hospital patients.

Children can be adversely affected if they are in hospital for a longer period of time and are unable to play outside as a result. “By wearing the VR glasses, children can imagine themselves to be at a playground just like any other playground they normally have in their daily lives. There are also fantasy elements such as a large music box or flying hoops,” Jason van Eunen explains, co-founder of PlaygroundVR. The whole thing is very colorful. “The children can see each other through virtual avatars. So they are really able to play together.”

How did you come up with this idea?

We are very passionate about the technology. That’s why we wanted to see how we might make Virtual Reality (VR) as useful as possible. The advantage of VR is that the user is able to forget for a moment where he or she is and can completely immerse themselves in another world. Subsequently, we soon discovered that our technology would be very beneficial for hospital patients. They aren’t able to play like they do at home. As the children themselves have indicated, they miss playing outside the most. That’s why we have developed a virtual playground.”

[youtube https://www.youtube.com/watch?v=6Ssiw0R3Bzo]

 

Doesn’t that just mean that you are making children sit in front of a screen even more?

Teunen: “We certainly don’t want children to be wearing VR glasses all of the time. That’s why we also work with play sessions, a short period when children can play with the glasses. Although we do consider that our virtual playground is different from the shooting games that children often play on screens. Of course we would like to see children playing outdoors. But if that isn’t possible, VR glasses are a pretty good solution. We still connect children with each other and give them the experience of playing outside.” Van Eunen adds to that: “It would be better if children could learn new games in the virtual playground and eventually apply them in real life. We want to further stimulate imagination and creativity that way.”

How are you going to finance this?

“We have launched a crowdfunding project this week. Our goal is to raise 55 thousand euros. Then three hospitals will be able to use the virtual playground in their children’s ward. Approximately half of that money will be used for the latest developments in technology. We will use the rest to further roll out the concept,” Teunen states.

“The goal behind crowdfunding is to show hospitals the pedagogical benefits of virtual playgrounds. This is difficult with VR because it is a completely novel technology. We first have to prove that it works. That’s what we’re going to do with the money that we raise. We think that children feel better because of the playground and as a result are able to recover much faster.”

“Financing this product with donations doesn’t seem right to us. That’s why next year we want to see how we could use the virtual playground as part of a course of treatment. We are hoping that the playground will become a healthcare resource so that it can be paid for out of medical insurance costs.”

Why did you opt for crowdfunding?

“We think crowdfunding is very appropriate for our objective. PlaygroundVR is a start-up and definitely contributes to society. We provide children with moments of happiness at a time when they are extremely vulnerable. Crowdfunding is a way to build support,” says Van Eunen.

© PlaygroundVR

How can VR be part of a treatment?

Van Eunen: ” Distractions are very important to children when they are in hospital. That’s why there are pedagogical staff in the children’s ward and  CliniClowns and iPads to keep the children busy. PlaygroundVR is distracting in several ways because it ensures that the child feels like he or she is completely out of the room. Instead of watching a movie, the kids are really immersed in it.”

“This distraction caused by wearing VR glasses can be used during a treatment. For instance, when a child is holding an infusion needle in their hand. This is an unpleasant experience because it doesn’t feel right and it’s not nice to watch either. A child can play outside with the VR-glasses on and is less aware that a needle is being stuck into their hand. This helps to avoid pain and maybe even painkillers. This allows a child to get through a whole treatment feeling a lot less fearful.”

“We spoke to a girl who literally said that she had forgotten for a short while that her leg was broken. Because she could just walk in a playground. Those are the wonderful things about VR. You can’t do that with an iPad,” Teunen says proudly.

Was there a time when you were afraid that the concept was not going to work?

“Yes,” says Teunen. “As soon as we started work on this three years ago. We took part in a pitch competition at the time, but were turned down right away. Back then, the VR industry had not yet developed very far and the technology was still quite unknown. Now we can see that healthcare professionals and parents do appreciate the value of it. They really like PlaygroundVR better than, for example, a Playstation with Call of Duty.”

“The challenge for us is to find hospitals that dare to take this innovation aboard. Institutions often find these kinds of new technologies exciting to implement. We are already in talks with a number of hospitals that are interested as it is. We will be launching the virtual playground at the Wilhelmina Children’s Hospital towards the end of the year.”

It sounds like an important aspiration to help children. Do you have a concrete goal that you want to achieve?

“Of course, it’s cool if we can improve the hospital experience for a few children. But in the long term we want to be able to connect children with each other who are in different hospitals. Then children who have the same conditions will be able to share their experiences with each other. This way, we not only want to forge new contacts between children, but we also want to bring children together with their families and friends. As an example, we could connect children in the hospital to their classmates so that they would be able to play together in the playground,” Van Eunen explains.

Teunen continues: “We want to show that it is more fun to invent a game that has a real impact. Aside from that, we want to contribute to the VR revolution in healthcare. VR can really make a difference in healthcare. Not just for children, but also for people with dementia, for example. We are  hopeful that PlaygroundVR will inspire others to make great applications for healthcare as well.”

Virtual reality stress training prepares police for high-risk situations

Virtual Reality-Stresstraining

Virtual reality is also providing new opportunities in police training. In the Shotpros project, psychology-based training scenarios are being developed in order to improve decision-making and action-taking behaviours while under stress. The aim is to avoid violence and collateral damage in high-risk situations.

Firearms attacks, rampages, terrorist attacks – new threat and risk scenarios pose new challenges for the European police. There has been a massive increase in the number of incidents in which police officers are deployed as first responders in threatening situations. First responders are faced with the challenge of assessing the situation accurately and deciding how to proceed. The success of the operation and its potential impact and consequences depend on these strategic decisions.

Virtual reality innovation factor

Traditional training focuses on aspects such as being able to run fast and to use weapons skillfully. However, stress management and strategy are called for first and foremost in high-risk situations. Stress limits perception and thus the ability to make the right decisions and act accordingly. In the Horizon 2020 Shotpros research project, innovative virtual stress training scenarios are now being created which would otherwise be impossible for security reasons. Thirteen project partners from academia and industry are working together with leading police authorities in this multidisciplinary team. The project coordinator is the Viennese user-centred design consultants Usecon.

Project partner Raôul Oudejans

Among the research partners, is also the team from the  Center for Technology Experience at the Austrian Institute for Technology (AIT). Its task is to examine the triggers, management and regulation of stress and emotions within virtual realities – and how they are measured. AIT project team member Sebastian Egger-Lampl  explains that the training principles are based on psychological research dealing with decision-making and its underlying influences. Project partners involved in this part are the experts led by Raôul Oudejans, associate professor at the Department of Human Movement Sciences, Vrije Universiteit Amsterdam, and at the Faculty of Sports and Nutrition, Amsterdam University of Applied Sciences. They also provide expertise in the training to help improve decision-making under stress.

Making the right decisions

The training should lead to better and more sound decisions with regards to law and ethics. Police officers are trained in the ability to maintain an overview in threatening situations, to minimize the use of force and to avoid casualties and collateral damage such as panic and escalation.

“Typically, under stress, very often a tunnel effect happens, which results in not even being able to see, much less process, everything that is going on.” Sebastian Egger-Lampl explains.

The project focuses on the design of a validated human factor model. In this study, the influence of psychological human factors on context-specific decisions and actions of police officers in high-risk situations are to be examined first. This model is intended to provide researchers with a deeper understanding of decision-making processes as well as provide the basis for the development of a future training program.

Training material

The training material is then created based on these findings. These consist of:

  • software with training scenarios in virtual reality;
  • a training plan that combines real training with virtual reality training;
  • project outcomes, which will be circulated as best practice in a Europe-wide virtual reality training network.

Training settings

The stress factors in the program can be altered to allow different settings. The training takes place with headsets and under the supervision of a trainer who controls the virtual reality environment and oversees the trainees. The trainees practice in teams of two, as is also the case in the actual deployment of police patrols.

Training scenario

A hypothetical scenario would be a phone call about a loud domestic dispute in an apartment where threats are heard. The situation on site is quite different: there are several men in the apartment who are attacking each other with various items. It is the trainees’ task to first assess the situation in the best conceivable way and gather all available information in order to then make the most appropriate decision. The virtual reality environment makes it possible to gradually escalate the scenes so as to ensure that the trainee reaches a certain stress level, Egger-Lampl explains.

Shotpros is funded by the European Union to the tune of five million euros. The project duration is five years. The establishment of a European network of various police and security authorities is intended to facilitate exchange and knowledge transfer in the field of virtual worlds within the police sector. The long-term success of the project will be ensured by involving police authorities and relevant stakeholders.

 

Also interesting:

Using VR to overcome fear of heights while climbing

VIVATOP: How Virtual Reality and 3D-printed models make surgery easier

Bildaufnahme OP-Saal mit Displaytechnink

Surgical procedures always  pose a risk. Moreover, x-ray imagery or data from CT or MRT do not reveal everything. A research project led by the Technology Centre for Computer Science and Information Technology (Technologie-Zentrum Informatik und Informationstechnik or TZI) at the University of Bremen aims to change this through new technologies such as 3D printing, virtual reality and augmented reality. These should help surgical teams to better plan their procedures and reduce the risks for the patient.

VIVATOP devises VR solutions for surgeons

Modern surgery has long been a high-tech domain. This applies not only to the operations themselves, but also to their preparation. Surgeons make use of X-ray images, computer tomography (CT) or magnetic resonance imaging (MRI) results and much more. Computer-generated 3D models are relatively new. But they still have to be viewed on 2D screens.

Surgeons still have to employ models made by 3D printers, such as those emerging from VIVATOP. However, there are already experiments with mixed reality applications. For instance, in the field of oral and maxillofacial surgery. VIVATOP stands for “Versatile Immersive Virtual and Augmented Tangible OP”. The project combines model organs from the 3D printer with applications from VR and augmented reality.

VR-glasses and computer simulations help preparing surgical procedures.
Foto: TZI, Universität Bremen

3D printed organ models have many advantages. You are able to touch them and look at them closely, or pass them around as instructional objects. VR glasses allow for an even more detailed display. Participants in meetings are able to hold the printed organ model in their hands and see additional information displayed on their glasses. Communication between doctors and patients is also easier. With the help of the model, doctors can explain more precisely how the upcoming surgical procedure should proceed.

VIVATOP technology facilitates surgery

VR technology and organ models support the actual operation. Because every human body is different, surgeons have to reorient themselves again and again. VIVATOP’s new technology facilitates both decisions and orientation during the operation. The operating theatre is peppered with sensor and recording technology. Depth cameras and other sensors record every movement of the surgical team and the operation on the organ itself.

Printed models such as this liver help to plan and explain a surgical procedure.
Foto: cirp GmbH

The surgical team wears VR glasses during the procedure. These VR glasses constantly display all vital information. The surgeons control their glasses via voice control. Their glasses also give them access to all planning data used in the procedure’s preparation. In addition, operations can be shown live in virtual reality.

According to project coordinator Professor Dr. Rainer Malaka, it would also be conceivable to use the system in telemedicine, for example, by involving experts from a different location. However, there would always be a doctor on site. Yet the possibility of simultaneously following and recording complex surgical procedures live, opens up new possibilities for worldwide cooperation. Experts could follow an operation and support the team without having to be on site themselves.

market introduction in the making

However, it will take some time before the solutions developed within VIVATOP reach operating theatres. “The project will run for three years, after which the industry will have to continue with it,” says Professor Rainer Malaka. In his experience, certifications for new medical devices are costly and time-consuming. Although applications for training purposes and patient education might reach the market sooner.

The Federal Ministry of Research and Education is funding VIVATOP with 2.2 million euros. In addition to Malaka’s research group, other partners are involved in TZI. Another research group is working on virtual reality and computer graphics. The Oldenburg Pius Hospital contributes medical expertise, the 3D organ models are products of the Fraunhofer Institute for Digital Medicine. On top of that, there are industrial partners for augmented reality and 3D printing. Malaka estimates that the first commercial AR applications could be available in 2020.

Using VR to Overcome Fear of Heights While Climbing

Admittedly, an enthusiastic climber rarely has fear of heights. But: he is afraid to fall and get injured. Depending on the climber’s level of expertise, this fear appears in every person at a different height. A beginner may already feel it a few meters above the ground, while an experienced climber shows quite a different tolerance threshold.

Exposure as an Approach

In order to overcome fear, there is a golden method in sports, that comes from psychotherapy: Exposure. The “patient” or athlete repeatedly exposes himself or herself to the situation that causes him or her anxiety. And the good thing about the fear of heights – even if in this case it is not an irrational phobia but is based on the instinct of self-preservation – can be treated particularly successfully. Through constant confrontation, the patient learns that his life is not threatened immediately just because he is a few meters above the ground and the depth lies beneath him. Ultimately, overcoming fear is a matter of practice that requires a lot of patience but is crowned with success.

But be warned against too much ambition: Those who leave their comfort zone too drastically risk panic attacks. And these are accompanied by heart palpitations, shortness of breath, outbreaks of sweat and shaky knees. Symptoms that are quite counterproductive for climbing rock faces. Especially since this bad feeling is also immediately memorized for the next tour. Thus, it makes sense to have a system that comfortably secures the comfort zone. One that guarantees the best possible therapy – or in this case the best possible training success.

Three Test Arrangements

And this is exactly what the enthusiastic climber, student and scientist of the Technology Centre for Computer Science and Information Technology at the University of Bremen, Peter Schulz, is working on. For his master thesis entitled “Effects of Physical Interaction While Sport Climbing in Virtual Reality”, he investigated the extent to which Virtual Reality (VR) can overcome fear of heights in climbing. For this purpose, he conducted three different test attempts with a total of 28 male and female climbers, some of whom were high-performers and some averagely good. All three were to represent climbing at a height of ten meters. He set up his research laboratory in the climbing center of the DAV Bremen.

Foto: Peter Schulz /TZI

On the one hand, the test persons climbed in reality and with safety by top rope. In the second test arrangement, they were on real grips while climbing and were on the same route but close to the ground. The special thing about this situation, was that the test persons were able to get off at any time without any problems, but at the same time had a well-founded fear of falling due to free climbing (Free Solo) in combination with the VR pictures of ten meters climbing height. In the third test arrangement, the test persons only climbed virtually. They were only wearing the VR glasses and had controllers, as they are known from game consoles.

VR with Simultaneous Physical Activity Realistic

In the experiment, which was supervised by Professors Dr. Rainer Malaka and Dr. Johannes Schöning, Schulz was primarily concerned with creating the most realistic impression possible of activities at heights. He wanted to use sensors attached to the test persons to observe the sensation of anxiety and stress. For his investigations, the student then conducted personal interviews. He also recorded heart rate (HR), pulse, rate variability (HRV), resorption rate (RR) and electrodermal activity (EDA), meaning increased sweat production with increased skin conductivity.

Schulz summarizes the results as follows:

We could observe that some effects occur at ground level with Virtual Reality, which without VR only occur at heights. This created a stress level for the climber that corresponds to that of real climbing.”

The triggered feeling of height is therefore related to two factors: The first is physical activity during climbing, and the second is virtual reality, which virtually raises people to the heights. “We were also able to show that something that can be touched is very important – real grips”, says Schulz. Climbing only with the controllers seemed less realistic for the users.

Further Steps Planned

The VR climbing goggles thus seem interesting for everyone who wants to raise his level in the vertical, but has not yet been able to make it, due to fear. This can be a beginner, but also an advanced climber. First of all, it is planned to integrate the findings into effective training plans for climbers. For this, tests will be carried out with the athletes.

Schulz contributed his many years of climbing know-how to the research work, as well as his experience as a youth leader at the DAV – where he is intensively involved in imparting climbing knowledge. The two experts, Prof. Dr. Rainer Malaka and Prof. Dr. Johannes Schöning, are also climbers. At the same time, they are intensively involved in the research areas of digital media and human-machine interaction, including VR. The entire work can be read here.

 

The story of how VR is revolutionizing KLM’s training strategy

Both today and tomorrow Munich will host the Augmented World Expo (AWE), an annual gathering that will be dedicated to three emerging and promising techniques: Augmented, Virtual and Mixed Reality (collectively known as XR). Both developers and representatives of interested companies will be present and attend workshops, participate in lectures and discuss each other’s XR-implementations. One of those representatives is Chris Koomen, VR-pioneer at KLM. Innovation Origins spoke to him about his notable career, his role as a panel member during AWE and about XR-applications in the corporate world.

It was 1996 when Chris Koomen first became acquainted with virtual reality, a technique that at the time still required large, heavy glasses and an incredible amount of computing power. “As a fanatic ‘flight-simmer’ I wanted to be able to step inside of my cockpit through those glasses. That really seemed fascinating to me. Eventually, I got it up and running, but there was no head-tracking built in, and the refresh-rate was not correct at all – you got nauseous very quickly. In addition, the communication between the eye and the hand – everything was controlled by using a mouse – caused a lot of confusion. It was not ideal, but I did see the potential the technique had”, says Koomen. “It simply was too early to use VR. The computers were too cumbersome, and the technology had not developed to the required degree.”

Koomen, who only considered VR a hobby back in the days, has a remarkable career: “From an early age, I wanted to become a pilot. However, I was rejected as a result of an accident. I started working in the agricultural sector for a long time, and for six months I’ve worked as a CV mechanic. Then I joined KLM. In 2007 I was hired as a mechanic, and I was allowed to work on three different kinds of aircraft.” After having worked in the hangar for eight years, time was ripe for something new. Koomen decided to work in another position within the company: “From that moment in 2015 on, I started working on VR again.” He explored the possibilities and was given the opportunity to experiment with a technique that is now widely used within the Dutch-French airline.

Proof of concept

As a proof of concept, Koomen decided to tackle an essential part of the mechanics’ profession: the safety trainings. “In my time in the hangar, I always considered it strange that we were required to evacuate the entire building year after year. Time and again it was to be done in the winter, and you were waiting in the cold until you could enter the building again; that did not seem like the right approach to me. That is why, together with some educational experts, I developed a VR experience in which you can experience a number of safety scenarios for yourself. What we found out immediately was that hangar staff experienced it as an incredibly fun way to learn and that they had also learned much more as a result. When employees make a certain choice in the VR environment, it immediately affects what they are seeing. For example, if you step into the elevator, you will see that it starts filling up with smoke,” says Koomen. “What we have noticed is that users experience it as being so realistic that they start breathing faster and become involved emotionally,” he continues.

Text continues below

“The past two years have been a very turbulent time. Regularly I question myself what actually happened. From scratch, I’ve set it all up, and now there are nine very enthusiastic people working on different types of VR programs,” says Koomen. The tools, which are developed from within the KLM Digital Studio, are used in all kinds of different departments within the company. The Cityhopper team, the catering department and the Engineering and Maintenance division have all recently trained newcomers with the help of VR apps that were developed by Koomen and his team. His efforts resulted in Koomen winning a prestigious prize; of the 32,000 people KLM employs, he was elected ‘Pioneer of the Year’.

Augmented World Expo

“The Augmented World Expo is an event where specialists in the field of augmented, virtual (and mixed reality, ed.) gather,” Koomen explains. More than 100 different companies like Microsoft, Disney and Volkswagen will be showing their ideas, while experts and pioneers will scatter across five stages. “During AWE, I will be part of a discussion panel together with a number of other experts in the field of virtual reality. It is a wonderful event, and I am very proud to be able to share my visions.”

A new year for Manus VR: after NASA follows the consumer market

Eindhoven startup Manus VR kicks off the new year at full speed. The startup is traveling to Las Vegas later this week to join CES, one of the most important technology events in the world. We meet with CEO Stephan van den Brink a few days before and talk about working together with NASA and when to expect the first Manus VR glove.

It was smartphone maker HTC that invited Manus VR to join them at CES this year. Consumer Electronics Show, a leading US technology tradeshow, will start this Thursday and Manus VR from Eindhoven will be there.

Participating in such an event can yield unexpected contacts. The most important contact of the past year? That was NASA without a doubt. We meet Stephan on the third floor of the Videolab at Strijp-S. A small international team is working here on the development of the VR glove. A glove that allows the tracking of your hands and fingers in virtual reality.

Screen Shot 2017-01-01 at 6.58.05 PM

NASA has used virtual reality to train astronauts for years and is actively looking for new technologies in this field at events such as CES. It was at E3, another major tech expo (focused on gaming), where the space agency came in contact with the glove makers from Eindhoven. A Manus VR test glove is now being used to help pilots to prepare for their trips to the International Space Station.

Working with a reputable organization like NASA is important, but the consumer market also awaits. For some time now it is possible to place your pre-order at the site for a VR glove. The glove is not (yet) meant for the consumer market, but a developer version for the early birds who want to start exploring the possibilities of the VR glove. Good news for those who already placed an order: just before Christmas Manus VR finally received the first batch of VR gloves, that will be shipped soon. The first consumer version will hopefully make it this year.

What is the hardest part of making a VR glove? The product is a combination of hardware and design. Besides doing its job (tracking in VR) the glove needs to be comfortable and able to survive sweaty hands and a machine wash. The development of the electronics is one thing, but integrating the hardware in the textile material is difficult, especially when you’re looking at large scale production.

Although the production process takes place at several parts of the world, Stephan is happy to have his company based in Eindhoven. “The availability of suitable people and companies here is an important factor. I can truly say that the ecosystem in this area is superb. The fact that Philips was located here and had so many spinoffs is an advantage. I feel it is one of the reasons that we have been able to grow so fast. Philips headquarters is gone but the knowledge is still around. Eindhoven is an ideal place to develop hardware. The only thing lacking in this region is access to funding.” Manus VR however did manage to get their first phase funded by local investors.

Screen Shot 2017-01-01 at 6.20.45 PM

The office of Manus VR at Strijp-S, left (and below) Stephan van den Brink

Screen Shot 2017-01-01 at 6.22.20 PM

Let’s take a look at 2017. Stephan says he is looking forward to the further development of mixed reality. Adding ‘real’ elements to the virtual world. “At NASA we have added a real closet to the VR simulations for astronauts. That closet is standing in the ‘real’ training room but is also visible in the simulated space environment that they see through their VR goggles.” Laughs: “And in the office we like to try things out as well. Recently we put a track on one of our barstools. It is a funny experience to see that barstool moving in your VR glasses while actually holding it in your hands at the same time.”

[DDW Special] Hidden Gem 4: Hyperspace Collective

During Dutch Design Week, e52 will be introducing you to the festival’s Hidden Gems. Nine special designers who we feel are the stand-outs of this edition. A different Hidden Gem will enjoy the limelight each day of DDW. Today: Hyperspace Collective

It’s all systems go in the world of virtual and augmented reality. That’s why Hyperspace Collective is back for this year’s DDW with an incredible programme at the Effenaar. The network in Eindhoven (and now even further afield) is a collective of entrepreneurs and organisations working with virtual, augmented and mixed reality. There will be young companies such as VRmaster, Dutch Rose Media and Manus VR, as well as institutes like Fontys and GGzE.

The theme of this year’s exhibition is Beyond… Curator Frederike Manders explains: “Last DDW was the first experience of VR and AR for many people, but this year we’re going one step further. We’re going beyond the novelty. It’s not the technology, but rather the concrete applications that are taking centre stage. What can you really do with virtual and augmented reality today?”

Screen Shot 2016-10-05 at 9.15.07 PM

Visitors can come to the Effenaar to experience the exciting things that are already possible or soon will be. The exhibition goes beyond gaming and focuses on applications in education, art, entertainment, healthcare and architecture. The programme varies each day, but here are some of our personal highlights:

Fobie Overcome phobias, like the fear of heights, in a secure environment using 3D glasses and an EEG device on your head. The equipment measures your brain activity as you see depth and adjusts the image accordingly.

AR Storytelling Storytelling which can transport listeners to their desired location, wherever it may be.

Dreamlaze A 3D experience without the glasses. The Dreamlaze is a reclining chair with a closed screen above your head giving you a unique way of enjoying audiovisual content.

Interupsy Discover the impact of an epileptic seizure.

Philips Pavilion Relive the architecture of the past.

We are data – Mirror room Experience what it’s like to become ‘data’ and decide which personal information you do and don’t want to share at an installation outside the Effenaar.

Space Experience Wander through the black and white dungeons of the 18th-century printmaker Piranesi. The installation appeals to all the senses and also involves the physical form.

Screen Shot 2016-10-05 at 9.14.18 PM

3D scanner Make a 3D scan of yourself, and edit and animate it for AR and VR applications.

Colouring pages Draw a dinosaur, for instance, and watch it come to life with VR glasses.

VisitU & Visyon The very latest technology brings patients in lifelike contact with friends and family.

The exhibition open on 22 October with a HoloLens make-athon, in which participants can experiment with Microsoft’s hologram glasses. Sign up here.

The Nine Hidden Gems of DDW was made possible through a collaboration with Dutch Design Daily and DDW curator, Katja Lucas. Want to check out the Hidden Gems yourself? Urban Exploring Tours and KOGA will be organising a special bike tour every morning and afternoon where you will see the selected designers. For more information, click here.