The sense of touch has been mostly neglected so far in technology. However, research in haptics applied to virtual and augmented realities is opening up new possibilities in endless fields, from surgery to gaming. With a touch of Star Trek science
“Computer, end program”. A fan of Star Trek science fiction TV series will immediately recognise this instruction. It is given to shut down the holodeck, where virtual reality becomes as tangible as actual reality itself. How far off is that technology, which basically consists of light you can touch and interact with? Actually, it seems we won’t have to wait for the 24th century to have something similar. Some applications do exist already, and they are based on what is called haptic technology.
Haptic is a fancy word from the ancient Greek haptikos, which means tactile, pertaining to the sense of touch. Haptic technology, therefore, is at the core of all devices that integrate touch. One very simple, everyday example is your smartphone: its interface makes you feel clicks, vibrations and other tactile input. But the possibilities are endless, and many of them might turn out to be revolutionary, for example in the medical field. Imagine a surgeon operating on a patient who is in a different city, and being able to feel the texture of the organs as if they were in the same room, and you will understand the scope of the innovation.
“The technologies that first sparked interest in haptics are those related to remote handling, often in the context of hazardous environments or products,” says David Gueorguiev, a researcher specialised in robotics from CNRS, in Paris, France. “In these cases, the user interacts with a manipulative device or robot through an interface that provides haptic feedback of the interaction.”
Another very important area is virtual reality (VR) or augmented reality (AR). Several technologies are being developed in this field, such as haptic feedback video game controllers, or force feedback gloves. One of these is SenseGlove, created by a Dutch startup. It is a “glove for virtual reality rehabilitation”, as it is described; for instance in stroke rehabilitation with haptic VR simulation, realistic training environments can be provided.
“Tasks of daily living can be simulated while all data is captured by the system. Furthermore, the VR environment can be self-adapting and is self-explanatory. In that sense a therapist can, for example, train with several patients at the same time,” explains SenseGlove co-founder Gijs Den Butter. In practice, the patient finds themselves in a virtual reality environment, where they can safely practice everyday actions such as pouring a cup of tea or removing a pan from the stove. The person can “feel” and “touch” the object even if it is not really there.
One step further, and closer to the dream of a Star Trek type holodeck, is H-Reality, “Mixed Haptic Feedback for Mid-Air Interactions in Virtual and Augmented Realities”. It is a collaborative project supported by the EU programme Future and Emerging Technologies (FET). The principle is basically the same as the SenseGlove. However, project team member Claudio Pacchierotti, a researcher in haptics, CNRS/IRISA, Rennes, France, says: “We believe that this interaction must be enabled in the most seamless way, without employing standard cumbersome haptic systems but instead with wearable and contactless solutions.”
This is made possible by integrating two separate technologies. One is non-contact ultrasound haptics. Pacchierotti explains: “It’s like a loudspeaker at a concert; you can feel it physically”. To note, this promising solution is also being investigated by another EU FET project, Levitate, where ultrasounds are used to make objects float in mid-air.
The other technology developed by H-Reality involves wearable vibrotactile devices, which resemble rings to wear on fingers, much lighter than a glove. Pacchierotti says: “We aim at reaching a level of haptic immersion previously thought unattainable without employing complex haptic systems such as exoskeletons. Our objective is therefore: achieve compelling and high-fidelity haptic sensations with a technology that is easy, comfortable, and enjoyable to use and wear.”
This technology has a wide range of possible applications. One is the representation of data in physical form. “Sorting folders and files on a PC will be a completely new haptified AR/VR experience,” continues the researcher.
“Users will be able to personalise the texture using the ‘start’ button and ‘desktop’. This will enable them to interact better with scientific datasets and apply more nuanced analytical reasoning. Users will thus be able to focus much more on the data semantics instead of on displays and manipulation”. The project intends to contribute to other fields, including accessibility for the visually impaired, arts and education, and, of course, robotics and gaming.
The H-Reality project started in October 2018 and is set to finish on 31 March 2022. So far, it has focused on developing the two technologies separately; the next challenge is to integrate them successfully.
The team is coordinated from the University of Birmingham. Pacchierotti thinks that Brexit will not hinder their research path: “The project will be financed until its natural end and no major impacts on its research outputs are expected. Moreover, it seems that the UK will still be part of the upcoming Horizon Europe programmes: we are very glad about that. However, many other points remain open. For example, Brexit is likely to significantly affect the flow of students and researchers from and to the UK, de facto limiting the opportunities for collaborations. From this point of view, Brexit may well impoverish our continent’s knowledge and innovation potential.”
Nowadays most VR and AR research still focuses on improving the realism of what we see. Nonetheless, there is no true immersion if we are not able to physically touch and interact with the environment surrounding us. For this reason, Pacchierotti and his colleagues consider the research on haptics technologies paramount to achieving any ‘holodeck dream’ in the future.