Wearable device enables users to feel objects inside a virtual world

UNSW Sydney engineers have invented a new wearable haptic device that recreates the sense of touch in virtual and remote environments.

Haptic technology mimics the experience of touch by stimulating localised areas of the skin in ways that are similar to what is felt in the real world, through force, vibration or motion.

Dr Thanh Nho Do, Scientia Lecturer and UNSW Medical Robotics Lab director, is the senior author of a study featuring the new device.

His research team featured lead author and PhD candidate Mai Thanh Thai, Phuoc Thien Phan, Trung Thien Hoang and collaborator Scientia Professor Nigel Lovell, Head of the Graduate School of Biomedical Engineering.

Dr Do said the sense of touch was something many people took for granted to perform everyday tasks.

“When we do things with our hands, such as holding a mobile phone or typing on a keyboard, all of these actions are impossible without haptics,” he said. 

“The human hand has a high density of tactile receptors and is both an interesting and challenging area to encode information through haptic stimulation, because we use our hands to perceive most objects every day.

“There are many situations where the sense of touch would be useful but is impossible: for example, in a telehealth consultation a doctor is unable to physically examine a patient. So, we aimed to solve this problem.” 

Dr Do said the researchers were so excited about their new haptic technology that they had applied to patent it.

“Our three-way directional skin stretch device (SSD), built into the fingertips of the wearable haptic glove we also created, is like wearing a second skin – it’s soft, stretchable and mimics the sense of touch – and will enable new forms of haptic communication to enhance everyday activities,” he said. 

“What’s also special about our new technology is that it’s scalable and can be integrated into textiles for use in various potential applications such as telehealth, medical devices, surgical robots and training, augmented and virtual reality, teleoperation and industrial settings.

“The device aims to solve a common problem in emerging systems – such as assistive devices, remote surgery, self-driving cars and the guidance of human movements – where visual or auditory feedback can be slow, unintuitive and increase the cognitive load.”

How the new haptic device works 

Dr Do said the researchers’ new technology overcame issues with existing haptic devices by introducing a novel method to recreate an effective haptic sensation via soft, miniature artificial “muscles”. 

“Our soft, wearable haptic glove enables people to feel virtual or remote objects in a more realistic and immersive way. The inbuilt soft artificial muscles generate sufficient normal and shear forces to the user’s fingertips via a soft tactor, enabling them to effectively reproduce the sense of touch,” he said. 

“It works like this: imagine you are in Australia while your friend is in the United States. You wear a haptic glove with our integrated three-way directional SSDs in the fingertips and your friend also wears a glove with integrated 3D force sensors.

“If your friend picks up an object, it will physically press against your friend’s fingers and their glove with 3D force sensors will measure these interactions. 

“If these 3D force signals are sent to your haptic glove, then the integrated three-way directional SSDs will generate these exact 3D forces at your fingertips, enabling you to experience the same sense of touch as your friend.”

This diagram illustrates how the new soft skin stretch device (SSD), developed by UNSW Engineering researchers, works. Image courtesy of UNSW Engineering

Implications of the new technology 

Dr Do said the ability to effectively reproduce the sense of touch via the new wearable haptic device would have a wide range of benefits; for example, during today’s COVID-19 pandemic when people were relying on video calls to stay connected with loved ones. 

“Unlike existing haptic devices, our technology is soft, lightweight, and thin and therefore, we hope users will be able to integrate it into what they’re wearing to provide realistic haptic experiences in settings including rehabilitation, education, training and recreation,” he said.

“Our technology could enable a user to feel objects inside a virtual world or at a distance; for example, a scientist could feel a virtual rock from another planet without leaving their lab, or a surgeon could feel a patient’s organ tissues with surgical tools without directly touching them.” 

Dr Do estimated the new technology could become available in the next 18 months to three years – if plans to commercialise the device were realised. 

“The next step is to conduct user evaluations to validate how effective our device is, because the main scope of our current research has been on the design, fabrication and characterisation of the new technology,” he said.

“In addition, we plan to implement the device in various haptic applications such as haptic motion guidance, navigational assistance for older people and those with low vision, tactile textual language, and 3D force feedback display for use in surgical robots, prosthesis and virtual and augmented reality.” 

The UNSW study was recently published in the Institute of Electrical and Electronics Engineers (IEEE) Access journal.