Back
A user should feel realistic haptic feedback on a touchscreen embedded with electrovibration technology – independent of the number of fingers used, the movements made, or the pressure applied to the screen. Unfortunately, not even the most advanced touchscreen devices provide a realistic user experience – all images feel like glass. In the near future, a completely rethought design of the electrical signals causing the haptic sensation could close the gap between reality and fiction. In an experiment, researchers at the Max Planck Institute for Intelligent Systems and Delft University of Technology laid the foundation for rethinking the design approach, generating more compelling tactile sensations on touchscreens, and improving the usability of mobile devices.
Stuttgart – An immense number of touchscreen-based devices with haptic feedback exist in the world, but almost none of them let the user feel what is shown on the screen. To give an example: a button on the screen doesn’t feel like a real button; current technology provides nothing more than a mechanical buzz when a person moves a finger across the button or presses it. In fact, the vibrations shake the entire device, and so there is no touch sensation at the fingertip that is comparable to touching a real button. These limitations mean you have to look at your phone, tablet, or car dashboard when you want to enter information, taking your eyes away from the people and environment around you.
Electrovibration offers a new approach to deliver tactile sensations directly to the fingertip. It causes haptic sensations by attracting the user’s finger-pad to the screen using oscillating electrostatic forces; the user can stroke the screen and feel rich textures and quick changes like a button click. However, researchers previously thought electrovibration can’t be felt when the user’s finger is stationary on the screen, which often occurs. Furthermore, if a second or third finger touches the glass while the first finger is moving, the perceived haptic feedback becomes much weaker compared to what one feels with one finger. Turning a virtual knob with several fingers engaged gives less haptic feedback than if only one finger dialed, unlike the real world. Ideally, one day, a user should be able to receive fingertip haptic feedback that feels realistic – no longer shaking an entire device – and can be felt by all fingers, whether they are stationary or moving.
Researchers at the Max Planck Institute for Intelligent Systems (MPI-IS) and Delft University of Technology investigated electrovibrations and the factors that significantly affect what a user feels when touching a screen with built-in haptic feedback. Yasemin Vardar, Assistant Professor in the Cognitive Robotics Department at Delft, and Katherine J. Kuchenbecker, Director of the Haptic Intelligence Department at MPI-IS, jointly published their findings in the Journal of the Royal Society Interface on March 31, 2021. Their paper “Finger motion and contact by a second finger influence the tactile perception of electrovibration” lays the groundwork for understanding how to design electrovibrations to provide better haptic sensations.
Firstly, Vardar and Kuchenbecker found that a single moving finger experiences much larger fluctuating electrostatic forces than a single stationary finger, which helps explain why electrovibration is difficult to feel without finger motion. Secondly, the haptic feedback varies depending on the number of fingers moving across the screen: the more fingers engaged, the less each one feels. Thirdly, it makes a difference whether all fingers move or whether one remains still while another moves. Motion greatly increases the electrical impedance between a finger and the screen, therefore increasing the electrostatic forces that finger experiences. And fourthly, to the scientists’ surprise, about one third of all participants in the reported study felt vibrations even when their finger remained stationary on the touchscreen. Most people feel nothing – the sensation disappears when the finger no longer moves.
Questioning conventional wisdom is key to advancing science
“When Yasemin and I started this project, we believed the popular wisdom that a person cannot feel electrovibration when they hold their finger stationary against the touchscreen, which is a major limitation”, says Kuchenbecker. “However, Yasemin’s careful observations in the lab soon started convincing us that about a third of all people can actually feel electrovibration very reliably without motion. We thus set out to disprove this incorrect scientific belief. Even though we had good experimental evidence, the reviewers of our publication were skeptical of this result at the start of the review process, which shows me we succeeded in changing their thinking and thus the knowledge of everyone in the field of surface haptics. Questioning conventional wisdom is a key part of science!”
“At first, we couldn’t physically explain why a moving finger undergoes larger fluctuating forces than a stationary one. The conventional electrovibration models do not account for finger motion, so they do not express why one cannot feel the electrovibration when holding one’s finger still”, Vardar adds. “We found that keeping the finger immobile alters the electrical circuit between the finger and screen significantly, reducing the perceived forces.”
Vardar also observed that when participants touched the screen with a second finger, the perceived electrovibration weakened. “This effect had not previously been documented either, and it greatly influences what you feel when touching such a device”, Vardar continues. “With this knowledge, future electrovibration devices can be designed so that the displayed force adjusts automatically based on the user’s touch gesture. That way, one can feel a button after all.”
Vardar and Kuchenbecker’s research project is one step toward generating more compelling tactile sensations on touchscreens to improve the usability of mobile devices, automotive control panels, or other interactive products. These new scientific insights may enable companies to create displays that deliver compelling tactile sensations directly to the user’s fingertips, so you can pause the music or answer a call without taking your eyes off the road.
About:
Dr. Yasemin Vardar is currently an Assistant Professor in the Cognitive Robotics Department at Delft University of Technology, and there she is leading the Haptic Interface Technology Lab (HITLab) She is also a guest scientist in the Haptic Intelligence Department of the Max Planck Institute for Intelligent Systems, where she was previously a postdoctoral researcher. She earned her Ph.D. in Mechanical Engineering from Koç University (Istanbul, Turkey). Her research aims to create natural, compelling, and personalized haptic interactions in virtual environments. She investigates the skin deformations that occur during interactions with natural surfaces or tactile interfaces and the resulting perceptual experiences. She also designs new haptic interfaces and rendering algorithms to emulate realistic touch experiences in virtual environments.
Dr. Katherine J. Kuchenbecker is the Director of the Haptic Intelligence Department at the Max Planck Institute for Intelligent Systems. She leads an interdisciplinary team that studies human haptic interaction, invents compelling haptic interfaces for touching virtual and remote environments, and endows autonomous robots with an astute sense of touch. To incorporate the sense of touch into computational and robotic systems, she combines inspiration from neuroscience with engineering, machine learning, and design to uncover the principles that are central for successful touch-based interaction.
Kuchenbecker earned her doctorate in Mechanical Engineering in 2006 at Stanford University in Stanford, California, USA. After ten months as a postdoctoral researcher at the Johns Hopkins University in Baltimore, Maryland, USA, she became the Skirkanich Assistant Professor of Innovation in the Department of Mechanical Engineering and Applied Mechanics at the University of Pennsylvania in Philadelphia, Pennsylvania, USA. She was tenured and promoted to Associate Professor on July 1, 2013, also starting a secondary appointment in Penn’s Department of Computer and Information Science at that time. She joined the Max Planck Society as a Director at the Stuttgart site of the Max Planck Institute for Intelligent Systems on January 1, 2017.