SoundVizVR: Sound Indicators for Accessible Sounds in Virtual Reality for Deaf or Hard-of-Hearing Users

Virtual reality (VR) is the use of computer technology to create immersive, simulated environments, capable of mimicking human senses. This technology has increased in popularity and function over the years thanks to affordable, consumer-friendly products like HTC Vive, MetaQuest and Playstation VR. These products offer more than fun, unique games. They allow companies to hold ‘in-person meetings’ while working remotely and give organizations like the military, police and firefighters the ability to provide immersive training experiences for their employees. As VR expands in functionality and use, a need simultaneously rises to make this technology accessible to all.

Professor Roshan Peiris and PhD student Ziming Li at the En-Ability Lab at Rochester Institute of Technology (RIT) realized this need and have created SoundvizVR to help combat VR’s inaccessibility to those with hearing impairments. The project started with the development of sound indicators that could be attached to different in-game objects.They tested several styles of indicators, including ones that displayed directly above the objects making noise, indicators that displayed only on in-game maps and indicators that displayed on both. Ultimately, the latter option was chosen for further development.

Next, Peiris’s team, led by Ph.D student Ziming Li, partnered with the MAGIC Center at RIT to develop two games to utilize SoundvizVR. One game involved locating different sounds such as a cow mooing and a baby crying. The second was a table tennis type game where noises were not the focal point but still critical for determining where and when a ball was hit. Within these games, the indicators took the form of text and icons displayed on in-game maps and both were evaluated to determine their sound accessibility. 

To learn more about this project, read their paper, ‘SoundVizVR: Sound Indicators for Accessible Sounds in Virtual Reality for Deaf or Hard-of-Hearing Users’ or check out his youtube @roshaLP for videos detailing SoundvizVR. To download the SoundVizVR Plugin for development on the Unity 3D platform check here.

SoundVizVR: https://www.youtube.com/watch?v=Yn1pe2mMHnA

Plugin: https://www.youtube.com/watch?v=m8DY4cI1Nck

Paper: https://dl.acm.org/doi/abs/10.1145/3517428.3544817