Caption Mask

Location

Golisano Hall - Atrium

View Exhibit on Map

The ongoing COVID-19 pandemic has necessitated everyday mask-wearing to prevent the further spread of the disease. Although masks are critical for public health and safety, they also muffle the wearer’s voice and cause speech comprehension difficulties during conversations with others. This is a significant access barrier for the deaf community, many of whom rely on lip-reading to understand speech. While clear masks help in facilitating lip-reading, not many people wear them. As a workaround, phone-based real-time transcription applications such as Live Transcribe, Otter, and SpeakSee are widely used by the deaf for auditory accessibility. However, using phone-based speech transcription applications requires deaf people to split their attention between the speaker and the phone’s screens, which forces them to miss non-verbal communication cues such as facial expressions, body language, and other communication signals. “Caption Mask” is a wearable device prototype to facilitate easy access in everyday conversations that uses Automatic Speech Recognition (ASR), a Raspberry Pi, and a 3D printed case/cloth to deliver real-time captions on a face mask.

Caption Mask 3D Printed Model

An image collage showing all the hardware components

Hardware Components

Location

Golisano Hall - Atrium

View Exhibit on Map

Topics

Exhibitor
Roshan Mathew
Emily Lederman
Carson Schwab

Advisor(s)
Prof. Tae Oh

Organization
Class Project


Thank you to all of our sponsors!