Caption Mask

The ongoing COVID-19 pandemic has necessitated everyday mask-wearing to prevent the further spread of the disease. Although masks are critical for public health and safety, they also muffle the wearer’s voice and cause speech comprehension difficulties during conversations with others. This is a significant access barrier for the deaf community, many of whom rely on lip-reading to understand speech. While clear masks help in facilitating lip-reading, not many people wear them. As a workaround, phone-based real-time transcription applications such as Live Transcribe, Otter, and SpeakSee are widely used by the deaf for auditory accessibility. However, using phone-based speech transcription applications requires deaf people to split their attention between the speaker and the phone’s screens, which forces them to miss non-verbal communication cues such as facial expressions, body language, and other communication signals. “Caption Mask” is a wearable device prototype to facilitate easy access in everyday conversations that uses Automatic Speech Recognition (ASR), a Raspberry Pi, and a 3D printed case/cloth to deliver real-time captions on a face mask.

Hardware Components