Research

Collaborators: Matt Huenerfauth, Caluã de Lacerda Pataca, Christian Vogler (Gallaudet), Raja Kushalnagar (Gallaudet)

Funding Source(s): Twenty-First Century Captioning Technology, Metrics and Usability, Department of Health and Human Services.

Project Abstract:  Captioning plays an important role in making video and other media accessible for many people who are Deaf or Hard of Hearing. What are the requirements and preferences of Deaf and Hard of Hearing users for captioning technology for video programming or for real-time captioning in live meetings? We are investigating the preferences of users for new video captioning services and the design of a tool to caption live one-on-one meetings using imperfect automatic speech recognition (ASR) technology, including how to best convey users' emotions visually.  This project studies the presentation, display, and user experience of people who are DHH viewing captions, including through focus groups, interviews, surveys, and experimental studies with DHH participants.  We identify key requirements from stakeholders as to the quality of captions, identify factors that can be used to implement software-based automatic metrics for evaluating caption quality, and identify new methods for modifying the presentation and display of captions, to boost the overall user experience of DHH users.

Project Years: 2018–2025

Collaborators: Dr. Garreth W. Tigwell, Dr. Anne Ross (Bucknell University), Sarah Andrew, Anisa Callis.

Funding Source(s): NSF

Project Abstract: Mobile apps are a vital part of our daily lives, serving billions of users worldwide. However, many apps remain inaccessible to people with disabilities. For example, if there is not enough contrast between the color of text and its background, it may be difficult to read for someone with low vision. Inaccessible apps can lead to exclusion, lost customers, and potential legal issues for businesses. Still, many app designers do not consider accessibility. In fact, they may believe considering accessibility will disrupt their creative process. This project will work with app designers and digital accessibility specialists to explore tools that can link accessibility and creative design, with the goal of improving both the accessibility of apps and designers' experience of designing for accessibility. The project outcomes can also support other mobile app creators with access to fewer resources, such as students and self-employed designers, to ensure they are also supported in creating accessible mobile apps. This initiative will not only improve the accessibility of mobile apps for people with disabilities but also promote a more inclusive digital world by enhancing the usability of mobile apps for everyone.

This research project aims to develop Creative Accessibility Design Tools (CADTs) to assist under-resourced professional mobile app creators in incorporating accessibility into their early design stages. Using qualitative and user-centered methods, the project team will identify the critical features necessary for effective CADTs, focusing on visual design and navigation accessibility. The methods will include diary studies, interviews, and design workshops to understand current design practices and challenges faced by mobile app creators. Insights from these activities will guide the development of CADTs, which will be iteratively prototyped and evaluated with feedback from accessibility experts and target users. The research will explore how CADTs can support creativity while ensuring accessibility, resulting in new guidelines and exemplar tools that academia and industry can use to develop their own CADTs. Outcomes will include improved support for accessible design among under-resourced mobile app creators and contributions to educational resources for teaching accessibility in design courses. The results will be disseminated through academic publications, workshops, and public repositories, ensuring broad access to the tools and knowledge generated.

Collaborators: Matt Huenerfauth, Roshan Peiris, Caluã de Lacerda Pataca, Lisa Elliot, Michael Stinson

Funding Source(s): NSF

Project Abstract:  To promote inclusion and success of D/HH employees in workplace communication, we investigate the use of Automatic Speech Recognition (ASR) technology for automatically providing captions for impromptu small-group interaction. This project includes interviews with DHH users, employers, and hearing coworkers; participatory design sessions and prototype usability testing with users; lab-based studies investigating how the presentation of ASR text output may influence the speaking behavior of hearing colleagues; experimental sessions with pairs or small groups of D/HH and hearing individuals collaborating on a problem solving task while using a prototype ASR communication system; and observations of the use of prototype designs in real workplace settings. The goal is to identify human-computer interaction design and evaluation guidelines for the use of ASR in small group communication; broader impacts include societal benefits and STEM research opportunities for DHH students.

Collaborators: Elissa Weeden, Alex Kalomiris

Funding Source(s): RIT GCCIS FEAD grant

Project Abstract:

Neurodivergent students may face challenges during the writing process. This mixed methods study investigates generative Artificial Intelligence (AI) as a writing support for neurodivergent college students. The research questions include:

• RQ1: What impact does the use of generative AI have on the quality of writing by neurodivergent college students?
• RQ2: How do neurodivergent college students utilize generative AI to assist with writing tasks?
• RQ3: What are the perceptions of neurodivergent college students on the use of generative AI to assist with writing tasks?

The experimental study is a pretest-posttest control-group design with 50 neurodivergent RIT student participants randomly assigned to an experimental or control group. Participants will compose a baseline written artifact, attend an AI workshop, and compose a final artifact, with the experimental group using generative AI. Survey results, written artifacts, and Zoom recordings of writing sessions will be analyzed to address the research questions.

Collaborators: Elissa Weeden, Catherine Beaton, Mamadou Bah

Funding Source(s): RIT AI Seed Funding

Project Abstract:

Composing written work in English can pose challenges for English as a Second Language (ESL) students. This mixed-methods investigation investigates the application of generative Artificial Intelligence (AI) as a means of support for ESL college students in their writing endeavors. This study seeks to answer the following research questions:

• RQ1: What is the impact of generative AI usage on the writing quality of ESL college students?
• RQ2: How do ESL college students employ generative AI to aid them in their writing tasks?
• RQ3: What are the perceptions of ESL college students regarding the utilization of generative AI for writing assistance?

In this experimental study, a pretest-posttest control-group design is employed, with 50 ESL students from RIT being randomly assigned to either an experimental or control group. Participants will initially produce a baseline written piece, attend an AI workshop, and then create a final written work, with the experimental group utilizing generative AI. To address these research questions, survey responses, written pieces, and recorded Zoom sessions of the writing process will be analyzed.

Collaborators: Elissa Weeden, Kathryn Schmitz, Isaac Zhang

Funding Source(s): RIT AI Seed Funding

Project Abstract:

Deaf and hard-of-hearing (DHH) students may face challenges during the writing process. This mixed methods study investigates generative Artificial Intelligence (AI) as a writing support for DHH college students. The research questions include:

• RQ1: What impact does the use of generative AI have on the quality of writing by DHH college students?
• RQ2: How do DHH college students utilize generative AI to assist with writing tasks?
• RQ3: What are the perceptions of DHH college students on the use of generative AI to assist with writing tasks?

The experimental study is a pretest-posttest control-group design with 50 RIT DHH student participants randomly assigned to an experimental or control group. Participants will compose a baseline written artifact, attend an AI workshop, and compose a final artifact, with the experimental group using generative AI. Survey results, written artifacts, and Zoom recordings of writing sessions will be analyzed to address the research questions.

Collaborators: Kristen Shinohara, Catherine Baker (Creighton University), Yasmine Elglaly (Western Washington University), Emily Kuang (York University), Paul Ezeamii, Sara Andrew, Liya Thomas, Kripa Kundaliya

Funding Source(s): NSF

Project Abstract: Creating accessible technology (i.e. technology that can be used by people with disabilities) is an important skill that is often left out of computer science curriculums. When it is taught, it is frequently taught in elective courses such as Human Computer Interaction and Web Programming. We believe that all students should be familiar with accessibility and thus seek to integrate accessibility into foundational computer science courses. In this project, we create and assess modules for core computing concepts that infuse accessibility in existing courses to: (1) effectively include accessibility-congruent technical skill and knowledge into computing topics; (2) facilitate accessibility know-how for faculty across a range of foundational computing topics, while (3) maintaining conceptual integrity in core topics. We have made these modules available for use by other instructors.For more information, visit: https://accessibilityeducation.github.io/

Collaborators (include affiliations if not RIT): Kristen Shinohara, Michael McQuaid (University of Texas, Austin), Murtaza Tamjeed

Funding Source(s): NSF

Project Abstract: This project showed that academic culture creates gaps in access between disabled and non-disabled computing PhD students, systematically disadvantaging disabled students. We identified how ableism—the act of privileging nondisabled people over those with disabilities—manifests across academic systems and culture. We initialized guidance for future students, faculty, and disability services staff for how to support PhD computing students with disabilities. 

We employed qualitative research methods to study how 30 PhD students with disabilities in computing disciplines work around access obstacles in their day-to-day research. Students in our studies were blind, low vision, deaf or hard of hearing, autistic or neurodivergent, and had mobility disabilities. We evaluated 18 university websites and interviewed 17 disability services staff to understand how institutions support disabled graduate students. We interviewed 14 faculty who served as PhD advisors to students with disabilities. Using an ethical/anti-ableist lens, we critically analyzed how resources and attitudes toward disability create barriers for disabled students at the graduate level. We developed guidelines and strategies that incorporate anti-ableist approaches for graduate students, faculty advisors, and disability service offices.

Project Years (if ended): 2019-2025