Sorry, you need to enable JavaScript to visit this website.

Search form

Students modify programming and math education applications to make them more accessible to the visually impaired

By Fran Broderick

During the hazy summer months at RIT, three groups of students have been hard at work helping enhance leading math and computer science education programs in an effort to make them more accessible to the visually impaired.

Led by software engineering professor Stephanie Ludi, the student teams are working on updates to Exploring Computer Science (CS) curriculum, a popular CS education software for teenagers, as well as AccessMath, a tool for learning math concepts, and Blockly a popular tool for learning programming.

Software engineering student Mary Spencer ’17 is part of a team that is using JavaScript and CSS to modify Blockly in an effort to make it more keyboard-driven. She explains, “Blockly makes it easy to learn by letting students drag and drop blocks to learn programming. However, visually impaired students use their keyboards more than their mouse so we’re working to create ways to make the program more keyboard-driven.”

The mouse is not the only hurdle the visually impaired face when using these programs. In an effort to help those students better understand their work when using a data analysis program, software engineering student Josh Miller ’19 has taken 127 musical tones and used pitch shift differences to help students understand whether graphs are going up or down. A student can select a type of graph, such as a bar or scatterplot, and then listen to tones that help the user understand the data presented in the graph and how it is displayed.

A separate team that includes software engineering students Christina Howard ’20 and Kimberly Sookoo ’21 is using the SpriteKit developer tool, typically used in game development, and applying it to an iPad-based notation program. Utilizing the team’s modifications, students can use the iPad’s camera to capture notes their teachers write on the board, have notes read back to them, and even upload their own audio notes to help them better understand curriculum.

The student Blockly team is using a similar approach of combining keyboard navigation with audio commentary to make Blockly more accessible to the visually impaired. Game design and development student Melissa Young ’20 explains the team is using recently revamped Accessible Rich Internet Applications (ARIA) standards that provide developers with guidelines for making programs more accessible across platforms and devices. “We’re trying to understand the program from the perspective of the end user and find ways to improve their experience,” she says.

To that end, the team is using a sonification approach similar to the one they used for the Data Analysis tool, only this time the audio is intended to help students navigate code. Anyone who has looked at a sea of code on their screen can imagine the difficulties for the visually impaired, inherent in navigating that code. By allowing users to store audio comments alongside their code, the Accessible team is providing a way for users to quickly jump to sections of their code, listen to their own commentary and understand the section of code they are working on.