Research Highlights / Full Story

Creating New Experiences for the Blind and Visually Impaired

Stephanie Ludi, associate professor of software engineering and the program's graduate director, is visually impaired. She has dedicated her research efforts to creating new experiences and opportunities for students who are blind and visually impaired.

In 2007, Ludi, along with RIT software engineering professor Tom Reichlmayr, started ImagineIT, a one-week summer camp that brought middle and high school students with visual impairments to RIT from all over the country. A grant from the National Science Foundation funded the five-year project aimed at increasing participation in computing among students with low vision. The students got hands-on experience in building LEGO robots and assembling computers. During the camp's five-year tenure, it traveled to university campuses in California, Arizona, and Maryland.

"The students we worked with over the course of five years were very happy with the experience," says Ludi. "They got to work with robots and program them to do different tasks, which for many students was the first time they got to do something like that. On another level they got to learn about all the different things you can do with technology. Thirdly, it gave students the opportunity to get to know other students who are visually impaired and make friends their own age."

ImagineIT has evolved into Computer Science or CS Academy. The CS Academy is part of a curriculum that Ludi customized called Inclusive Exploring Computer Science, or IECS, which is funded by the National Science Foundation. IECS is based on an established Exploring Computer Science (ECS) curriculum developed at UCLA. ECS is used in schools nationally, but it's not a program that's accessible to people with disabilities. The goal of IECS is to increase the level of participation of visually impaired students in university- level computing degree programs.

"We've taken the idea of accessible activities in robotics and turned it into a curriculum that will have a greater impact by extending the curriculum's original scope to include those with visual impairments," says Ludi.

For two weeks in the summer, the CS Academy will bring high school students from around the country to work on modules, including in robotics and human-computer interaction. The students will use robotics-programming software that RIT has built called JBrick. The programming is text-based and is compatible with screen readers, commercial software that reads what's on the screen to the user.

While here at RIT, the students will live in the residence halls to experience on-campus living. New to the CS Academy will be an online component after the camp's completion. Ludi says the online follow-up is critical so that students don't feel isolated following their shared-group interactions.

Prototype Could Equal Ease of Use in Math Lecture

Ludi also recently received a grant from the National Science Foundation for a project titled "AccessMath: Improving Math Lectures for Low Vision Students through Integrated Video, Note-Taking and Search."

Anurag Agarwal, professor in the School of Mathematical Sciences, and Roger Gaborski and Richard Zanibbi, professors in the Golisano College's department of computer science, are co-principal investigators on the grant.

The goal is to make math lectures more accessible to students with low vision. Initially, it will be tested on university-level students but the application could be used at any grade level. Here's how it works: The commercially sold product Mimio captures text as it's written on a white board that's connected to a magnetic bar. The pens are encased in sleeves that act as transmitters. When the pen is pushed against the whiteboard, the Mimio tracks the location of the pen and records a digital "stroke." The pen strokes will be captured on a server along with the video and audio from each lecture. Simultaneously, the lecture is broadcast in real time on the students' iPads.

"It's a very exciting project to be part of," says Zanibbi. "For students with low vision it's not easy to search through large amounts of visual information. We are trying to develop algorithms to search through on recorded videos and pen strokes as well as course notes using images from lecture videos and handwritten sketches as queries to better support the students' participation both during and outside of lecture."

Zanibbi's and Gaborski's expertise in computer vision and computational acoustics is needed to develop algorithms that index and search the video data.

Agarwal, who teaches linear algebra courses to first- and second-year students, is creating course notes and content for lectures on linear algebra. For example, if a student were to search for the Pythagorean theorem, a2 + b2 = c2, the theorem could be retrieved from the database of archived lectures.

The next phase of the prototype is to roll out some mock-up lectures to low vision students for their feedback on the proof of concept's design and usability.

Increasing Braille Literacy Among Blind and Visually Impaired

It may be a surprising statistic that only 10 percent of children with low vision use and learn braille. Learning braille can be beneficial to build spelling, writing, and general literacy skills. Receiving information purely by ear with the use of a screen reader doesn't enable students to learn concepts like punctuation and spelling.

Ludi obtained seed funding in December from the Effective Access Technology Program offered through RIT's Office of the Vice President for Research to launch the project, Access-Braille: Improving Access to Braille Literacy for Visually Impaired Children.

"Braille literacy is shown to correlate to later success in attaining education and employment," says Ludi. "With AccessBraille we want to create a suite of mobile apps that will facilitate braille literacy along with the students' vocabulary and writing skills."

She is working with software engineering undergraduate students Michael Timbrook and Piper Chester to develop apps for the iPad and iPad mini. The apps include Flashcards, a spelling game and an app in which students can share braille stories and writings with others. The apps will use the iPad touch screen that mimics a braillewriter, enabling users to designate where on the screen they want to type.

Assistive Technologies in Computing To Take Center Stage in Rochester

The design, evaluation, and use of assistive technologies like these in computing will be the focus of an international conference, ASSETS, to be held in Rochester, N.Y., in Oct. 2014. The Association of Computing Machinery (ACM) special interest group SIGACCESS focuses on accessibility and computing and runs the annual ASSETS conference. Andrew Sears, dean of RIT's B. Thomas Golisano College of Computing and Information Sciences, is chair of the SIGACCESS group.

"It's exciting that Rochester is hosting the annual ASSETS conference in 2014 because it brings together researchers, practitioners, and students from around the world," says Sears. "With the conference in RIT's backyard, we hope to get the community of attendees to come to campus to see what research we are doing in this area of effective access technology."

Human-Centered Computing Expert Leading RIT's Computing College

Sears, who's been involved with SIGACCESS for more than a decade, joined RIT as dean of the Golisano College in Aug. 2011. He is an expert in human-centered computing and information technology. His primary interest is accessibility but his research has also explored issues in the areas of mobile computing, health information technologies, interface design, and speech recognition.

"What's most exciting from my perspective is that we have a number of people not just within the Golisano College but across the university who are looking at these access issues from a variety of perspectives," says Sears. "When you look at RIT and the history of the university with NTID, it's hard to imagine an environment that is a better fit for building research in the accessibility area."

Sears' current research with a doctoral student at the University of Maryland, Baltimore County, examines cognitive impairments and cognitive decline in elderly people by looking at how and what they type when using computers for everyday tasks such as e-mail. Characteristics like the timing between keystrokes and the types of words used offer a great deal of information about a person's cognitive state.

"When you put these different cues together, you can do a reasonable job of recognizing a temporary change in an individual's cognitive status. You can also detect differences between older adults who do not have any known cognitive impairments and older adults who have a very mild cognitive impairment."

Sears believes that RIT needs to take a holistic approach to helping people with disabilities and the elderly. He has found that research in this area is often done by individual researchers in isolation, so the resulting approach is to tackle a relatively narrow slice of the problem. "One of the big challenges for people doing research in this area is identifying the real problems and finding real users that can help us define and evaluate new solutions," says Sears.

Working with Rochester area agencies that serve children and adults with special needs gives RIT a broader perspective that simultaneously considers the physical, cognitive, and perceptual issues of people with disabilities. As a result, we can develop more effective and robust solutions that will empower these people.