COLA Connections Newsletter: Summer 2016

Undergraduate research in computational sensing connects computing and the human experience

At the annual RIT Undergraduate Research Symposium in August, you can expect to see the outcomes of many students’ summer research projects. Some of these presentations will showcase the work of students participating in the Research Experiences for Undergraduates Site in Computational Sensing. The REU Site is funded by a nearly $360,000 grant from the National Science Foundation over three years. Each year a cohort of students spends ten weeks at RIT engaged in research projects in computational sensing. Learn more about the projects, students, and mentors involved from our interview with Cecilia Ovesdotter Alm, a computational linguist in RIT’s English Department. She is the Principal Investigator and coordinates the transdisciplinary summer program with Co-PI Reynold Bailey from RIT’s Computer Science Department.


Can you give me a general overview of the projects students are involved with?

C. Alm: There are five ongoing projects, all of which involve multisource human signals. Two REU students work on each project, mentored by at least two RIT faculty members. One team investigates how to computationally measure attention and engagement when we view a video lecture. This is a valuable area of study for potentially enhancing online learning. Another project seeks to understand how readers interpret and react to emotional texts, including texts about interpersonal violence, using computational linguistics and sensing resources. A third team deals with the challenge of a computer meaningfully mapping where people look in an image to what they say about it and their corresponding facial expressions. A key application is computer-based naming of image regions, and this project builds on research with RIT doctoral student Preethi Vaidyanathan who also serves as co-mentor. In yet another team, students are seeking to gain understanding of how users respond to an educational game environment. This team’s collaboration involves Seneca Park Zoo and addresses issues related to biodiversity. The final team is developing computer models based on medical experts’ multimodal behaviors, for advancing image understanding and image retrieval and categorization technologies.


What is the purpose of the studies? Who are the findings going to benefit?

CA: All projects focus on computing - and computing where humans have a very important role. Projects address issues and problems whose solutions are going to benefit people. We have real-world impact applications in mind, in education and learning, wellness, play and human-computer interaction. Another central characteristic is the connection between different forms of human data – some captured with physiological sensors as well as recordings of speech, face, and eye-movement patterns. Multisource data raise interesting questions and research challenges. How do we jointly acquire different forms of human-elicited data? How do we bring the multiple modalities together in computational analysis? How do we deal with human variation and ambiguity of meaning? Human emotion in computing is further part of several of the studies.

The REU is about computational sensing. How do these projects relate to that topic?

CA: We’re exploring data that are elicited from and related to human beings within our scope of sensing. In the past, this has often been associated with physiological measurements. We are extending beyond that by taking a broader view of sensing, for example involving the language we use, how we look at things, and what our faces convey. These are signals we can collect in noninvasive ways. For example, the eye-tracker doesn’t have to be worn and language data can be text or a recording of what we were saying. In their work, students are challenged to think about complex human behaviors and cognitive processes, asking research questions involving sensing, and working with related hardware, software, and analysis methods.


What students are involved? How competitive was acceptance into the program?

CA: We have a fantastic group of students. Participants are from different universities and colleges. This is our first year and we had a large initial number of applicants, around 150. The program cohort is ten students, of whom five are women and five are men. We value a diverse group. Students span different years and majors, have varying demographic backgrounds, and some are from universities with limited research opportunities. The program also connects students and faculty across disciplines. For example, the team of mentors ( include faculty from College of Liberal Arts, Golisano College of Computing and Information Sciences, and Kate Gleason College of Engineering. Additional RIT faculty are involved in offering workshops to students.


Why did students get involved? What are they gaining from this experience?

CA: Generally, students are interested in experiencing research. We wish to reach students early on in their educational careers and help them get involved in research already at that time. We are hoping to entice a passion for discovery and to encourage participants to continue an engagement with research, perhaps in sensing with humans and computers. The transdisciplinary nature of our program also seems to be an attractor - bringing disciplines together is key for solving challenging problems. Students gain hands-on experience with the research process - exploring the problem, formulating a research plan, designing and conducting experiments, processing as well as analyzing and visualizing data, interpreting results, and disseminating findings. We also have several programmatic activities. A journal club meets weekly in which students discuss scholarship relevant to their projects, and we arrange a weekly seminar class on topics such as technical writing, graduate school, networking, grant writing, and outreach. There are faculty-led workshops, such as a recent one about critical considerations related to data acquisition (with Dr. Laura Shackelford). Students also get to know the Rochester region in team-building field trips, including to Niagara Falls, and we are planning an industrial research lab visit. We are involved in an event with other RIT programs which brings together undergraduate participants with graduate students. Our students also gain practice in documenting and disseminating research by giving talks and preparing technical reports. Overall, the program provides valuable experiences for future careers and research endeavors.