Professors and students develop groundbreaking tangible display apps for tablets

By Fran Broderick

Computer science professor Reynold Bailey is part of a team of faculty and student researchers that have been developing groundbreaking tangible display apps for tablets that let users experience digital content in stunning, lifelike ways.

Bailey is part of a cross-disciplinary team led by associate professor and Xerox Chair in Imaging Science Jim Ferwerda, that has joined researchers from computer science, computer engineering, computer graphics, and color science to build apps that take advantage of tablet sensors such as gyroscopes and accelerometers to offer a more realistic viewing experience.  Ferwerda, writing in a recent article for the International Society for Optics and Photonics (SPIE) explained, “we developed custom software that takes advantage of these components to allow the orientation of a device and the position of the observer to be tracked in real-time…tilting or moving in front of the device produces realistic changes in the surface appearance.”

Bailey further described the collaboration: “Jim is an expert on material appearance and digital imaging systems, and using my experience in computer graphics and applied perception, we were able to develop dynamic visual representations which behave more like real-world objects rather than flat static images. One example of this is paintings - in many computer-based paint programs, the paint you put down is the same height as the surface you’re painting on. This is impossible in the real world, so instead, we simulate the actual buildup of paint on a canvas. The variation in surface height allows us to take advantage of light source position, and viewing angle to generate more realistic views of the painting.”

The team’s surface viewing app tangiView and its sibling app tangiPaint, which offers users the ability to create their own lifelike oil paintings, are both available through the iTunes app store.

However, the research team, which includes computer science student Okka Kyaw, computer engineering student Anthony Blatner, and PhD in Imaging Sciences candidate Ben Darling, is not stopping there.

The group recently set their sights on developing more realistic 3D models for tablets. Their latest app, PhantoView, takes advantage of a special stereoscopic rendering called a phantogram, to make 2D images appear 3D. And according to Bailey, this is only the tip of the iceberg when it comes to tangible tablet and mobile experiences.

“I think an exciting next step will be to actually feel the textures of paintings on your screen. Teams at Disney Research are already exploring how to use varying electric fields to produce rich tactile sensations on flat display surfaces without the need for moving parts.”

As tablets and mobile devices continue to offer more fluid and lifelike experiences for their users, RIT researchers are contributing novel developments that are likely to affect user experience in significant new ways.