‘Tangible’ Display Technology Makes Virtual Objects Appear Real
RIT scientists find new use for iPod, MacBook
A. Sue Weisler
A new way of displaying virtual surfaces could reinvent how we color our world with paint, fabric and artwork, and how professionals preview digital pictures before printing.
Developed by Jim Ferwerda and Ben Darling at Rochester Institute of Technology, “tangiBook,” a tangible display system, offers a way to interact with virtual materials. Potential applications include accessing virtual museums, soft proofing—or previewing digital prints—and flipping through digital swatch books of paint, textile or printing samples.
“The way it works is you have a model of some surface like an oil painting or fabric,” says Ferwerda, associate professor in the Munsell Color Science Laboratory in the Chester F. Carlson Center for Imaging Science at RIT. “You can render an image of this model to the screen and as you move around the screen or tilt the laptop, the pattern of highlights and reflections changes the way a real object would.”
Darling, a doctoral student in the color science program at RIT, developed the software application that drives the tangible displays. The custom software blends computer graphic rendering tools and sophisticated lighting techniques to give virtual surfaces realistic appearances. The software interfaces with the standard components of a MacBook Pro laptop computer—the display screen, webcam and an accelerometer that tells how the computer is oriented.
What makes the technology fresh is the way the appearance of the rendered surfaces changes as the viewer adjusts the device or moves around the display. To create this illusion, the software tracks the real-time orientation of the laptop screen and the position of the observer, essential logistics for making virtual objects look realistic from any angle.
“Our system renders new images fast enough that it looks like it’s happening instantaneously,” Darling says. “It takes a lot of processing. For every location on the surface, the system has to figure out how light from every direction is affecting that particular pixel.”
The tangiBook is only the first in a series of display devices that has grown to include the tangiPod—piggybacking on the iPhone—and the tangiDesk, which uses a larger screen attached to an arm. Future possibilities include creating applications for digital paint that convey a richness of real materials and developing three-dimensional tangible displays.
For now, Ferwerda and Darling are exploring how to make the monitor seem to disappear. “We have a painting displayed on the tangiBook and it’s reflecting everything around it,” Ferwerda says. “So, it looks like a painted surface, not a monitor surface. You could hang it on your wall and, as you move around, the properties change the same way the real artwork would.”
Ferwerda and Darling debuted the tangiBook last year at the Imagine RIT: Innovation and Creativity Festival. Since then, they have shared their research at numerous conferences around the world. Darling received the 2009 Mitsubishi Electric Research Laboratories Award for best student paper at the Imaging Sciences & Technology 17th Color Imaging Conference.
Note: Rochester Institute of Technology is internationally recognized for academic leadership in computing, engineering, imaging technology, and fine and applied arts, in addition to unparalleled support services for students with hearing loss. Nearly 16,800 full- and part-time students are enrolled in more than 200 career-oriented and professional programs at RIT, and its cooperative education program is one of the oldest and largest in the nation.
For two decades, U.S. News & World Report has ranked RIT among the nation’s leading comprehensive universities. RIT is featured in The Princeton Review’s 2010 edition of The Best 371 Colleges and in Barron’s Best Buys in Education. The Chronicle of Higher Education recognizes RIT as a “Great College to Work For.”
March 15, 2019
Data science community to gather at RIT for regional DataFest hackathon March 29-31
Teams of three to five students will have 48 hours to mine a complex data set pertaining to a real-world problem. Teams will test their statistical analytic and data science skills to find the best solution.
March 13, 2019
RIT Associate Professor Suzanne O’Handley nationally recognized for mentorship
RIT Associate Professor Suzanne O’Handley has been selected by the Council on Undergraduate Research and the Goldwater Scholarship Foundation as the 2019 CUR-Goldwater Scholars Faculty Mentor Awardee. O’Handley, a faculty member in RIT’s School of Chemistry and Materials Science, was chosen from 10 finalists for her considerable achievements as a dynamic scholar, teacher and mentor.
March 8, 2019
RIT Venture Creations technology business incubator launches four new companies
From eye tracking to thermal cooling to all things mushrooms, RIT’s Venture Creations technology business incubator celebrated the launch of four new startups.
March 6, 2019
Student Spotlight: Showing the artistic side of science
Meet Deirdre Cannon, a first-year biotechnology and molecular bioscience student who loves drawing.