Color scientists Jim Ferwerda and Ben Darling don’t need magic to bring virtual objects into the real world: They have the tangiBook. This display technology is more than a digital trompe l’oeil, making two-dimensional objects appear real to the viewer, it’s a useful tool for interacting with virtual materials.
The idea for the display device came to Ferwerda during a brainstorming session.
“We had a research group meeting and we were sitting around talking about material appearance and computer graphics and it just clicked in my head,” says Ferwerda, associate professor in the Munsell Color Science Laboratory in the Chester F. Carlson Center for Imaging Science. “Then I talked with Ben about how to develop it and he went off and created the prototype.”
The computer interface Darling produced combines custom software with the standard components of a MacBook Pro laptop computer—the display screen, webcam and sudden motion sensor, an accelerometer that tells the computer when it is falling and should retract its disk heads. Darling’s software tracks in real-time the orientation of the laptop screen and the position of the observer, essential logistics for making virtual objects look realistic from any angle.
Computer graphic rendering tools and a sophisticated lighting technique imbue virtual scenes with realistic surface lighting and complex textures depending on the observer’s position. The appearance of the materials changes as the viewer adjusts the device or moves around the display.
“The way it works is you have a model of some surface like an oil painting, fabric or an illuminated manuscript,” Ferwerda says. “You can render an image of this model to the screen and as you move around the image on the screen or tilt the laptop, the pattern of highlights and reflections changes in appearance the way a real object would.”
The tangiBook is only the first in a series of display devices that has grown to include the tangiPod—piggybacking on the iPhone—and the tangiDesk, which attaches a bigger screen to an arm. Potential applications for tangible displays include accessing virtual museums, soft proofing—or previewing digital prints—flipping through digital swatch books of paint, textile or printing samples, and creating applications for digital paint that convey a richness of real materials, Ferwerda says. Another possibility for tangi-technology includes developing three-dimensional displays.
“That’s a difficult technical problem,” Darling says. “If we could do that—if it didn’t look like an image at all and could look like a real physical object—I think that would be the ultimate goal of the project.”