RIT has a depth of experience in a variety of other established and emerging research areas, including astrophysics, microsystems, and modeling and simulation.
by: Kara Teske May 2010
Game developers at RIT are bringing to life real-world experiences through realistic interactive simulations, providing a new medium for entertainment, communication, and education.
For decades, games have provided a source of entertainment for people of all ages and have had a significant influence on our culture. Today, games are being used to address a wide range of challenges from academia, industry, government, and the military. "At the core is interactive graphics and real-time computing to create simulations that provide a media-rich, realistic, and social experience," says Andy Phelps, associate professor and chair of the interactive games and media (IGM) department at the B. Thomas Golisano College of Computing and Information Sciences.
The research brings together a wide variety of disciplines, to combine visualization and communication in new contexts and platforms. "Other programs might be more heavy on programming or design, but at RIT it is a true blend of left brain-right brain," explains Alex Lifschitz, an undergraduate student in game design and development. "It's RIT's innovation mindset that prevails," adds Phelps. "Where some theorize about a new technology, we go out and build it."
RIT game developers are venturing into the emerging areas of augmented reality and multi-touch/multi-user interfaces. These technologies have the ability to engage people by merging real and virtual worlds. However, it is the interaction in the shared mixed media space that defines the experience. Developers are focused on both the technology and defining the experience in these environments.
"Augmented reality has interface potential similar to that which Nintendo recognized with the Wii," explains Dr. Chris Egert, assistant professor of interactive games and media. "The innovation isn't necessarily the technology, but how it facilitates collaboration. The mere fact my parents can play the Wii and be successful at it actually says something very powerful. Augmented reality has the ability to provide a similar intuitive venue with greater accessibility."
A recently developed game entitled "Zeal" explored the unchartered territories of 3-D augmented reality. Developed by a team of six RIT graduate students—Nimesh Desai, Chip Hilseberg, Jon Lobaugh, Eric Moreau, Josh Wilson, and Jay Austin—the game creates an augmented boardgame where players use their hands to wage war on a shared tabletop. Using a Web camera and a set of printed fiduciary markers, the Zeal system is able to translate real-world movements into game-world actions. Each fiduciary marker is assigned a value or action inside the game and as the players move the markers around the table, the camera picks up the effects of the movement inside the game world. "The game is a very simple test of concept, but it represents some of the future capabilities of games," says Phelps.
The release of the Windows 7 operating system has spurred the development of other multi-touch interface applications. RIT game developers are looking at how to integrate this technology into networks. Multi-touch interfaces operate the same way most surface computers do, but instead of detecting the motion of a mouse sensor, fingerprints are detected with tracking software and used as input into a computer.
Last year, students in the new media interactive development program (within IGM) partnered with new media design and imaging students from the College of Imaging Arts and Sciences to construct their own version of a surface interface, entitled "Sociable." The back-end development was made utilizing both ReacTIVision technology and Touchlib, and the software was programmed in Adobe Flash with Actionscript 3.0. This year, a game development team is also integrating the technology into a tiltable surface table, with similar functionality to the iPhone. A prototype is hoped to be unveiled at this year's Imagine RIT: Innovation and Creativity Festival.
Game developers are also looking at ways to improve the gaming experience through alternative interfaces. To date, multi-touch interfaces allow only a single point of two-dimensional, or in some cases three-dimensional, control. A team of students led by Dr. David Schwartz, assistant professor of interactive games and media, has designed a novel controller that allows multiple points of contact in all three dimensions. The deformable surface controller uses a flexible piece of cloth or rubber, a set of lights, and simple Web camera to capture three-dimensional input within a large area of motion. The user interacts with the controller by pushing down on the flexible surface. The current prototype is able to detect deformations ranging from 2 cm to approximately 30 cm.
These technologies are pushing the frontiers of game playing and interactivity. The future lies in the ability to make these technologies accessible and intuitive for a broad audience.
RIT game designers are exploring to what degree games can be used as a motivator for learning. Dr. Jessica Bayliss, associate professor of interactive games and media, has introduced a foundational course using games to teach introductory computing and programming.
The course was developed with support from Microsoft in an effort to raise retention among computer science students. "The course teaches critical computing skills, which is not always the most interesting topic for people," admits Bayliss. "By using games as a way to teach the curriculum, students become more engaged and attracted to the study of computing." Students take on the perspective of a game developer and software developer to understand how games are designed and programmed. "By putting the skills in the context of something students can relate to, they are able to learn better," adds Bayliss. While the course is taught through the interactive games and media department, it teaches foundational computer science skills that can be applied across all computing disciplines.
In the first year alone, the IGM department saw a 90-percent retention rate among first-year students who took the course. The curriculum has also attracted a wider variety of people who may not normally be interested in the study of computing.
Bayliss and Schwartz are also exploring ways game design and instructional design can inform each other. "Much can be gained by understanding how learning occurs through games," says Bayliss.
Phelps explains, "At its root, games are the embodiment of what children learn as the scientific method: form a hypothesis, conduct the experiment, look at the results, refine your hypothesis, and repeat. Games provide a medium to do this virtually, through trial and error. Players learn the rules of the game and are thus exploring abstract concepts through in-world interactions—the exact process education aspires to."
"The power of games happens when the player experience is so enjoyable the learning becomes second nature," adds Schwartz.
Stephen Jacobs, associate professor of interactive games and media and director of the Lab for Technological Literacy, is leading classes in which RIT students build educational math games for users of the One Laptop Per Child (OLPC) XO computer. The effort brings together Rochester educators, software developers, and RIT students with the international OLPC community. Class efforts support the Math4 project initiated by Sugar Labs, developers of the software platform used by the XO. RIT's participation is bolstered by a donation of 25 XO laptops from Fedora, six second-generation XO laptops from OLPC, and a grant from Red Hat.
The class project that has received the most outside interest is "Fortune Hunter," a dungeon-style computer game in which students solve math problems to conquer monsters and collect loot. The prototype was developed by RIT undergraduate students Mike DeVine, Kevin Hockey, Eric Kenvin, Preston Johnson, Justin Lewis, Jonathan Meschino, and David Silverman. If the final game meets with Sugar Labs' approval, it will be distributed with every copy of Sugar software, putting the game in the hands of over a million students around the world.
The class is being cited by The Humanitarian Foss Project as a model for open source education around the country. Both open source projects and humanitarian projects are being cited as drivers for computing education.
Games are also being explored to influence strategy and decision-making skills. Schwartz has worked in collaboration with the Air Force Research Laboratory on wargame design theory and software development. The military has used wargames since World War II for planning and training. Today's wargames create realistic simulations and allow players to model their own scenarios.
In partnership with the Chester F. Carlson Center for Imaging Science, RIT researchers have been able to automate synthetic scene generation by applying computer vision techniques. The process referred to as Airborne Synthetic Scene Generation, or AeroSynth, blends photogrammetry with computer vision techniques and applies it to geographic scenes imaged from an airborne platform. "By augmenting this technology, we can increase the fidelity of 3-D structure for realistic scene modeling," explains Dr. John Schott, Frederick and Anna B. Wiedman Chaired Professor and project leader.
The team is also looking at how to integrate the visualizations with external factors critical to the scene, such as newspaper clippings, employee records, and other data sets. Using game technology, they are attempting to simulate all of these processes to provide decision makers with a more complete perspective before making decisions.
Critical to any game's success, whether educational or entertainment oriented, is the game's ability to capture fun. "You can build the most technologically brilliant game out there, but at the end of the day it has to be fun," says Egert. Important lessons can be learned from some of the classic games that define the industry. RIT is playing an integral role in preserving these virtual worlds through the Preserving Creative America initiative administered by the Library of Congress, in addition to a partnership with the International Center for the History of Electronic Games (ICHEG) at the Strong National Museum of Play.
As time goes on, it becomes increasingly challenging to preserve the technology and experience of historical games like Atari 2600 and Donkey Kong. Just as it's important to archive film and other historical documents, it's important for students to be able to understand the history of their major and the influence it has had on culture. "With games perhaps the need is even more significant, because unless you can actually play and experience the game, you will not completely understand the significance," explains Jacobs. "That's why we are so pleased to be aiding ICHEG in interpreting their collections and in the design of their emerging national exhibitions."
"Thirty years ago, game designers had to create a fun experience with extremely limited hardware, audio, and graphic choices. Yet despite those constraints, they were able to do so and to this day people still play them for hours and hours." These are the important lessons that will help enable future game designers to capture fun as technology continues to advance at warp speed.