Big Bangs from supercomputers
When black holes crash into each other at the center of a galaxy, the safest place to be is on the other side of the computer simulating the drama.
Scientists at the Center for Computational Relativity and Gravitation simulate cataclysmic collisions and the evolution of galaxies using supercomputers to churn out computations that would sizzle the latest desktop model.
In fact, the center is home to one of the fastest computers in the world: gravitySimulator, a special-purpose machine David Merritt purchased in 2004 with $600,000 from RIT, NASA and the National Science Foundation.
Merritt, a professor of physics, uses gravitySimulator to study gravitational forces causing black holes to form, evolve and interact with stars and to predict what happens after black holes collide.
The cluster contains 32 nodes, each housing a special-purpose accelerator board called a GRAPE, or GRAvity PipEline, and processes data at the speed of 4 teraflops, or four trillion computations (floating point operations) per second. The GRAPEs, imported from Tokyo, are specially designed to carry out gravitational force calculations.
“GravitySimulator is 1,000 times faster than a standard desktop computer,” says Hans-Peter Bischof, associate professor of computer science and member of CCRG. “The machine can handle four million particles – each representing a star. And for this kind of problem, that’s enormous.”
Bischof illustrates the data Merritt collects from the gravitySimulator using a visualization system he designed. His mini-movies are among the first to depict gravity-force calculations of such large size.
Merritt hopes to double the size of the three-year old cluster and use the gravitySimulator to visualize other components of galaxies, such as gas clouds.
Currently, he is tooling up to use gravitySimulator for the first stage of the Virtual Galaxy, a scheme to simulate the entire Milky Way on a star-by-star basis.
“This project probably won’t be completed in my lifetime,” Merritt says, “but we hope to be able to simulate the central bulge of the galaxy, roughly a billion stars, in the next few years.”
A second computer cluster known as “newHorizons,” unveiled in October 2007, will maintain the center’s competitive level of research in computational astrophysics and numerical relativity, a research field dedicated to proving Einstein’s theory of general relativity. This state-of-the-art computer was designed to compute the numerical-relativity evolution of binary black holes. The $470,000 computer was purchased with funds from separate grants, including an award from the NSF Major Research Instrumentation Program won by Manuela Campanelli, the principal investigator on the grant, Carlos Lousto, Merritt and Yosef Zlochower.
The computer, built with hardware from California-based Western Scientific, boasts 85 nodes – each with its own dual processor – and four amounts of computing units per node and high-speed Infiniband interconnections.
Today’s typical desktop computer has 2 gigabytes of memory. By comparison, each node in newHorizons has 16 gigabytes or a total of 1.4 terabytes of memory. In addition, infinite band technology makes the computer especially fast, moving “packages” of information with a lag time or latency of 12.9 microseconds. The computer, which will have 36 terabytes of storage space, will – like the gravitySimulator – operate at its maximum capacity 24 hours a day for four to five years.
“Other scientists have satellites and telescopes to do scientific research,” says Zlochower, an assistant professor in the School of Mathematical Sciences. “We have supercomputers. It’s how we implement and test ideas. And because our simulations can take weeks, we needed the fastest machine possible.”
The two computers share an air-conditioned room that never rises above 62 degrees Fahrenheit. They were configured to maximize airflow and space between the clusters to prevent heat-related damage. An automated alert system connected to a heat sensor will detect a rise in room temperature. And, if the electricity fails, powerful back-up batteries will keep the computers going for 15 minutes, allowing the machines to shut down without damaging hardware or losing data.