Sun eyes supercomputing glory

You could call it switchzilla.

Sun Microsystems on Monday revealed the Constellation System, a high-performance computing platform that company executives claim will vault the company back into the top ranks of supercomputer manufacturers.

The linchpin in the system is the switch, the piece of hardware that conducts traffic between the servers, memory and data storage. Code-named Magnum, the switch comes with 3,456 ports, a larger-than-normal number that frees up data pathways inside these powerful computers.

"We are looking at a factor-of-three improvement over the current best system at an equal number of nodes," said Andy Bechtolsheim, chief architect and senior vice president of the systems group at Sun. "We have been somewhat absent in the supercomputer market in the last few years."

The Texas Advanced Computing Center (TACC) at the University of Texas is currently preparing a Constellation system. If TACC can get enough Barcelona chips from Advanced Micro Devices by Oct. 15, its system will land near the top of the next Top 500 Supercomputers list, Sun says.

The TACC system will provide a peak performance of around 500 teraflops, or 500 trillion operations a second. A fully built-out Constellation system, with contemporary components, could hit a peak of 2 petaflops, or 2 quadrillion operations per second. In the last Top 500 Supercomputer list, published in November, IBM's BlueGene topped the list with 280 teraflops. (The new list comes out later this week.)

More details, along with other supercomputing papers from competitors, will be presented at the International Supercomputing Conference in Dresden, Germany, this week.

Sun's Magnum switch, based around the InfiniBand high-speed networking technology, is a honker. The largest InfiniBand switches on the market contain 288 ports, according to Bechtolsheim, and require leaf, or helper switches. (TACC's system will have two of the Sun switches.)

The density of ports, and the large number of them, creates a cascading effect in performance and pricing, he asserted. By deploying Magnum, which sports a "fat tree" style architecture where servers branch out from the trunk of switches, customers will need to install far fewer switches when building large computers, he said. Fewer networking boxes mean about one-sixth the number of cables.

"The cables cost more than the silicon" when it comes to the networking systems inside supercomputer clusters, he said.

Overall, Sun claims a fully built-out Constellation system will take up 20 percent less floor space.

The architecture of the system also cuts down latency, a big factor in performance. Because more boxes can connect directly to the switch, processors at distant nodes don't have to leap through as many connections to communicate, according to Sun. Specialized connectors further boost performance.

Sun has also improved the density of the blade servers that are part of Constellation. A 42U-high rack of the blades will hold 768 processor cores, assuming four core processors are used.

The storage system that comes with Constellation can hold one petabyte in two racks. While Constellation supercomputers are constructed out of these separate blades, storage systems and switches, the parts will be sold together rather than separately.

Constellation vs. Blue Gene/L
Bechtolsheim extrapolated on how a hypothetical Constellation system would do against a similarly configured hypothetical IBM Blue Gene/L system.

A Constellation with 131,000 processor cores could churn 1,080 teraflops, or calculations, per second. (A teraflop is a trillion operations a second). The system would also have 3 terabits per second of I/O bandwidth from the storage system.

A Blue Gene/L with 131,000 cores would operate at 360 teraflops and have only one terabit per second of I/O bandwidth with disk storage, according to Sun.

Both Constellation and Blue Gene/L are clusters--large computers created by lashing together large numbers of smaller servers.

"The main difference with BlueGene is the topology of the fabric," Bechtolsheim said. "The advantage [with Fat Tree architectures] is that you have constant latency between nodes."

IBM, Cray and others, of course, aren't exactly standing still. Each company is readying its own products and will make announcements at the International Supercomputing Conference. One source said IBM plans to debut its next-generation Blue Gene design, called Blue Gene/P, in which P stands for petaflop--a quadrillion calculations per second.

Sun also has to wait for Advanced Micro Devices.

Constellation blades can accommodate Sun's UltraSparc chips, AMD processors and Intel chips. AMD, however, currently provides better performance on floating point calculations than Intel's chips, according to Bechtolsheim. The TACC system is based around Barcelona. Whether or not the TACC system can make the next Top 500 list revolves around availability of Barcelona, which is due in the third quarter.

"It depends on AMD," he said.

Interestingly, switches generally have higher margins than servers, Bechtolsheim said. Sun, however, won't sell the switch separately. In that case, Sun would have to partner with IBM, Hewlett-Packard or other server competitors to sell the switch, and they likely aren't too interested.

Bechtolsheim also pointed out that supercomputers generally are less profitable than selling high-end servers to corporations, but companies use supercomputers to conduct research for their other product lines. Sun hence says it will play a more prominent role than it has in the past few years.

"We're getting better and more successful at bidding on these deals," he said. "We're back to where we wanted to be all along."