Supercomputing now dominated by X86 architecture

HIGH PERFORMANCE COMPUTING - the area where supercomputers and clusters rule the roost - is the pinnacle of the computer speed pyramid, but not its profit pyramid unfortunately.

Some of the landmark deals in this business are a marketing expense. Instead of putting $$$ millions into a yearly Wall (Fall?) Street Journal ad campaign, a similar expenditure on an IBM BlueGene or HP c-Blade supercomputer will do a better marketing job plus the account presence, of course.

IDC have specialised expert conferences and seminars on the topic all over the world, and held its first event here in Singapore this week.

This market, worth a total of US$ ten billion and with a ten per cent growth rate, is worth keeping track of, especially since the power of these machines today will reflect what our personal computers could be doing a decade from now.

IDC segmented the market into true supercomputers (half a million US bucks or more per unit), divisional (quarter to half million), departmental (US$50K to 250K) and workgroup (below US$50K down to compute workstations). This may vary as we've seen researchers with machines costing way above hundred US grand sitting by their desks, and large multi-teraflop machines on sale for not much above that.

The departmental segment - usually small-to-medium clusters - is the dominant one with a third of the total market value. Why? Well, with so many cores per compute node - a typical dual-socket Xeon 5400 Harpertown, Xeon 5500 Gainestown (yeah, that might be the Nehalem DP name, you heard it here first) or AMD Shanghai machine gives you eight cores, so sixteen of such boxes in a single rack provide 128 cores altogether, plus the added management and storage node processors on top.

With a fast interconnect and loads of cheap memory, such a rack can provide well over a teraflop of peak power and couple of terabytes total RAM - enough for many HPC tasks. And all that is affordable enough to be below the mandatory open tender bid limit for many public institutions who are still the main buyers of such gear.

Notice that, along with the cluster proliferation, the key PC server vendors providing such cluster nodes have gained market share: the old guard like Cray, SGI and NEC are pushed aside while HP, Dell and IBM rule the roost. New kids on the block are Chinese vendors like Lenovo, Dawning, Powerleader and Joint Harvest who ride on the huge Chinese market of high tens of thousands of cluster nodes every year. Expect these guys' share to rise, moving them up from the "Others" group.

Unfortunately, at the processor front, it is the old adage that nice guys - here we mean RISC - finish last.

Alpha, MIPS and HP-PA, the old leaders in performance, features and architectural elegance, are long gone, with only the successors of IBM POWER6 and Fujitsu SPARC64 VII continuing to lead the RISC way. The leader in messiest processor architecture and instruction set over the years, the X86, has entrenched its lead in this otherwise technically discerning field too: welcome to the messed up opcodes and even more messed up memory space, not to mention not too many registers even in the 64-bit mode (which, by the way, enabled X86 supercomputing - thank you, AMD).

But, at least in this century, you can't blame Intel for this alone, as AMD had plenty of opportunities, including the Alpha takeover, to seriously challenge Intel on the instruction set front previously. Talking about Intel, the - still - overbuilt and underpowered cruiser Itanic does have some serious HPC fan base, especially in the Far East, and it seems to match the overall RISC share in this business.

Why did X86 gain so much share of the HPC pie? As mentioned, it's all in the cluster.

Notice that the year of 'sea change' was 2001, the year of Alpha, HP-PA and MIPS demise.

As you can see, the cluster advance brought forward the X86 lead above all others. After all, 2.8 million processors shipped in all cluster nodes combined may not be much compared with the rest of the PC market, but they drown out the SMP and NUMA machines, usually non-X86. And, cluster CPUs grow in numbers at over 40 per cent year on year, according to IDC.

Clusters are popular for their price/performance ratio, power and space savings as well as off-the-shelf hardware, however you have to handle many OS instances and "interesting" cluster resource management and load-balancing issues.

In the second part of the story, we'll talk more on the expansion of HPC beyond academia, as well as IDC's expectations for the near future of this market. µ