Sneak Preview: Intel "Tiger 4" Quad-CPU Itanium 2

SO, WHILE BUSY compiling the massive feedback on our 64-bit saga for its third part [Ed: you thinkin' of selling movie rights already?], I actually had a time to continue looking at one box that consumed a lot of my time since this March. Yes, that's "Tiger 4" or "Bandera" or simply SR870BN4, its final production name.

This is the long-talked about Intel reference quad-processor Itanium 2 design for OEMs, designed so well as to give even "zero R&D" (Ed: last time we said "-1%") 32-bit companies like Dell a shot in the balls to jump onto the Itanic2 luxury ocean liner, without having to pay first-class cabin tickets. Anyway, it seems HPQ has already reserved most of the first class, in exchange for burning their "Alpha" and "HP-PA" lifeboats - whether they'll end up like "Jack" DiCaprio in the movie (you all know how), of the fat girl's rich, nasty fiance (shot himself a while later), we'll see...

Good looking...
SR870BN4 comes in a compact 4U rack optimised enclosure. As of today, 4P-4U 64-bit servers are still rare - The HPQ ES45 Alpha takes 8U (simply because their corporate did not embrace the HPQ Custom Systems division's excellent "LP" 5U design for the general use), and HP rx5670 quad-Itanium 2 server takes out 7U (presumably to allow in-box upgrades for the guinea pigs who had their first Itanium server in the same form factor). The Sun V480 is at 5U - so, the only similar 4U quad-CPU box is the POWER4-based IBM p630, the only direct performance competitor to Itanium2 at this stage (again, I'm not counting already murdered platforms).

When coupled with its accompanying black front bezel, the box looks pretty cool - under the huge area for fan air intake, there are three hot-swappable Ultra320 SCSI HDD bays, plus a slim notebook DVD-ROM and LS-240 superfloppy drives (OEMs could use combo DVD/CDRW or even DVD+RW drives instead). On the right edge there are diagnostic (system, power, cooling fault) and power LEDs, as well as power, reset and system interrupt buttons. The 1,200 Watt redundant (if you are on 200+ volts) autoswitching power supplies require two power connectors (pne per power supply) for this baby to have redundancy.

...easy to undress...
Taking this baby apart was the easiest job of any quad-CPU server I've seen up to now. Literally, everything is modular and pulls out by two plastic handles, without proprietary screws - easy serviceability was the goal here. Once the fron bezel is off, you'll see two front modules - the larger top CPU/memory module, and the bottom storage module. The CPU/memory module, when pulled out by opening two plastic handles, is an EMI-certified, structural foam with steel mesh-enclosed jewel suitcase-like box, with only one long connector visible - the Itanium2 E8870 bus to the Scalability Port of the E8870 chipset. In practice, an OEM could design their own backplane with the switch to put up to 8 of these quad-CPU modules to have an easy-to-deploy, modular 32to512-CPU NUMA Itanium2 configuration.

Once you open the metal clamshell, you'll see the contents of the CPU/Memory module: a dense, dual-sided 11x15 inch board with two CPUs, the E8870 SNC (System Node Controller) chip, and 8 DDR DIMMs on each side of the board. The SNC connects to 4 DDR Memory Hubs or DMH. The 4 high-speed interfaces connecting to the 4 DMH has a maximum aggregate bandwidth of 6.4 GByte/sec. So, the SNC and DDR DIMMs are interfaces, guess what, through translator chip (not infamous MTH, but MRH-D this time). Why?

Well, E8870 SNC actually has quad Rambus channel memory interface, a legacy from the design days when Rambus was Intel's corporate choice. Since Rambus is still too expensive for servers, our friends at Intel designed MRH-D, which converts each PC800 1.6 GB/s Rambus channel into a PC1600 1.6 GB/s DDR interface. You got four of those in parallel for 6.4 GB/s total bandwidth, the same as Itanium2 FSB bandwidth. Since it is synchronous, the latency penalty is minimal compared to the infamouse "Camino" i820 MTH, but still exists - if you have a chance, compare E8870 with HP zx1 assuming everything else in the system is identical.

Each CPU has a heat sink very similar to those seen on Xeon processors, plus a voltage regulation module (PowerPod) next to it. The CPU itself is on a credit card-sized minimodule "interposer", the bottom of which contains its 611 pins, and one of the sides contains a gold-plated connection to the PowerPod. The top and bottom board side are each other's mirror. With 2 GB registered ECC DDR DIMMs, you could pack 32 GB linearly addressable RAM into the box.

In this system, instead of the large NUMA switch, the Scalability Port connects to an I/O board run by E8870 SIOH chip, accessible directly by a top cover opening. The modular, removable PCI card cage has 8 64-bit PCI-X 64-bit hot-plug slots via three E8870 P64H2 PCI/PCI-X bridges, across 3 PCI-X 133 MHz buses (1066 MB/s each) and 3 100 MHz buses (800 MB/s each) - this gives you a plenty of I/O bandwidth (up to 4 GB/s total sustainable from four hublink 2 interfaces), almost enough to saturate the 6.4 GB/s memory subsystem. This enables you to, for instance, use Quadrics Elan supercomputing interconnect (yet another piece of brilliant, but not that widely know UK technology) in parallel with multichannel SCSI, Fibre Channel, 10 G Ethernet, all simultaneously. The early betas also had an unused HI connector for an optional InfiniBand daughtercard using the now-discontinued E8870 VXB, allowing for 4 1x or possibly one 4x InfiniBand (2 GB/s). Intel will not offer that card. The slot will not be present in the final systems.

The I/O board also has dual Ultra320 SCSI (total of 640 MB/s peak throughput) using LSI 53C1030 intelligent PCI-X based controller, as well as i82540 low-end copper Gigabit Ethernet (I'd rather like to see the much faster i82546 Dual PCI-X Gigabit Ethernet chip in this kind of system). There is on-board ATI RageXL graphics, and the usual bunch of USB 1.1 and other interfaces (but no PS/2 keyboard or mouse - got to use USB). The platform is IPMI-compliant - with Intel Server Management 6.0 supporting the standard server alerting, remote access diagnostics and repair; emergency management and hardware monitoring. It also managed hot-swap redundant cooling with 4 big hot-plug fans.

...and doing it well
Well, with the awful performance experience from the first Itanium, we did not expect this box to have stellar performance - nevertheless, it was quite a positive surprise. Due to continuous firmware, OS, kernel and compilers updates before its launch in the next few months, the numbers are due for quite a change, so no benchmarks yet to show publicly - hold on, they'll be coming soon...

I have run 64-bit Red Hat Linux 7.2 and Windows .NET 64-bit limited edition (whatever that means) on this system - both are Itanium-native, but not Itanium2-optimised distributions. Even then, the .NET thing installed and ran flawlessly (strange for a M$ product - I even set up cluster head and later cluster compute node automatically through a wizard!!!) while RH 7.2 needed a fresh kernel and some SCSI & Ethernet device driver patching during the installation. After that, everything went fine. The machine's noise is unique - generally, it is not among ear-blasters, but once in a while, if you leave it doing something, you'll head a sudden burst of fan noise as the fans are pushed to higher rpm to get the hot air out - but it will run for days and get the job done. I do not know yet whether HP will support this system under HP-UX, since Intel's E8870 and HP zx1 chipsets are quite different.

More oomph, please...
Talking about that, these differences are the key problem for wider acceptance of this system among non-HP vendors. Namely, zx1 has better memory bandwidth (up to 12.8 GB/s if using repeaters on the highest-possible config, or 8.5 GB/s without bridges with PC2100), somewhat better memory latency due to direct DDR interface, added AGP 4X for 3-D, plus similar PCI-X I/O - all in far fewer chips than E8870. This does lead to cheap yet fast HP Itanium2 systems which leave little room for competitors trying to sell Tiger 4-based solutions - even if HP's is a 7U fatso, while this one is a 4U healthy specimen. Of course, HP's also got a slim dual-CPU 2U Itanium 2 "Twiggy" which supposedly goes around for less than US$ 10K with two CPUs, while there is no dual-CPU reference Itanium 2 design by Intel. In any case, Intel still got time to work out very competitive pricing for the Tiger 4 before its soon expected official announcement.

So, to help the poor "non-HPQ" OEMs, maybe the successor to E8870 chipset should have some critical improvements: besides supporting the 533 MHz effective FSB (Itaniums are quite sensitive to the memory performance), we also hope to see direct quad-channel or eight channel DDR for more capacity and bandwidth, AGP 8X for workstations, and at least one working direct-attach InfiniBand 4X for inexpensive clustering with single interconnect taking care of everything: network, storage and clustering protocols all on one ultrafast copper cable. Such "Madison" ready boxes will not only help these vendors, but also help Intel hold its stand against upcoming POWER4+ IBM boxes. Those little, slim and increasingly affordable systems, when combined with improving IBM compilers and hopefully competitive pricing, will aim for the performance and marketshare top... µ