Dell enters Opteron Hell

WE TOLD YOU ABOUT DELL and Opteron boxes a few days ago, and yesterday, I got a nice letter about it from Dell.

They seem to not like it that we published some pictures of the benchmarks they are sending to customers. The fact that they were pictures, not the benchmarks itself appears lost on Dell. Here is a tale about those benchmarks in Fairy Tale format, starting with what they said:

'Charlie: In your Dec. 10 article entitled "Dell sells AMD Opterons through back door," you display images of two slides that are labeled as Dell Confidential. I'd like to ask you to pull those from your Website. Thank you.'

Well, I never liked those numbers at the best of times, reading this, you can see I thought they were slanted badly at best, with no disclosure of methods as icing on the cake. It was almost like they were afraid of people not NDA'd to death being able to discuss their numbers. The fact that they are supposedly publicly available in other forms also seems lost on them.

Oh well, I can only conclude that they are petrified of the situation they face in 2005 on the server front. In fact, at least one source tells me that bids on large clusters are being subsidized by third parties to the exact price/performance numbers of an Opteron cluster. I guess it pays to be a name nowadays, even if you don't get Pixar-esqe discounts. No word on whether or not Dell will subsidize your electricity bill.

Now, normally, I would tell them no, but luckily I strongly disagree with the numbers they presented. It would take a lot of drugs to have the current reality mesh with those numbers on a good day. Sadly, I am way to lazy to hunt down accurate ones. Luckily, an enterprising reader was not so lazy, and did an incredible job of hunting down the numbers. He even did something that is apparently taboo at Dell, included references.

This is what they were afraid of you people seeing, and if you are thinking of purchasing a Dell, the x52 Opterons are just around the corner, which as you know, increases subsidies in relationship to benchmark numbers.

Here is what, we'll just call him A. Nonymous, wrote, starting with the benchmarks. Please note, I had to chop them up and redo them to fit on the page. Any minor inconsistencies are my fault, not a conspiracy. Also, all numbers are normalized to the Opterons at 100%.

And now to the letters:

Hi Charlie,

I did a thorough investigation of the Dell benchmark slides in your story to see whether or not they could be trusted. See the attached image for a summary of my findings; where the Opteron results are normalized to 100% and Xeon results relative to that.

Here are my findings in more detail:

Abaqus/Standard 32-bit:

Dell claims that the Xeon is 4% faster than the Opteron, but it's actually only 3.1% according to this.

Amber 8 64-bit:

Dell claims that the Xeon is 15% faster than the Opteron, but they're then comparing Xeon 3.4GHz with Opteron 244 (1.8GHz) which are quite unreasonable I think:

See here.

Fluent 6.1 64-bit:

There doesn't exist any 64-bit version of Fluent 6.1, since 6.2 will be the first version that gets 64-bits support and that's not available yet according to this:

here and here.

Dell claims anyhow that the Xeon is 4% faster than the Opteron in Fluent 6.1, but that dosen't comply with official benchmark numbers that clearly shows that the Opteron outperforms the Xeon: here.

Perhaps Dell is using an unofficial beta version?

LINPACK peak 64-bit:

Dell claims that the Xeon is 60% faster than the Opteron, but that's not correct according to this And see this.

Even though a comparison of Xeon 3.6GHz to a Opteron 248 (2.2GHz) perhaps is a bit unreasonable, it's only 16% faster and far from what Dell claims.

SPECfp2000 64-bit:

Dell claims that the Xeon is 9% faster than the Opteron, but that is not correct for either Base or Peak Score.

Xeon is only 2.9% faster than the Opteron in SPECfp2000 (Base Score) according to this.

But the Opteron is 3.8% faster than the Xeon in SPECfp2000 (Peak Score) according to this.

Linux build 64-bit:

Dell claims that the Xeon is 9% faster than the Opteron, but are they talking about compile time of the Linux kernel or just general 64-bit Linux performance?

Opteron is up to 29.8% faster than the Xeon compiling the Linux kernel according to this.

And according to this there is little doubt that the Opteron is generally faster than the Xeon on 64-bit Linux: here.

Dell calls this HPC benchmarks, but where are LS-DYNA, BLAST, BLAT, HMMER, STREAM, etc?

SPECfp2000 (Base Score):

Dell claims that the Xeon is 8.8% faster than the Opteron, but it is only 2.9% faster according to this.

However, in SPECfp2000 (Peak Score) the Opteron is 3.8% faster than the Xeon: See this.

Perhaps it's this magical 9% number that Dell used in the benchmark first slide?

SPECint2000 (Base Score):

Dell claims that the Opteron is 3.2% faster than the Xeon, and for once that is correct according to this.

However, in SPECint2000 (Peak Score) the Opteron is 6.4% faster than the Xeon according to this.

SPECjbb2000 (OPS/s):

Dell claims that the Opteron is 1.6% faster than the Xeon, and again that is correct according to this. However, Dell elegantly leaves out the SPECjAppServer2002 (TOPS) benchmark where Opteron is 17.1% faster than the Xeon:

Here.

SPECweb99 (Connections):

Dell claims that the Xeon is 1.2% faster than the Opteron, but Opteron is actually 4.1% faster than the Xeon according to this.

SPECweb99_SSL (Connections):

Dell claims that the Xeon is 0.4% faster than the Opteron, but Opteron is actually 6.4% faster than the Xeon according to this.

SAP R/3 (SD Users):

Dell claims that the Opteron is 4.3% faster than the Xeon, but Opteron is actually 13.9% faster than the Xeon according to this.

TPC-C (tpmC):

Dell claims here that there are no results available for the Opteron, but that is simply not true since the Opteron is actually 16.5% faster than the Xeon according to this.

MMB3 (Score):

Dell claims here too that there are no results available for the Opteron, but that is simply not true since the Opteron is actually 1.2% faster than the Xeon according to this.

To summarize: This is nothing but marketing deception, since very few of the numbers are actually correct and the rest are either false or directly misleading I think.

Best regards,

Name withheld by request

P.S. I've also included some "missing" benchmarks in my figure, and I got these data from here:

http://www.amd.com/us-en/Processors/ProductInformation/0,,30_118_8796_8800,00.html

In a followup email, the following was added:

I'd also like to add that Dell also left out SPECfp_rate2000 (Peak) where the Opteron is 47.6% faster than the Xeon, and SPECint_rate2000 (Peak) where Opteron is 11.2% faster than the Xeon. An updated plot is attached that also includes these benchmarks as well.

The updated plot is what was chopped up and posted above.

So, what do we have? To paraphrase modern political parlance, a massive credibility gap. Could it get any worse? Yeah, probably, especially when someone who I would put firmly in the category of 'he knows his stuff' writes me with the following tidbit. Why Dell left out the system specs will most likely be a lot clearer after this bit.

In your article on Dell's AMD vs Intel benchmarking material, you noted that the first page of benchmarks was oriented toward FP applications and that the Xeon system looked stronger. It is important to note that the numbers may be more misleading than they appear. For example, the Xeon SPECfp2000 lead of 9% applies only to the case where you use one cpu and leave the other idle. If you actually use both processors in the dual-cpu systems, you find that the picture is dramatically different -- the 2p SPECfp_rate2000 for Dell's 3.6 GHz Xeon system is 25.2, while 2p SPECfp_rate2000 scores for 2.4 GHz Opteron systems range from 32.9 to 42.2 (depending on the O/S and compiler). This shows an advantage to the Opteron system of +30% to +67%.

Of course no one who buys a 2p server would really be interested in using that second cpu, would they?

As I said, this is in Fairy Tale format, and that means, like all good fairy tales, it ends with a moral or two, eight in this case.

Moral 1: When you are trying to disseminate information using an NDA to keep questionable things quiet, and they get out, don't do anything to call attention to them.
Moral 2: Don't bend the truth.
Moral 3: Wicked stepmothers rarely win, and get eaten by bears in 37% of made up statistical research.
Moral 4: There is always someone smarter than you, and they tend to be several steps ahead also.
Moral 5: The people from Moral 4 are usually not Inquirer writers.
Moral 6: The people from 4 usually are Inquirer readers.
Moral 7: Don't feed the bears, unless it is with wicked stepmothers.
Moral 8: There are people out there with to much free time. This is a corollary to Moral 2.

The end, happily ever after, etc etc. µ