That fish will soon be caught that nibbles at every bait
|
|
|
Bapco claims Sysmark 2012 uses applications that have market share
BENCHMARKING CONSORTIUM Bapco has defended itself against allegations made by AMD claiming that Sysmark 2012 is not representative of real world performance by saying that it "selects applications based on market share".
Chip designer AMD announced that it has left Bapco and will not endorse Bapco's Sysmark 2012 benchmark. AMD's head of communications, Jan Guetter told The INQUIRER, "Sysmark [2012] is not representative of real world performance." Guetter also claimed that parts of the Sysmark 2012 benchmark test suite do not take into account the built-in GPU in the firm's latest line of Llano accelerated processing unit (APU) chips.
Bapco fired back at AMD's claims, telling The INQUIRER, "It is not the goal of the benchmark to highlight any specific technology. Sysmark reflects overall system usage by measuring application response time. Bapco selects applications based on market share. As more applications support newer technologies, then by default, those applications will become part of the benchmark."
Bapco does have a fair point about choosing applications based upon market share. After all, that is perhaps the only reliable indicator of what percentage of the market uses an application. However it is also easy to see AMD's frustration that stems from being the first chip designer to put a DirectX 11 GPU onto a CPU die, something that might only be exploited by the popular software applications in a year or so.
One area of the Sysmark 2012 benchmark that both Guetter and Nigel Dessau, CMO at AMD, marked for criticism was optical character recognition (OCR), a task that Dessau said "an average user will rarely if ever do". Bapco defended its decision to have OCR in Sysmark by saying, "Sysmark has a broad representation of different usage scenarios. OCR is featured in one of the six scenarios, and it is relevant in the usage model of that particular workload. We look forward to publishing further information in the whitepaper shortly."
Share this:
Share
When can I record a week's worth of work and use it to create the benchmark ?
AMD, Nvidia, Via et. al. may band together to create yet another platform benchmark suite that has more to do with marketing than reality and just as irrelevant at Sysmark. What is "average" and "normal" is really a lot different from the range of uses and tasks we do today. Eg. Excel for data sort and science/business analytics is a joke, you would be better with Stata or SAS BI (R is much slower).
AMD et. al. should throw more money into collaborations with 3rd party SW developers to optimise HW for apps and optimise apps for HW.
Intel and BAPCO want you to buy a computer based on something that you can not sense at all with any of your senses.
AMD is saying and quite rightly, make your purchase based on how well interface with your computer.
I use a computer at work for 10 hours a day and usually at home for another 1-3. This has been since 1991. I have only used OCR ONCE!
That was to scan a foreign language chess magazine article into a word document so that I could dump that text into Babel Fish to get a rough translation. That was in 1999.
OCR worked poorly as it only recognised about 90% of the text.
And I really didn't care how fast it worked. I bought the scanner just for that scan because until then I didn't need a scanner.
I wasn't about to run out and buy a new computer that had the highest OCR benchmark either.
If we are gauging todays cpu performance with something as mundane and almost stupidly irrelevant as OCR performance then I would have to agree that BABCO is sadly misguided and also irrelevant.
I want my computer to work well and quickly. The idiotic fact that I should run out and buy a computer based on the observation that my cpu can solve some obscure math problem to the millionth decimal point 6 seconds faster or slower is again sublimely absurd. What I do want though is smooth, crisp graphics!
I want to see a great graphical representation. And good sound but that is not hard to get.
THAT is what AMD is driving at!
Graphics is wholly subjective and quite obvious. BAPCO Symark while it may be objective the results are far from obvious. In fact the results are really there only so that Intel can say that it's fastest cpu beat AMD's fastest cpu by a few nanoseconds.
MOST CONSUMERS DO NOT BUY THE FASTEST CPU AVAILABLE ANYWAY!!!
They buy what looks good! Works for them is on sale, in stock or what the salesman told them to buy.
So what we really need is some blind taste testing like COKE vs PEPSI!
And dollar for dollar AMD will come out the clear winner.
Radeon graphics can't be beat.
Intel is also getting high scores by selling a 3.0 gig cpu as a 2.6 gig cpu.
As soon as you run code through it it speeds up. So the benchmark is skewed from the start by a marketing ploy.
Bapco = Propraganda
If the scientific method you used to reach a conclusion is not available, then the conclusion itself is worth nothing.
If on top you claim those conclusions reflect reality, then you are engaged in propaganda and disinformation.
Bapco is in the business of selling their soul to the highest bider and they are not alone. Most benchmark are in a way or another. We could also include hardware review website, PC magazines, and even some technology related blog to the list.
Even without direct proof, it's more than probable that some forums and comment section on hardware review are hijacked by paid people to post the ''right' comment.
The bottom line is, there is not much truth out there when it comes to the computer business as a whole. No black and white, just an infinite shades of gray. Sad, but true.
Ramon
I don't blame AMD at all. BAPCo has been using bias benches for years to favor Intel. It's time people knew they were being duped. 80% approval doesn't mean that AMD agrees with BAPCo deceiving consumers with stacked benches.
But how it arrived at the usage scenarios and the weightings it gives to each sub result are shrouded in mystery.
In an earlier incarnation of Sysmark it was discovered that the Excel benchmark was altered to give a 90% weighting to the sorting results. This was when the Pentium 4 was introduced, which surprise surprise, was good at sorting.
If Bapco are not prepared to publish their methodology and score weightings how can we have any confidence that what they are testing matters to real world users?