BENCHMARKING CONSORTIUM Bapco has defended itself against allegations made by AMD claiming that Sysmark 2012 is not representative of real world performance by saying that it "selects applications based on market share".
Chip designer AMD announced that it has left Bapco and will not endorse Bapco's Sysmark 2012 benchmark. AMD's head of communications, Jan Guetter told The INQUIRER, "Sysmark  is not representative of real world performance." Guetter also claimed that parts of the Sysmark 2012 benchmark test suite do not take into account the built-in GPU in the firm's latest line of Llano accelerated processing unit (APU) chips.
Bapco fired back at AMD's claims, telling The INQUIRER, "It is not the goal of the benchmark to highlight any specific technology. Sysmark reflects overall system usage by measuring application response time. Bapco selects applications based on market share. As more applications support newer technologies, then by default, those applications will become part of the benchmark."
Bapco does have a fair point about choosing applications based upon market share. After all, that is perhaps the only reliable indicator of what percentage of the market uses an application. However it is also easy to see AMD's frustration that stems from being the first chip designer to put a DirectX 11 GPU onto a CPU die, something that might only be exploited by the popular software applications in a year or so.
One area of the Sysmark 2012 benchmark that both Guetter and Nigel Dessau, CMO at AMD, marked for criticism was optical character recognition (OCR), a task that Dessau said "an average user will rarely if ever do". Bapco defended its decision to have OCR in Sysmark by saying, "Sysmark has a broad representation of different usage scenarios. OCR is featured in one of the six scenarios, and it is relevant in the usage model of that particular workload. We look forward to publishing further information in the whitepaper shortly."