CHIP DESIGNER AMD has left the benchmarking consortium Bapco, claiming that its Sysmark 2012 suite does not represent real world usage.
Nigel Dessau, CMO at AMD announced that AMD has pulled out of Bapco and will not be supporting Sysmark 2012, claiming, "BAPCo chose to ignore the opportunity to promote openness and transparency." Dessau said that the score delivered by Sysmark 2012 is generated by seven applications, incorporating tasks such as optical character recognition, which "an average user will rarely if ever do".
Dessau's announcement follows similar decisions by Nvidia and VIA to part ways with Bapco, leaving the benchmarking outfit with reduced credibility. Jan Guetter, head of communications at AMD told The INQUIRER, "Sysmark 2012 was the tipping point," and that AMD's decision to leave Bapco "was not an easy one and was the final step".
Guetter admitted that the decision to leave Bapco was fuelled by AMD's A-Series Llano accelerated processing unit (APU). Guetter said, "Sysmark  ignores APU acceleration," and reiterated Dessau's views by saying, "Sysmark 2012 does not reflect real user scenarios at all," adding, "Sysmark  is not representative of real world performance."
If you thought that was hard, then Guetter continued the demolition job on Bapco by saying that firms "would be ill-advised to use Sysmark 2012" for buying decisions, as "it is not functioning". Ouch.
Guetter suggested that users look at multiple benchmarks to get an accurate representation of hardware performance, something that any hardware review website would recommend. Guetter also told us, "AMD is planning on creating an open benchmarking consortium," and said that he would "not be surprised if others leave Bapco".
Many companies still use a single benchmark such as Sysmark to evaluate individual components that end up in systems. Asked whether AMD's decision to pull out of Bapco will lead to fewer design wins, Guetter told The INQUIRER that he doesn't see this as a problem.
Guetter's view that leaving Bapco won't affect AMD's chances of winning deals is perhaps accurate, after all if the benchmark doesn't show APUs in a particularly good light, then being part of Bapco is hardly worth the effort.
For AMD, it's decision to bung a fully featured DirectX 11 GPU into the same die as a quad-core processor means that current benchmarks that aim to look at overall system performance need to take into account desktop applications that make use of GPUs, such as Microsoft's Internet Explorer 9 and Adobe's Photoshop. Although Sysmark 2012 might not accurately reflect an APU's performance, Guetter wasn't rubbishing other synthetic benchmarks such as Futuremark's 3DMark, reiterating his earlier point that users need to run multiple benchmarks.
Bapco having lost AMD, Nvidia and VIA all in a matter of days casts significant doubt on whether Bapco and Sysmark will have much future, unless Bapco mends its ways. µ
Slack, hack and crack
A flaw in the protocol affects iOS, macOS and Windows 10
Wig wearer has issue with non-wig-wearer