- Saturday, 11 July 2009
- INQ Mobile
- RSS
When you have to kill a man it costs nothing to be polite - Winston Churchill
Nvidia is just doing its job
AFTER THE DECONSTRUCTION of Nvida's latest graphics driver by Charlie on Monday, the wibble has been buzzing with claims and counter-claims and rants about the same subject - is Nvidia cheating in 3DMark?
The Green Team's new 177.39 drivers enable GPU physics in the Vantage benchmark. This means that any standard system running the physics test is running on the CPU - whereas any system with a compatible Nvidia GPU and driver is running the physics test on the GPU, resulting in a performance increase of roughly 10x.
Charlie made the arguments against this practice pretty elegantly - the new Nvidia drivers aren't even running the same codebase as competing platforms, and moving a calculation from one component to another is hardly a level playing field. Not to mention the fact that these new drivers aren't Futuremark approved.
But Nvidia has kicked back with some ranting of its own, over at HotHardware. In its defence, Nvidia's Roy Taylor said that "The physics test simply switches from the PPU to the GPU. The workload doesn’t change at all but where it is processed in the test machine is. This doesn't change the benchmark, just which processor in your machine handles it."
And is this an unfair advantage? "The reason we bought Aegia was to enable our GPUs to provide these sorts of effects." Good point.
Taylor's core argument is - the entire point of running physics on the GPU is to enable better performance, so why complain when we do so?
That leaves out the argument that this is not a Futuremark-sanctioned driver, so to publish results with it is misleading. Taylor seems plain in admitting that this is not a sanctioned driver, and most likely Nvidia will not be submitting it - and that, consequently, such results are merely demonstratory.
"In terms of 3DMark Vantage, if you’re interested in making a driver change and submitting that, Futuremark's BDP (Benchmark Development Plan) process has strict guidelines you must adhere to. We have followed them to a tee and this new beta driver has not been submitted for consideration."
So there you have it - yes, the driver fundamentally changes the way Vantage processes physics, yes, that upends the playing field and no, consequently, it doesn't look like Nvidia will be officially rolling this out as its 3DMark driver.
But as far as a technology demonstration goes, it's hard to argue against the fact that processing physics on the GPU is clearly much more beneficial - from a performance point of view - than on the CPU. And in the real world, away from benchmarks and their controversies, that has to be a good thing. µ
Share this:
Business Intelligence Software
Customer Relationship Management (CRM)
Enterprise Accounting Software
Enterprise Resource Planning (ERP)
Enterprise Systems Management
Sounds to me like Charlie is either a AMD fanboy or just wants to whine. I already looked at the game charts of Nvidia's new 200 line of cards at Tom's Hardware and yup, they rock. The only bad thing is the price, Newwegg does not even have the 260 for sale yet, only the high dollar 280 but accourding to Tom's the 260 will come in at about the price and it beats about anything out there including Nvidia's last batch of GTX2 cards in some places if memory serves. Will never do more than one card myself.
Destruction? Must have skipped that bit of the article... 

Vantage fails pretty miserably to represent real world gaming, but even so, if gpu physics is the way forward, why should they ignore it? Sooner or later they'll have to cave in, or they'll lose even more face if they only sanction crippled drivers.

Only reason I can see for it is because ATI can't do it. Boo bloody hoo.
T WHOLE point of GPGPU stuff is to move things like this to the GPU! 
Why have CPU do it when GPU can do a better job?? it's certainly not 'cheating' and it's not as if Vantage will be the only thing to make use of it.
doesn't using gpu for physics simply mean trading some gfx computing power off. In a real gaming scenario (that is what 3dmark is for) both gfx and physics are calculated all the time, if gpu wastes time on physics this results in slower frame rendering. If the 3dmark results are to be fair with gpu physics enabled they should cap the gfx tests (trading physicMarks for 3dMarks, how to do it fairly is another good question though, the thing is when 3dMark runs physics on new NV hw, it utilizes 100% of gpu for it, IRL this would mean 0 3dMarks for gfx as the whole gpu is used up already). On the other hand these are just partial tests, not realworld tests, that's their beauty, you can tweak your machine for some special task - but people just don't get it :). This is different with overall score which counts the same actual gpu power twice.

Maybe worth mentioning in an article...
Sure it's a good thing if the GPU can handle physics 10x faster then the CPU but that almost everyone seams to be forgetting is that games are heavy on physics usually are heavy on graphics too. This means that the GPU will not be able to handle both at the same time with acceptable performance.

The CPU-test in Vantage is light on graphics so that it would not be bottlenecked by a weak GPU. Because of this a powerful GPU would have lots of time to do physics and inflate the CPU-score.

This situation is unlikely to occur in an actual game. That's why the final score is misleading.
ok, so it increases 3dmark scores, maybe bending some rules...

I DON'T CARE

what I care is: will it increase performance?

and here's the twist: I suspect it won't

3dmark's physics test is a CPU test, not a GPU test, so physics can be run on the GPU... BECAUSE IT IS MOSTLY IDLE!!

if nvidia cards can increase your fps by taking care of the physics only when they are not busy rendering the images, then this test is useless, and it just comes to prove that 3DMARK IS USELESS
so, if i get unreal tournament 3 and install it, would that be considered cheating on the framerate?

who cares what 3dmark considers cheating and what it doesn't? if it gets a higher score, it gets a higher score: frames per second are still frames per second. it's because it has the ageia engine that the Vantage developers decided to use - i'm sure at the time they were applying their APIs to their program, they were told "hey, nvidia's gonna purchase us..." they KNEW they were going to be bought-out.

so, is it wrong that nvidia isn't being straight-forward? probably. that's what these articles should be focusing on. rather than "OMG THEY ARE CHEATING BASTARDS." sure they are, but cheating still unfortunately yields results. this reminds me of the bill clinton years. he got a blow job in the oval office and nvidia gets a higher vantage score because of a smart buy. who cares? it's politics.
Number 9 ... got caught with their pants down with "optimised" drivers before being bought out.

NVidia have cheated on image quality in order to produce faster benchmarks.

NVidia have cheated with unfair comparisons to ATI products in the past.

7900GT ... crap chips that lockup

8800GT single slot coolers that can't cope and lockup systems on hot days.

Geforce 5 series ... RIP

I can't see how they can make a buck out of the new 260's (these are crap compared to the ATI 4850 and 4870) and overproced.

The 280's won't see any volume sales because of their low yields and cost.

Watching their stock market price srop as we speak.

Sell sell sell ....

Karma I say/
So ... ATI could do the same huh?
Does NVidia document all the hooks to let other GPU vendros implement this in their drivers as well ?
NVidia has done a good job here. Such a good job that it 3D Mark cannot cope.

Future bench marking software will need to accept the Physics can be rendered anyware, CPU, PPU, GPU and state how good the performance is for that unique machine.

Perhaps a Graphics and Physics test at the same time will be an appropriate bench mark to include for the sake of analysing a given computer.
...where the physics is done: if Nvidia driver does the physics on the GPU - AND that that is the publicly available driver - then all Nvidia has done is provide faster physics. Nothing wrong with that.
The Futuremark scenes for physics are misleading because they measure fps when graphics are effectively disabled. If you like to play games that look like these scenes do (sw rendered image quality), then you may be OK with GPU physics. I won't.
Who cares if Charlie likes or hates Nvidia?

Does he make a valid point?

Is it fair to score (and get to compare that score) on 3D mark in the manner in which Nvidia makes use of their hardware?

That's all I care about...I don't want to hear excuses or who likes whom more...

Many years have gone by & nothing has changed. Synthetic benchmarking is as crappy as ever.

Read ---> http://www.hardocp.com/article.html?art=NDkx

Hardocp & others told you this sort of stuff YEARS AGO.

Real life gameplay & dynamic application use are the only way we can really test things fairly...else everyone will optimize a driver for your syntehtics.
Funny I run my 8800GT single slot cooler hard on hot days and have not had one lock-up. Must have other problems. Even run it oc'd without a hiccup.
I just got a GTX 280 the other day, on tuesday.

It. Is. Freaking. Amazing.

Over my old 8800GT 512MB, *ALL* of my games saw about a ~200% increase in FPS.

And most of my games weren't even truely MAXXXED with the 8800GT (AA, AF, shadows, water usually turned DOWN). They are with the GTX 280.
Daamit is already working on hardware acceleration of the Havok engine. Sure, they won't gain anything in Vantage but Futuremark went with Physx, not havok. IRL there are tons of games using Havok where ATI users will gain performance.

I noticed a couple of fellow commenters claiming "It's a CPU test, not a GPU test". Uhm, no, it's a physics test, especially seeing as it allows people who bought Ageia PhysX cards to score higher than people without those cards. Whether it's running on CPU's, PPU's or GPU's doesn't really matter (except for the score) since it's all using the PhysX libraries. nVidia just created a framework to stream physx data through the Geforce SPU's instead of using the software running on the CPU. 

Somehow TheInq seem to think 3dmarks are the most important thing in the world, making such a fuss about it.
According to FuturreMark, because drivers in question are Beta, NO evaluation has been done. However, after looking over situation, FutureMark is Confident that Test Scores Reflect Actual performance & NO Error is being made.

PhysX migrated from PCI slot to Game Card it self. So it is much more powerful sitting On Top of that Powerful Engine, GPU.

Its Design Plus for Nvidia & affects older 8XXX+ game cards with software Push.

PhysX is Screaming Torch, while Havox just fits bill.As article States:Nvidia is just making fastest Card it can Design.
drashek
This article below from PC Perspective has an interview with ATI/AMD/Epic/FUTUREMARK/Nvidia and all of them stated for a fact that this what not cheating. the PhysX test is exactly what it is and if a company brings new inovation to push the boundaries of PPU then it's not cheating. Charlie should read this article cause unfortunatly ATI/AMD camp is not on his side :)

http://www.pcper.com/article.php?aid=584
This is rather comical from Nvidia, the last time they resorted to cheating in 3DMark was when ATI had much better hardware and they had to cheat to try and stay ahead.

Some strong arm tactics and wads of notes waved around at 3Dmark last time to claim that it was all a mistake...

What is really interesting again here is now they are cheating again, and guess what ATI have the better hardware again.

All this does is vindicates ATI superiority again. And finally who really cares about 3DMark, history has shown they can be bought so who gives a toss.

This proves once again the green machine is worried and they have to turn on the cheatomatic fud machine to try and score points.

Those of us in the know just sit back and smile....
CPU:
Vantage - near 100% physics
Gaming - AI + physics + otherstuff = 100%

GPU
Vantage - near 100% physics
Gaming - shaders + physics + otherstuff = 100%

PPU
Vantage - 100% physics
Gaming - 100% physics + some crappy stuttering

So unless you want to run an extra GPU doing only physics or your game can run threads on an idle core that it wouldnt otherwise use there are tradeoffs. All methods are merely producing peak perormance.

The only one screwed out of this (ati hacked physx drivers apparently out) is futuremark. Since it makes their benchmark into a joke. And they will need to either do a test where they load physics and graphics at the same time then invalidate results which show the pure physics load isnt possible in the real world or include a hell of a lot more categories.

One possible workaround is if NVIDIA sets up physx to use x% of shaders for physics and x% of shaders for graphics for an entire run of Vantage (could involve SLI with 2 cards doing gfx and one physx). Would be interisting to run tests with a sliding scale of shaders from 0% to near 100% for physx.
Vantage will useless if this continues. Why, because they add all scores to come up with the 3Dmark score. And as others mentioned, it's like scoring the same GPU twice (for Graphics and Physics).

We don't need to scrap running Physics on GPU, or on anything. This great! Gamers don't need to invest for the fastest processor as we can off-load some tasks to the GPU. 

What's needed is a separate score again like old 3D marks so it won't be misleading to everyone. Either Patching or completely re-doing Vantage to correct this??? Now nVidia really stirred something up again here. Poor Vantage team.
The problem isn't with Nvidia using the gpu to process. The problem is since Nvidia bought Aegia and futuremark is using Aegia physics there is no way for ATI / AMD or Intel to compete. If other graphics companies could just turn on Aegia physics anytime they wanted to then the test is still fair. But by cornering the market and not allowing others to use the Aegia drivers you have created an unfair advantage and in doing so have made the result invalid because for all we know if ATI could turn it on it there drivers they might kick Nvidia's ass in physics but we will never know for sure because we aren't comparing apples to apples.
I still don't understand why you guys seem to be making this unsubstantiated claim that somehow the Nvidia drivers are running different physics code. Is 3DMark actually using a different path when hardware acceleration is enabled? It would seem that Nvidia would actually be implementing this API, therefore running the exact same codebase.

That is unless you want to make the claim that AMD and Nvidia are running different codebases on every D3D game out there as well because the codebase are their respective drivers which implement the same API and run the exact same application code.

Charlie was anything but elegantly and you're only helping further perpetuate the same lies.
Anyone else notice the most important statement by Futuremark in that Hot Hardware interview?

- "Outside of this matter, we have been introduced to this technology from NVIDIA and it is truly innovative for future games and game development. As you know, we have an actual game coming as well and it could also make use of PhysX on the GPU.” -

Wow, Futuremark is making a game with PhysX in it. What are the chances it will be a TWIMTBP game, and that they get that PhysX license for free along with a bunch of other payoffs -oops- I mean support? :-)

Of course they're not cheating it's just the Way It Meant to Be Paid. 

Once again, 3Dmarks should go the way of the Windows Experience Index, straight into the trash.

As others have mentioned if the GPU is idle while doing the physics that's never going to happen in a game because what are you not drawing that it's spending all the time calculating?
As long as the subject is on nVidia and cheating, does anyone have any idea why the following apparent "paradox" occurred between the screenshots in the following post?

http://www.rage3d.com/board/showpost.php?p=1335487633&postcount=24

I realize that the 175.80 Forceware drivers are leaked and that bugs are to be expected, potentially. My first observation was that to get strange colors in the DMC4 demo as shown in the actual screen output of the first picture might result from one part of the rendering pipeline using the wrong pixel format, for instance A8R8G8B8 instead of the A2R10R10B10 it was supposed to use. Another source for the bug could also be a problem with bit or byte endianness somewhere. Since users of this driver in the thread report other, less major problems in DX10 rendering, my guess is the driver had new, but buggy microcode for the graphics chip that caused these "issues" to occur. I am not an expert on this subject much though. Incidentally, many of the other posts in this thread indicated they were going to use a different Forceware driver.

What concerns me the most, however, is the fact that the actual screenshot captured by Windows, when using the same driver, does not match the actual screen output with bugs, as it should. This is not good. Regardless of whether during the capture of a screenshot, the output from the GPU is checked, corrected, and improved, or a special GPU or CPU rendered frame is produced for the recorded screenshot, the screenshot becomes useless for image quality comparisons against other screenshots from other graphics cards. (This assumes, of course, that the drivers for the other graphics cards are not "enhancing" their quality either.)

There are also some anecdotal reports of image quality issues from a 8800GT after the 8800GT replaced an HD3800 series card. However, no actual screenshots were produced.

Assuming the actual screen output is different from a captured screenshot, image quality could be compared if for both cards, an external DVI pass-through frame grabber card, which exist, and if the card worked transparently as far as the the output computer knew, which may be the case, then image quality would potentially be comparable again.
What I find problematic is how the Benchmark is designed : The physics Test is one of the "CPU" tests. As such it has NO place on the GPU. But it uses the PPU if available, which is NOT consistent with the "CPU test" label.

Since the "GPU as a Physics processor" capability is tested in the Feature test 4 and 5, I don't think it's fair to also use it in the Physics "CPU" benchmark.

What misses here is something that stresses globally all the system : Heavy Graphics AND heavy AI AND heavy Physics, AND heavy network AND the kitchen sink...
That way, if some cheating GPU/CPU/Network maker offloads part of the work somewhere else, it will have an averall impact on the target processor's results ...
CPU or GPU bottleneck? If you have a physics heavy game that is CPU bottlenecked, I can see how this would help. Not to mention, many of you seem to think all of your video card is working all of the time. I don't know if physics is harder on the parts of the card that handles shaders... texture etc. but different things stress different parts. Physics may just be another thing to keep the 512 wide bus fed.

Bottom line is.. what's the effect on what we use.... games. (today and tomorrow)