What's wrong with this pixel shader?

Benchmark firm Futuremark has uncovered extensive cheating by NVidia in its 3DMark03 suite.

After an initial report at ExtremeTech, Futuremark revisited the tests and discovered eight instances of cheating, which improved the performance of NVidia's Detonator FX and WHQL drivers by as much as 24.1 per cent.

But NVidia achieved this not through brilliant optimization, but by alternative means which omit graphic details: so the output, while at times similar, does not resemble what it should.

As Futuremark explains:

"The cheating described here is totally different from optimization. Optimizing the driver code to increase efficiency is a technique often used to enhance game performance and carries greater legitimacy, since the rendered image is exactly what the developer intended."

NVidia deployed a variety of cheats: introducing its own pixel shaders and vertex shaders specifically for the benchmark. The shaders improved performance at the expense of image quality.

FutureMark is revisiting some ATI benchmarks, too. ATI, which has grabbed the lead from NVidia in a ferociously competitive battle, was caught cheating before.

In 2001 ATI was discovered to sacrificed quality for frame rates in a Quake 3.0 benchmark: when Quake was renamed "Quack", the drivers produced much better output but frame rate performance fell.

NVidia withdrew from FutureMark's beta program earlier this year.

You can see FutureMark's NVIDIA audit here (PDF).