Diablo 3 on console: when is 60fps not really 60fps?

UPDATE 14/8/14 6:28pm: Yesterday, Blizzard contacted Eurogamer to say that the dual frame-rate issue we uncovered in Diablo 3 was a bug it was looking into patching as soon as possible, and offered to talk us through it one-on-one at Gamescom. Thomas Morgan met up with production director John Hight in Cologne earlier today for a quick chat.

"First off, I want to thank you for going through it, digging through it. As it turns out we do have an interesting bug in the game," Hight explains. "Essentially what's going on is we're updating at 60 frames per second, except on the consoles it's 59.94. Effectively it's like a round-off here - we're updating 59 frames on the first second, and 61 frames on the next [meaning] we're literally double-writing a frame."

So it appears as though elements of the renderer operate at a pure 60fps, while the HDMI standard actually incorporates the legacy NTSC drop-frame standard, running at 59.94fps. The discrepancy is apparent on both versions, but according to Hight differences in the way they handle the rendering account for the more pronounced hit to PS4 performance.

"After we we read your article we went back through, looked at the code, and determined that we made mistake on, essentially, updates. So we're now at full 60 frames. We're now posting an update that will go up before the day one patch, and will be part of the day one patch," Hight continues.

"As soon as we saw your article I sent it over to our core engineering lead, and I'm saying 'hey what's this this about?' Those guys are awesome - they jumped right on it, looked through the code, and were like 'I'll be darned we actually have a problem here with the way we're updating'... So our goal is to have that out day one, so on the 19th. We actually put the Sony [PS4 patch] into format today, and Sony's going to give us a pretty quick turnaround on it. I don't think we're going to have a problem getting it day one."

Original Story: When is 60fps not truly 60fps? That's the question we asked ourselves as we played through the PS4 and Xbox One renditions of Diablo 3 - a game that targets that tip-top number but with bizarre consequences owing to the unusual rendering set-up of Blizzard's in-house engine.

To cut to the chase, the PS4 version isn't as smooth in action as its Xbox One counterpart, despite both versions sharing identical assets to PC on its highest settings. Having tested both extensively over the last few days, with a mind to produce a full Face-Off feature once the Gamescom mania is dispensed with, the promise of 60fps is, on the face of it, delivered on both of the new-wave consoles. We ran both versions of the game through our analysis tools, which reported a flawless frame-rate on both - but there's a catch.

Unique to Diablo 3 is its method of producing frames, where character animations, alpha effects and its lighting model are rendered independently of camera movement. As you can see in our video below, actual gameplay can stutter down to near the 40fps line on PS4, while the camera continues to pan across the environment at a clean 60fps. It's difficult to ascertain exactly why the game presents itself like this, but it may well be the case that Blizzard has adopted a rendering strategy that ensures a 60fps refresh regardless, while some important elements of the scene to be updated at varying intervals. If so, it's an interesting technique that doesn't quite pan out.

Our test is running the full, disc-based UK versions of each game, on the most up-to-date patch available at the time of writing. Specifically, the PS4 runs at full 1080p as promised, with patch version C2.0.0.24791 instated on the menu screen, while the Xbox One renders at a lesser 900p resolution with the C2.0.0.24514 build in effect. Of course, we're promised a day-one update to 1080p on Microsoft's platform, with Blizzard seeing an opportunity in the June SDK to optimise further.

Based on our video analysis, each game renders out a perfect 60 frames per second - but in analysing the all-crucial gameplay, we see frame-rates on both platforms running with different practical results. Based on our initial playthrough, the worst of it occurs at the very start on PS4; around the gates of the New Tristram hub area. Characters and projectile arrows can run at essentially half their frame-rates as skeletons rise from the perimeter. Despite repeated tests, the Xbox One version only produces occasional hiccups in matching scenes.

Diablo 3: PS4 vs Xbox One Frame-Rate Test - First Look

So what's actually running at 60fps? From close study of each frame, camera motion, HUD elements such as the glowing Fury orb to the left, and shader distortion effects each run a full refresh. In terms of playability, the PS4 does not quite control in the same way as a result of stutters elsewhere - with our Barbarian character lagging behind occasionally during both combat and traversal.

In essence then, while Diablo 3 can in some part claim to be operating at a perfect 60Hz refresh, the core hack-and-slash mechanics don't quite hit the same lofty target. It's unfortunate, and affects the heart of the combat at times where inconsistency in controller feedback can be jarring. Equally confusing is the fact it impacts the PS4 version primarily, while the Xbox One - which clearly utilises the same technique - only suffers its wrath to a light degree.

So what could be the cause? It's unusual to see an engine rendering different elements of a game at different rates, but we've seen tricks in this vein before to save on performance. Recent entries in the Forza Motorsport series, for example, halve the frame-rate of certain reflections to maintain the 60fps of its core handling. Elsewhere, we've seen Dark Souls 2 strip away frames from enemy animations in the distance - noticed in expansive areas such as No Man's Wharf - where drawing closer brings back their full rate of motion. Trials Fusion's shadows also render at varying levels of refresh (either quarter or halved) based on distance.

But this is different - the opening battle at the New Tristram gates is an extreme case, but we also see frames being cut on Sony's platform where the Xbox One is smoother throughout - and often without any obvious correlation to the on-screen action. Wandering uncontested through the more simplistic Fields of Misery area, for example, can cause occasional dropped frames, while hectic maelstroms of effects around Act 3's Battlefields run as smooth as butter.

Evidently, this isn't just down to the PS4's GPU struggling to render multiple enemies on-screen or complex scenes. Even with huge bursts of alpha effects and dozens of enemies flying from all angles, the console regularly proves it can hold up at 60fps where it counts - even in co-op multiplayer. This suggests a GPU bottleneck from running at 1080p is, in all likelihood, unlikely to be the cause. As a possibility, these dips may come down to Diablo 3's managing of background processes; CPU-side calculations, or asset streaming beyond the player's field of view.

Launch comparison tool

Based on its PC incarnation, this theory has merit. The engine has shown a limitation in support for multi-threaded CPUs, which has become a core facet of new-wave console design. It's conceivable that parts of the engine are hitting a bottleneck here, where the boost to the Xbox One's CPU clock speeds (up 1.75GHz per core, as opposed to the 1.6GHz on PS4) could possibly make an impact. But in truth, this is just speculation for now.

Despite the dual frame-rate issue, Diablo 3 on the new consoles fundamentally remains an excellent title, forging the top-end PC graphical standard with the four player drop-in, drop-out living room co-op that made the last-gen releases such a pleasure to play. This release also features the Reaper of Souls expansion's additional fifth act, the Crusader class, and adventure mode, which is an extra expense for PC owners.

The technological glitches are unusual though - and something we'll explore in more testing scenarios in our upcoming Face-Off. We hope the PS4 side sees some attention so the refresh on all counts matches its Xbox One sibling. Both platforms share effects quality ramped up to the same level, with each a match for the PC's highest smoothed shadow setting. The only other discrepancy on paper is the 900p and 1080p resolution divide, but in practice with a game of Diablo 3's gloomy aesthetic, the PS4's resolution advantage doesn't quite shine through as brightly as we'd hoped, where high contrast edges are uncommon. Each version is paired with heavy post-process anti-aliasing too, making the wide chasm in pixel-rate trickier still to catch by eye.

And yet, once the next patch hits, we're soon to see the Xbox One come up to par with a 1080p frame-buffer. But with its current performance advantage in actual gameplay, it's unclear whether such an update will in fact do more harm than good - or if future patches stand to resolve the PS4's issues. We'll return to the game with a full Face-Off - including co-op stress tests on some really demanding levels - once the launch day patches are in full effect.