Yesterday we reported on Fallout 4 console performance, finding that both PS4 and Xbox One target 1080p30 gameplay - with somewhat variable results. Both versions have issues maintaining their frame-rate targets, but it's the Xbox One version that has a tangible disadvantage - there's a 'macro-stutter' issue that seems to be related to the game's background streaming technology. Entering new areas can see second-long pauses, and even switching weapons sometimes causes problems.
The question is, what can be causing it? We know from performance profiling of the PC version that CPU load can increase substantially as new data is streamed from the hard drive - but this would not explain 1000ms frame-times. Similarly, a GPU issue is even more unlikely: after all, there's very little difference visually between the frames before and after the stall, where the tasks run by the GPU would be very similar from one frame to the next. By a process of elimination, the most likely culprit is therefore storage - and that's curious, bearing in mind that both PS4 and Xbox One both feature 500GB 5400rpm hard drives, and the Sony console has no similar problem.
After extensive testing, it does seem that the issue relates to a storage bottleneck on the Microsoft platform, but until this is addressed via a title update, there is a workaround - albeit a somewhat impractical one. Completely eliminating the issue - at least on the test areas used in our initial performance analysis - is possible, but it requires mounting an SSD into an external USB enclosure, then copying the game data onto that drive. This is not completely ideal for a number of reasons: SSDs aren't cheap, and the USB caddy required only adds to the expense. So we thought we'd try another couple of solutions - a 7200rpm hard drive and a hybrid SSHD. The latter consists of a 5400rpm drive paired with a smaller SSD cache. The results on both drives are intriguing.
Performance improves on both units, to the point where the 0fps drops we encountered are gone, but the stutter - though reduced - is still there, whether it's through standard traversal through the environments, or in accessing weapons you've not used for a while. Across our test clips, the 7200rpm drive seems to offer a small advantage generally over the SSHD, but there are hitches and stutters that seem to show more improvement on the hybrid drive. Both drives show a substantial uplift over the internal unit in that the 0fps macro-stutter is gone or massively reduced, but it's clear that only the SSD showed a complete fix in effect. And again, we should stress that owing to the sheer size of the game, our testing was limited to areas where we had existing data.
Although it doesn't appear to have anything like the same problem, we also tested the PlayStation 4 version of the game running from solid state storage. There's a very small ironing out of performance, but nothing to write home about and that makes sense as storage bottlenecks on the stock drive appear minimal. Of course, loading times are improved with the move to faster drives and that's a situation common to both consoles.
Here's a quick look at our data, showing that PS4 also has a loading time advantage over Xbox One in a straight stock drive comparison. Switching to an SSD lops 30 to 50 per cent off the time taken to get back into gameplay. Loading times aren't really intrusive to the Fallout 4 experience, but it is safe to say that when they do appear, the momentum of play is compromised - and the faster we're back in the game, the better. What's also curious here is that running the Xbox One game from a solid state drive actually provides faster loading times than doing the same from PS4, and this may well be a matter of contention: the Microsoft console still has its internal drive to use for OS tasks, while the Sony hardware is running everything from the internally mounted SSD.
In conclusion, we suspect that we'll have to wait for an upcoming patch to resolve the stutter issue in the Xbox One version of Fallout 4 - but from a technological standpoint, we don't see any reason why it can't be done: PS4 and Xbox One have very similar storage solutions and the Sony platform is unaffected. Our only concern here is that Xbox One loading times from the stock internal drive are consistently longer than they are on PS4's, suggesting that in the case of Fallout 4, there is an inherent PS4 advantage here that may extend to the background streaming tech. This doesn't explain the second-long pauses in gameplay we often encounter on Xbox One though - the ball is very much in Bethesda's court to address this, along with the other stability reports coming in post-launch.
|Loading Times (Seconds)||Xbox One Stock Drive||Xbox One 7200rpm HDD||Xbox One SSD||PS4 Stock Drive||PS4 SSD|
|Exiting Vault 111||28.6||23.2||16.0||22.8||11.8|
|Entering Museum of Freedom||19.2||15.0||7.5||16.0||10.5|
|Entering Dugout Inn||13.6||10.9||5.8||12.2||8.8|
Will you support the Digital Foundry team?
Digital Foundry specialises in technical analysis of gaming hardware and software, using state-of-the-art capture systems and bespoke software to show you how well games and hardware run, visualising precisely what they're capable of. In order to show you what 4K gaming actually looks like we needed to build our own platform to supply high quality 4K video for offline viewing. So we did.
Our videos are multi-gigabyte files and we've chosen a high quality provider to ensure fast downloads. However, that bandwidth isn't free and so we charge a small monthly subscription fee of €5. We think it's a small price to pay for unlimited access to top-tier quality encodes of our content. Thank you.Support Digital Foundry