Tech Analysis: How The Last of Us 2 pushes realism to the next level
Naughty Dog's state-of-the-art animation system under the microscope.
To what extent can further technological innovation be delivered on the current generation of console hardware? Sony stood up to be counted at E3 2018 with a series of superb gameplay reveals - and taking centre-stage was Naughty Dog's The Last of Us Part 2. On the surface, it looks as great as you might expect. From its detailed characters to its lush environments, it's clear that the game is shaping up well - but if you look closer, there are some remarkable new technologies on display. From our perspective, animation and interaction are two key areas set to separate this game from its rivals.
What's immediately apparent is that there's a fluidity to the animation quite unlike anything on the market, to the point where some don't even believe that the demo is authentic in-game action. Of course, Naughty Dog has a rich history in pushing back the boundaries in this respect, with the expertly crafted blending and transitions seen in both the Uncharted titles and The Last of Us, but clearly, this latest E3 reveal is on a whole new level - and it's all down to a new animation technology known as motion matching.
"It's this crazy science fiction stuff where you take just hundreds and hundreds of animations of like walking forward and turning or whatever, and you put them in this huge bucket, and then based on what the player is trying to do or what an NPC is trying to do, it pulls from that bucket, sometimes two or three different animations, and blends them together to make this totally seamless thing," co-game director Anthony Newman told IGN.
"The motion matching technique is used by other studios, but we've kind of taken it and put the Naughty Dog spin on it because responsiveness is always a huge thing for us," co-game director Kurt Margenau added. "So we've taken it and kind of built on it and made this hybrid thing of the responsiveness and quickness of something like an Uncharted game, which is pre-existing and incorporating this very fluid, very realistic animation that still communicates those real stakes."
So what makes motion matching special? In traditional motion capture, the actors are suited up and perform any number of necessary actions for the game. Animators take this data, cut it up into chunks and then massage it to produce usable animation chunks that are implemented into the game. That's a somewhat basic summary of a highly involved procedure but the bottom line is that it requires a lot of work to achieve lifelike animation and aspects like believable transitions can require a lot of bespoke animation work.
The idea behind motion matching is to simplify this process while producing superior results - the basic gist involves capturing lots of data from a motion rig and feeding that data into a database where the game can look up and pull animations based on context related to what the player is attempting to do. This data is constantly scanned for appropriate animation, while blending between the two occurs in real-time. It's a more systems-driven approach that results in more fluid, easier to work with animation that doesn't sacrifice the responsiveness - something that is key to an interactive experience.
This isn't the first game to make use of motion matching, Ubisoft's For Honor was also designed in this way, but Naughty Dog's approach feels even more refined. There's no real evidence of animation snapping between various transitions and Ellie moves naturally through the world with proper weight. It's subtle in many ways but ultimately very effective - it also introduces far more variety and realism into the way Ellie reacts and moves through the game world in the E3 demo, to the point where in traversal, there are various jump animations shown, seemingly chosen dependent on context - Ellie's current position, her momentum and the make-up of the surrounding environment. She's also shown bracing herself by reaching out and placing her hand on the car she's hiding behind. This is likely achieved using raycasting to determine the distance from the object, and the appropriate animation is then selected.
The demo also shows remarkably well-choreographed transitions during combat - Ellie smoothly pulling out her bow, firing an arrow, then concealing it before dodging an attack from an incoming opponent. The Last of Us Part 2 features dedicated crouch and dodge buttons this time around, so we can imagine that this is a fully real-time series of moves input by the player - and possibly one of the most important challenges for the motion matching system. The demo shows this as a fully seamless, entirely natural series of moves and it looks simply incredible, with no snapping between different animation states. From our perspective, we have to wonder just how responsive this is in gameplay, whether it feels like you're actually in control, or whether you're dialling in a series of commands that play out in sequence. Comments from Naughty Dog staff do emphasise responsiveness though, so we are hopeful.
The entire system here is created to simulate realism via choosing and then customising the best motion sequences depending on context, and the more you look at Naughty Dog's E3 demo, the more you see just how well this system works, to the point where some great moments can pass by almost unnoticed. For example, death animations on the enemy AI are entirely seamless, and then there's the physical cost of Ellie's injuries - as she takes a rest while in cover, she noticeably winces in pain. Or how about the moment of pure fear on a downed opponent's face as Ellie prepares for the killing blow?
The violence in The Last of Us has always been brutal. It resonates, it makes us feel ill at ease, and to be frank, it has turned off some people - and that's entirely understandable and may well be part of the developer's plan. This emotional connection with the action is clearly something that Naughty Dog has invested a lot of time, effort and technology in perfecting. The crucial aspect here is that the player has to buy into it, realism is of paramount importance, and if the E3 demo is indicative of the final game, we should expect a stunning - if unsettling - experience.
At a technical level, the motion matching system takes care of how to animate the characters, but the effect of the animation is further enhanced by the addition of collision-based mechanics and simulated momentum. When an enemy swings at Ellie and misses, he realistically bashes through the scenery, with incidental objects flying off in the appropriate directions. Debris flies realistically after bullets hit, while melee weapons have a genuine impact on the character mesh. However, physical contact goes much further than that, with some subtle effects. Assuming Naughty Dog has stayed with real-time cutscenes for The Last of Us Part 2, there's a slight mesh deformation between Ellie and Dina during the trailer's kissing scene, giving a noticeably more realistic look.
The demo also teases other important elements that Naughty Dog is working towards. AI plays a huge role in a game like this and the team is working on realising more natural behaviour between non-player characters. Enemies are set to feature a wider grey area with fuzzy knowledge of the player's location - AI can communicate using voice and sight while searching for traces of Ellie.
The E3 demo showcases accomplished stealth mechanics and managing that grey area realistically is the most difficult aspect. The trailer looks promising here, with a wide-range of context-driven search behaviours and the AI seemingly working as as a proper team. The extent to which this will work as effectively in the final game remains to be seen - the first Last of Us title promised much in its initial E3 gameplay review and as good as the game was, AI wasn't the first game's strongest aspect.
Rendering though? This is an area where Naughty Dog are masters of the game, and The Last of Us Part 2 doesn't disappoint, clearly evolving the systems and technologies it built for Uncharted 4 with a stunning implementation of indirect lighting, volumetrics, reflections and their interaction with materials. Model detail, of course, is second to none, but as good as the game looks, the developer seems to understand that great graphics on their own can't sell the experience. Our big takeaway from The Last of Us Part 2's E3 gameplay reveal is that it's not just about rendering a realistic-looking world, it's about players being able to believably immerse themselves in it. The demo looks brilliant, and we can't wait to see if the final game delivers on the immense promise shown here.