The making of Gears of War: Ultimate Edition

I still remember the day I found out. I was called into Mike Rayner's (Studio TD) office and after the surprise and excitement settled, we discussed the possible approaches which basically broke down into three options: start with the original, start with the latest UE3 engine or update to UE4.

We had a philosophy of keeping the game playable throughout development so we felt that to achieve our goals and ensure we were playing the game as often as possible, our best starting point was with the original codebase. From a content perspective this meant things would already be working on day one. Our engineering team has a long history working with Unreal technology but at that time we had fairly limited exposure to the gameplay code for Gears. We discussed the approach as a team, did a site visit to Epic and discussed it with them and outside of the challenges of working with 10-year-old tools, we agreed this was the best approach. That is how our journey with the original codebase began.

The key challenge in either of the other options was that all the level data would be tough to keep working through the transition and would result in a large amount of time when the game was unplayable. And ultimately we wouldn't be able to guarantee the same functionality as the original since the underlying systems had changed. The only other option would be to throw it all away and rebuild all the level scripting from scratch and we felt the chances of missing something in the gameplay experience would be too great.

With that decision made, our engine team began work immediately on getting the game up and running on Xbox One while our gameplay team went to 'Gears of War Gameplay University' to dissect and learn everything that makes Gears work. This combination allowed us to evaluate gameplay additions based on trying them in-game on Xbox One at 60fps earlier in development.

We are very happy with the decision to go this route but we did underestimate some of the challenges of working with such an old editor. Unreal Engine 3 went through numerous updates and improvements in the editor and we could have improved our efficiency by bringing more of these over earlier in production. The trade-off with staying on the original engine with all the original gameplay scripting was that we didn't need to rebuild everything from scratch to create something that just looked like Gears 1 gameplay, our version would be true to the original.

We started with the original codebase and brought changes and upgrades in based on recommendations from our rendering team. The Coalition and Microsoft are active contributors to the Unreal Engine and have a really good understanding of how different rendering features perform on the Xbox One. That knowledge allowed us to start with the original games features running on Xbox One and then we harvested later additions from UE3 as well as a few from UE4. Most of the editor-facing features were pulled from UE3. An example of this was Lightmass (the Unreal Engine lighting solution), which went through several months of work to get features up to roughly the 2011 time frame. Other features were drawn more from UE4 or our own work on UE4-like physically-based shading and many of our own Xbox One specific enhancements like ESRAM management. We also experimented with some asynchronous compute for optimisation that wasn't available in the UE3 time frame.

We decided 30fps for campaign and 60fps for multiplayer very early on as a design decision and not a technical limitation. The original was a visual showcase for Xbox 360 and we wanted to continue that on Xbox One by significantly updating the campaign visuals at 1080p while also keeping a more consistent 30fps frame-rate than the original. Running the campaign at 60fps is doable, it just doesn't look nearly as good as the 30fps version.

As part of our 'always playable' methodology, most of our budgets are set and maintained by monitoring real world performance in the game. Through our development this allowed us to know where we stood from a performance standpoint with all the original content and monitor and update as each map was remade. We had both automated and manual daily playtests of each map in active development. Specifically for the GPU we targeted between 8-12ms on a multiplayer map with only your character in it to allow for fluctuation with gunfire, grenades etc.

There were many rendering challenges throughout development but we were more often CPU-bound feeding the GPU due to the number of objects added and we had to employ tools to merge geometry together as well as offloading additional rendering to more cores. A lot of hard work went into making our GPU performance solid and this involved rewriting nearly every post process effect in the game, maximising memory bandwidth usage with ESRAM and many more optimisations.

Fun question and you get a description right from the primary developer of our ESRAM management.

Initially we only put the colour/depth buffers into ESRAM as we are forward-rendered, so main buffers (colour/depth) fit nicely into ESRAM at 1080p - this was a huge performance boost. Later in the project we looked at our ESRAM usage again to get extra performance back. From looking at the analysis you can see that only certain passes and resources will give you any benefit inside ESRAM as we were not ROP-write bound in many cases (ie ALU or texture read bound).

We added a new ESRAM allocator that leverages virtual memory to map pages to ESRAM/DRAM per 64k. We added lifetime management for resources we want inside ESRAM so that once they are released the physical pages are returned for other resources to use. Depth and colour targets on Xbox One can be compressed and we noticed in some cases splitting that planes between DRAM/ESRAM can also be a win as it frees up ESRAM memory for other more important resources.

The new allocator was a good performance win for us, especially during shadow passes where we could do all our shadow depth writing and reading inside ESRAM. Using a naive approach of putting 'everything' into ESRAM you could certainly hit the 32MB limit, especially with a deferred renderer. In our case it wasn't an issue and we still had the option to use the DMA engines to move resources in/out if we needed more space.

We do use the audio processing available on Xbox One and the good news here is that we never saw audio as a CPU bottleneck and things just worked. Given the increase to 7.1 this is a huge increase in the number of individual sounds we're processing and the hardware compression is very handy for helping to manage this.

For sure! We are using Lightmass for the lighting and started with the 2006 version. We went through two major integrations, one to bring it up to roughly 2009 and one to bring it to 2011. This work sometimes has to go in stages to make sure all the pieces are working to move to the next level. The first integrations got us global illumination, which in itself was a big step up visually. Later we brought in support for dominant directional lights as well as signed distance field shadows and pre-shadows. This improved specular lighting as well as allowing for dynamic objects to receive and cast shadows that correctly blend with the environment. Our physically-bases shading is based on the UE4 method.

The lighting model was switched to use the UE4-based PBR. This is Lambert for diffuse with Schlick Fresnel and geometry terms with GGX normal distribution. Normal maps could be optionally processed with Toksvig mapping to help with Spec AA. Keeping the UE4 model was very intuitive for the artists who were very familiar authoring using the model.

All lighting is now in full linear HDR space. The post-processing stage was completely rewritten to do all processing in linear HDR space and performs dynamic exposure adjustment, physically correct bloom, lens dirt, bokeh DOF, motion blur, vignetting, filmic tone-mapping, gamma correction, colour grading via 3D LUT and FXAA. Multiplayer omitted motion blur and DOF and was 1.1ms for all other effects enabled. [For] Ambient occlusion, SSAO was depth based on volumetric obscurance and twin pair sampling with a separate dilate and blur pass.

We used the 32-bit RGB101111 format instead of FP16x4. Using FP16x4 gave us no noticeable quality increase so we kept the full rate 32-bit format, which also helped with ESRAM utilisation.

I'm going to answer this from a technical perspective in terms of what features we brought to enhance the experience. Peder, our design lead would have more feedback from a design perspective on the design choices. From a technical standpoint we wanted to provide dedicated servers, responsive controls (fast frame-rate, client side hit detection) and a consistent high frame-rate.

We also wanted to ensure that the matchmaking and multiplayer experience was as up to date and as solid as possible. This meant adding support for such things as squads, playlists and the ability to interact and be able to communicate with the community. Ultimate Edition also relies far more on external services then any of the previous Gears of War titles. From the matchmaking point of view our goal was to get you into a game as quickly as possible, with the best match possible. We added skill-based matchmaking, introduced the levelling system (similar to Gears 3), and in general worked closely with the platform folks to ensure a best-in-class multiplayer experience. One of the things that allowed us to do this was various playtests, both internal and external throughout the development of the project.

We did some updates to the guts of co-op to make replication and joining in progress work better in some areas but the most visible things would be going with split-screen pillar boxing for better FOV (field of view) visibility and all of the improvements for versus play that also apply to co-op.

The short version is that we started with the original and made sure we replaced everything but the collision as a replacement phase and then enhanced it further with lighting, VFX and additional hero assets and work. Outside of collision we re-did the levels from scratch.

On the technical side we had to do a few things to support this. First we build out a big database of all of the assets in Gears and what levels they were referenced from. This helped the art team in planning asset creation. Our tools team further analysed the meshes and determined where objects were scaled and if that scaling was uniform so we had an idea of how many additional assets would have to be made to keep the texel density consistent throughout the game. The original development team was very creative and efficient in re-using assets but there are visual tradeoffs we didn't feel were acceptable in 2015.

Another thing we did early on was to lock down all of the collision etc so that things couldn't be modified by accident. It is sometimes easy to move something by mistake so we defaulted to locking collision to avoid small errors. The bulk of the technical work came from partnering with the art and technical art team through each phase of visual development to ensure that the features needed for levels were working and that the map performance was in where it needed to be. We were optimising and adding features through all of the 18 months of development, which sometimes made it difficult for the art team. We had a really strong team of technical artists both at The Coalition and Splash Damage that really helped balance these issues.

For environments, our initial budgets were for the replace stage and covered vertices and textures. For vertices, our budgets were fairly light on increases in polygons as our artists found they could rebuild the models and with only a 10 to 30 per cent increase in geometry and improve them substantially. With textures, we have twice as many and a resolution higher for 4-16x the memory usage in some cases. The larger additions came from replacing 'kit-bashed' assets and enhancement phases where additional assets beyond what was in the original are created and placed. Finally, optimisation passes would be done that would trade grouping objects together into stitched similar assets for faster rendering. Characters in some cases like the Corpser doubled the vertices and had similar texture increases.

The raw animation data for characters was kept the same to ensure the gameplay experience was preserved. The character models and their rigging as well as secondary animation was overhauled. For example the characters armour vs skin was weighted more appropriately so that armour did not stretch as much and objects on character's belts animate in the campaign. Weighting was also extended to allow up to eight bones per vertex rather than the original four.

In our concept phase we looked at what it would take to provide a full toggle similar to the Halo Combat Evolved Anniversary Edition. We didn't want to compromise how far we could go to accommodate a full Xbox 360's worth of memory as well as the impacts on load times so this feature was removed in pre-production. This led to a discussion about providing a colour filter or estimating the original look with a lower resolution. Ultimately however we felt it would be better if we focused all efforts on just making it look the best it can.

The level designers and artists were empowered to reimagine the lighting as they saw fit so long as the levels stayed within the performance restrictions. This enabled enhancements like you mention as well as light functions throughout the levels that allow lighting textures with colours in areas or around some fires.

It is essentially the exact same code. The Xbox team converts the 360 game and 360 flash PPC executables into native x64 executables, packages those up with the 360 game assets, 360 flash and emulator as a regular Xbox One game, and publishes it.

We are still hard at work optimising the game. DirectX 12 allows us much better control over the CPU load with heavily reduced driver overhead. Some of the overhead has been moved to the game where we can have control over it. Our main effort is in parallelising the rendering system to take advantage of multiple CPU cores. Command list creation and D3D resource creation are the big focus here. We're also pulling in optimisations from UE4 where possible, such as pipeline state object caching. On the GPU side, we've converted SSAO to make use of async compute and are exploring the same for other features, like MSAA.

We have put significant effort into making the Windows 10 version a showcase at 4K, geo and textures were re-authored with 4K in mind so the visual fidelity will really scale up on higher end hardware. We plan to uncap the frame-rate and will ship with a built-in benchmark mode. For anti-aliasing we'll support MSAA and FXAA.

While we haven't locked how far the game will scale down, it is important to us to ensure good performance (at least 30fps) on a variety of hardware. We will support a broad range of resolutions as well as a variety of graphics options for gamers to tweak their setup.