Mark Cerny talks Dolby Atmos support for PlayStation 5
How heightened surround sound works in the new system software.
In the wake of our reporting on PlayStation 5's new Dolby Atmos support in last week's DF Direct Weekly, Sony's marketing team got in touch with a response from PS5 lead system architect, Mark Cerny, on how the new surround sound set-up actually works. In the Direct, we speculated that the PS5's existing, very impressive Tempest 3D audio data was most likely being recalculated and injected into the Dolby Atmos 'container'.
If you think about it, this is an innovative solution but it's also the only way that the PS5's existing library of titles can actually work with Atmos set-ups as until now, developers would not have mastered audio to support the 7.1.4 speaker system.
In his statement, Mark Cerny takes us through the entire audio pipeline in PlayStation 5 and how support has been extended to accommodate the extra speakers that an Atmos set-up supplies. The nature of Sony's solution means that the Atmos support is entirely lag-free, in contrast to some issues we've had with Xbox and Windows PCs. Is this solution actually an Atmos mix though? It's 'as good as' from a mathematical perspective, but with full support now added, sound engineers can master their audio mixes on full Atmos-spec equipment, meaning there's scope for further improvement.
Here's the complete breakdown:
It's probably easiest to talk about Tempest-based 3D Audio and the Dolby device support in terms of Ambisonic audio, which is increasingly popular these days (note there are other strategies for 3D Audio, including ones that use discrete 3D audio objects, but situation is rather similar).
Ambisonic audio can be viewed as a pretty radical extension of stereo audio. With stereo audio, the game's audio engine (or the middleware being used) will add a sound source into one or both channels based on its location - if the source is to the right of the listener it's primarily added into the right channel, and so on. With Ambisonic audio, there are a lot more channels - fifth order is very common and uses 36 channels, so it allows pretty good locality to the audio. A sound source is then added into those 36 channels based on location; the math is a bit more complex than when using stereo but not overwhelmingly so. Because the audio processing is channel based (albeit at 36 channels rather than 2 channels), the audio designer keeps very good control of mixing, filters, etc., and strategies like dynamic range compression (where audibility of certain important audio such as player character voice is ensured) can be used as usual.
The Ambisonic audio channels are then handed off to the Tempest 3D AudioTech engine for rendering, which is to say that the Tempest engine uses the player's HRTF and the speaker locations to create an appropriate audio stream for each speaker. The Ambisonic audio channels encode all directions, including above the player; even if rendering for headphones, this is very important, because it allows a sound "above" the player to be processed in such a way to sound as if it is truly coming from above - this is of course where the HRTF with its encoding of head and ear shape comes in.
Up until the most recent update, the Tempest engine would render the information in the Ambisonic channels into headphones, stereo TV speakers, and 5.1 and 7.1 audio setups. Now 7.1.4 has been introduced, with its four overhead speakers, but really nothing changes in the overall Tempest rendering strategy - the 36 Ambisonic channels already include audio coming from all directions, including above the player. To put that differently, the support of the four overhead speakers is "first class" support, they are handled just like any other speakers. Also note the rendering latency for these new speaker setups is identical to what it has been in the past for stereo, 5.1 and 7.1.
As a result, the 7.1.4 experience for existing games should be quite good. It is true that the game teams could not test with these speaker setups but support should be pretty automatic, the necessary game audio data is already there in Ambisonic form. Going forward, there's an opportunity for improvement as the sound designers can verify the highest quality of audio on 7.1.4 speaker setups as well.
- Mark Cerny, Lead System Architect of the PS5
This statement made its way into the upcoming DF Direct Weekly #124, going public tomorrow, where John Linneman and myself discuss Mark's comments and also our 'crowd-sourced' testing on surround sound latency. That's on top of a range of other news topics that have combined to deliver what may be the longest Direct we've ever done, so look out for that tomorrow at 16:00 BST.