WGDC 2000 - DirectSound and DirectMusic
New sound and music features in DirectX 8
Graphics seem to get all the attention these days. Games often rise and fall based on their visuals, and gamers lay down hundreds of pounds on the latest graphics cards and big screen monitors to get the best possible quality. But at the end of the day, as games like System Shock 2 and Thief have proven, sound can be just as important to a game's atmosphere.
In the last version of DirectX, Microsoft improved support for audio effects in DirectSound, and introduced their own dynamic music API in DirectMusic. Now, as part of our on-going coverage of last month's Windows Game Developers Conference in London, EuroGamer takes a look at the audio side of DirectX, and has a peek at what Microsoft has in store for us this autumn...
Spice Of Life
DirectSound and DirectMusic have had a fairly radical overhaul since last year, and the two APIs are now much more closely linked, making it easier for developers to apply DirectSound effects to DirectMusic, or to use DirectMusic to create dynamic sound effects.
The aim of DirectX 8 is to allow real-time movie-quality interactive music and sound effects, "and we're not talking about some Jean Claude van Damme straight to video movie here". Microsoft audio guru Brian Schmidt compared traditional wave file sound effects to using 2D sprites for your graphics - they soon get boring and repetitive.
So what is Microsoft's solution? A mixture of dynamic sound effects and music created using DirectMusic, and applying DirectSound's audio effects to your pre-recorded sounds in real time. DirectSound now includes positional audio, parametric EQ, flange, echo, pitch shifting, reverb and decay effects, as well as a whole host of other options for developers to tinker with.
By applying audio effects to a basic sample, you can make it sound radically different. One obvious application for this is to make your monsters sound more individual by pitch shifting their voices and applying other effects to them. Now if you have a dozen similar looking monsters on screen together, they can all sound slightly different to each other simply by adjusting the audio effects applied to their voices.
To hear it straight from the horse's mouth, download the MP3 (1.2Mb) of Brian Schmidt playing around with DirectSound's effects during his DirectX 8 audio overview talk. Ignore the 3D positioning bit at the beginning of the helicopter example - the speaker system in the conference hall was only mono...
Not Just Music
The important thing to realise is that, despite the name, DirectMusic is not just about music - it can also be used to control sound effects, and one of the major additions in DirectX 8 is a new audio scripting system.
The scripting language is very simple, the idea being that the composers and sound engineers will be able to edit scripts themselves, instead of having to go to a programmer every time they want to add a new piece of music or change how their sound effects are handled in the game. Programmers simply implement a series of hooks in the engine so that the audio scripts know what is going on in the game, and the sound department can do the rest.
Imagine a sports game where the performance of the two teams and information about events such as tackles, intercepts or goals are passed on to an audio script by the engine. Cheers and other crowd noises can then be dynamically altered according to what is happening in the game, all controlled by audio scripts and mixed together by DirectSound. You could even apply positional audio effects to the noise, so that you can hear the cheers coming from your team's end of the grandstand as you score a goal.
The example which Brian gave was of a helicopter, with a noise made up of a high pitch whine, the whirling of the blades, and the low throbbing of the engine. DirectX can treat each of these sounds as a seperate audio stream and, using an audio script getting information from the game, apply various effects to each stream depending on factors such as the engine throttle and the movement of the vehicle.
The seperate streams are then mixed together by DirectSound, and the result can have any environmental effects (such as reverb) added to it, and finally a 3D positional audio effect so that the noise of the helicopter seems to come from the right direction. It all helps add to the realism, and should help make game sound less repetitive.
Following The Script
Of course, audio scripts can also be used to control dynamic music, making music react to what is going on in the game, with the aim of allowing game music to be more cinematic.
In a movie, the composer knows what is going to happen in advance because the action is completely linear, and so they can write a soundtrack based around that. In a computer game though, the composer doesn't know what the player is going to do, how long they are going to take to reach the end of a particular mission, or where and when battles will take place along the way.
The solution is to write several variations to cover different situations - a slow soft piece for background ambience, and increasingly intense pieces for action scenes, building to a climax for the big end-of-level boss encounter. DirectMusic can then switch between these depending on the player's actions, changing at the end of a beat or section of music, and playing transitions as necessary. To hear this in action, listen to this MP3 (2.5Mb) of a demonstration given by Brian Schmidt.
Individual enemies, weapons and locations can also have their own "themes", which can be introduced into the soundtrack by DirectMusic at the appropriate time. Just as there is a piece of music for whenever Darth Vader appears on screen in a Star Wars movie, now your game's soundtrack can introduce themes for different characters and mix them into the background music in real time.
And the flow of information can go both ways - your audio script can tell the game what it is doing, allowing the action to fit the music as well as vice versa. Imagine you are approaching an area where the developers have laid an ambush - the music starts to crescendo, and just as it breaks loose into a hard grinding climax the monsters leap out to attack you, right on cue.
Variations On A Theme
DirectMusic also allows you to store up to 32 different versions of the same section of music. For example, if you record several takes of a live guitar solo, you can put them all into a "wave track" in the song, and DirectMusic will then select at random which version to play each time it loops.
Or you could record several slightly different drum beats to make your rhythm section sound less repetitive and mechanical. And by combining these various pieces randomly, the soundtrack should sound more like music, and less like a repetitive loop waiting for something to happen.
Naturally this can also be applied to sound effects - the example Brian gave was background jungle noise. The ambient sound could include various bird calls, crickets chirping, water dripping and other noises; each one in its own track in DirectMusic Producer, and each with several variations using different sounds, or different effects applied to the same basic samples.
When the ambient sound is played back in the game, DirectMusic will randomly pick which version of each track to play every time it loops. So even if you only have half a dozen tracks and half a dozen variations of each one, that still gives literally thousands of possible combinations, meaning that each time it loops you will hear something slightly different.
To hear an example of this being used in music, as well as Brian explaining the jungle sound idea and trying to make a bird call noise (and failing miserably), listen to the MP3 (2.3Mb) of the demonstration from one of his talks at the WGDC.
Conclusion
Microsoft are particularly pushing DirectMusic as a solution for sound and music on their X-Box console, and with DirectX 8 they are trying to make it easier for developers to use dynamic music and sound effects in their games, and to apply a variety of audio effects to them.
If developers take full advantages of the capabilities that are there, then the music in your computer games should adjust in real time to match the on-screen action, every monster should sound slightly different, and background ambient effects should be more than just short audio loops.
The end result should be a less repetitive, more cinematic feel - the future of gaming audio has never sounded so good...
-
WGDC 2000 - Future Of Windows Gaming