The battle for living room supremacy is on: No longer a fringe element in the consumer electronics universe, videogame consoles are fast becoming all-in-one entertainment systems, delivering high-def gaming, video, audio and Internet connectivity in one easy-to-use system. And with the prices of high-def-capable LCD TVs and multichannel home theater systems plummeting to new lows, the buying public is quickly getting cozy with all things HD. So it's not surprising that the gaming industry is striving to meet and push this demand.
On the audio side, the news just keeps getting better for developers. All three of the major gaming consoles — Sony PlayStation 3, Microsoft Xbox 360 and Nintendo Wii — feature some form of surround sound, and the PS3 and Xbox 360 both include support for next-gen optical media formats — Blu-ray and HD-DVD, respectively. Add to that faster CPUs, more physical RAM and access to massive amounts of game data, and it's easy to see the vast potential that these systems represent for audio content creators.
WHAT'S IN THE BOX?
As most people already know, videogame consoles are essentially boxed computers, and as computers go, all three of the major consoles sport some rather impressive features and specs. The PS3 features the multicore Cell processor. Jointly developed by IBM, Sony and Toshiba, the Cell combines a 64-bit PowerPC processor with eight additional processing cores on a single chip. The current Cell processor hums along at a stout 3.2 GHz per core, and speeds could go as high as 6 GHz within a year's time. The PS3 includes 256 MB of system RAM and 256 MB of dedicated video memory. Users can choose between a 60- or 20GB hard drive or drop in a Serial ATA drive of their choosing. Processing and memory aside, the PS3 includes both optical audio and HDMI connectivity (with full 1080p HD video resolution) and supports up to 7.1 surround sound, including DTS and Dolby True HD. When surround hardware is unavailable, the box defaults to Pro Logic II and stereo. And, of course, the PS3 includes a standard Blu-ray high-def DVD player.
Microsoft's Xbox 360 uses three custom PowerPC-based CPUs, each running at 3.2 GHz. The unit includes 512 MB of physical RAM and a user-replaceable 20GB hard drive. The unit supports 5.1 DTS and Dolby Digital via an optional optical audio output, and like the PS3, the unit defaults to Pro Logic II or stereo output. The unit can be further upgraded with the recently released Xbox 360 HD DVD Player, which, as the name attests, adds complete HD-DVD video support and a wireless remote.
Nintendo is a little more tight-lipped about its console's specs than Sony and Microsoft. What is known is that Wii uses the IBM Broadway CPU, another PowerPC-based chip. Unofficially, most industry insiders have the chip clocked at around 729 MHz. The unit has 88 MB of RAM, though 24 MB is dedicated strictly for video. For storage, the Wii ships with 512 MB of Flash memory, which can be expanded to 2 GB. Audio output is limited to analog stereo outs with Dolby Pro Logic II, though the Wii controller itself includes a built-in speaker. The unit lacks either HD-DVD or Blu-ray compatibility, but standard DVD playback is rumored to be a forthcoming feature.
That said, it's encouraging to see surround becoming a reality for all of the platforms. “We finally have an environment where all three major game consoles are supporting some kind of Dolby surround sound,” says Jack Buser, worldwide technology evangelist for Dolby. “So I think people that are creating audio for game-development houses no longer have to negotiate with their producers over whether support for 5.1 surround be included or not. It's one of those things that have sort of become the de facto standard for game audio. So as a result, rather than argue over format, they'll be able to spend their own clock cycles in a much more productive way, going out there making game sound even better.”
THE FIGHT FOR MORE
With extra horsepower, expanded memory and support for multichannel audio, the approach to creating audio content for these platforms has changed dramatically from the early days of game audio. Audio teams are now brought into the development process sooner than ever before because, in addition to creating multichannel audio from the start, all of the major consoles use shared memory, where all game-play elements are drawing from the same finite resources. “Moving from PS2 to PS3, the big difference is that we've got HDMI on the box, which allows us to have up to eight channels of discrete audio for game use, something we weren't able to do on the last platform,” explains David Murrant, sound design manager at Sony.
“Fundamentally, that is a huge change for us, having to deal with and manage assets that have to be developed for the environment,” Murrant continues. “The most we had done is Pro Logic II audio, which is supported out of the box for game use. The other big difference is that the PS2 had dedicated audio memory, which was a dedicated sound processing unit. So we had two megs of RAM on the previous box. Now on the newer system, we have to essentially fight for our corner because it is a flat memory model. So with the PS3, we have 512 megs of total memory, and half of that is used for video. So of the main memory that is left, approximately 256 megs, we try and shoot for approximately 10 percent of that for audio.”
Developers working on the Xbox 360 faced a very similar shift when moving from the original Xbox, which, like the PS2, had dedicated audio processing hardware and memory. “I'll be honest, it's sort of a double-edged sword for us because there is so much more processing power in the 360, but it's all shared processor power,” explains Marty O'Donnell, audio director and composer with Bungie Studios (owned by Microsoft). “And that means we have to grab for as much of that processing power as we can get. On the Xbox 1, that fight never happened because there was a chip just dedicated to audio processing. And we got what we could get out of the Xbox, which was really nice.
“On the Xbox 360,” he continues, “there are three beautiful processors, and if you wanted to do a graphics-only game, you could use all three processors to make the most incredible graphics. So there is a little more of a give-and-take kind of thing. We have to make sure that the engineers and the artists and the audio guys get together early in the process and stake out their respective territory.”
HOW DOES IT SOUND?
But hardware support for multichannel audio and fixed-memory models are not the only major changes happening on the audio front. With more memory available to audio, as well as the processing power to render and process more voices effectively in real time, the overall quality and fidelity on the PS3 and Xbox 360 have improved significantly from the previous versions.
“We can get more content and do more streams with the PS3, but the biggest difference is fidelity and the quality of the content we can create,” Murrant continues. “Typically on a PS2 title, if you've got a ton of sounds, you've just got the two megs of RAM, and there were some creative ways to get more memory. But you've essentially got to start downsampling. A lot of samples can be anywhere from 17k to 22k for sound effects, depending on the frequency range of the sound. On the PS3, with the additional memory, we're able to have full 44.1 samples. The fidelity and the quality of the output is significantly better.”
“On the first Xbox, we were capped at 64 surround sound voices,” says Jay Weinland, audio lead at Bungie Studios. “And those are voices that could be encoded as 5.1, not stereo or mono. And now the sky is the limit. We've been told that it is possible for the Xbox to handle up to 320 simultaneous [surround] voices. I don't think anyone has actually been able to hit that. But we certainly are able to do a lot more surround sound voices than we were able to before. And the second big change is that we've moved from a fixed-rate compression scheme to a variable bit-rate compression scheme, which allows us better compression that sounds better.”
“Most people who are doing audio for television or movies never have to think about [data] compression,” adds O'Donnell. “They might have to think about the medium and make sure that their mix sounds good out of a TV speaker, but we also have some data restrictions. We're recording everything at 24/48, but when it comes down to what we put in the game, we have to go through steps to get things compressed. And now that we have variable bit rate, we don't have to compress everything in the same way, so that is pretty cool.”
HIGH-DEF: NOT SO FAST
On paper, support for next-generation optical media formats — Blu-ray and HD-DVD — would seem like an instant shot in the arm for game consoles, allowing game developers to create games up to 50 Gigabytes in size. The reality is that while it's possible to store many times more data than a standard-def DVD, seek times haven't ramped up at the same pace. And for games that require streaming, the added storage won't necessarily translate to enhanced capabilities until access speeds can be increased. “We look at something like HD-DVD and see all of the increased memory footprint, and that's great,” says O'Donnell. “But getting data off the disc is actually more important to us than how much data you can put on the disc. Speed of access is hugely important to a real-time game. DVD is not faster, in terms of access times, than a CD. And HD-DVD and Blu-ray, I believe, are even slower.”
“Resistance: Fall of Man and NBA 07, which were two launch titles, were both on Blu-ray, and essentially what that means for us is that there is a greater volume of content that we can store,” adds Murrant. “The bottleneck comes down to whatever the game is doing. If you're streaming lots of textures or maps, or who knows what, we're all kind of fighting in the background to keep those resources moving. That's the issue for us. On the PS2, we were able to say that typically most levels would be loaded into memory, and as you'd finish each level, the next one would load. There wasn't a lot of game content being streamed so we could have up to eight streams of audio playing off the disc because we could pretty much take care of it. The things we have to contend with on the PS3 is that everyone is trying to stream, and these worlds are big and detailed, and everyone keeps wanting to bring more stuff in and they're hitting the disc harder and harder.”
While integrated surround sound and the ability to stream more voices of audio content are certainly allowing game developers to create more detailed and complex-sounding games, some of the biggest advancements in game sound are happening as a result of increased fidelity and the ability of the game engines themselves to intelligently process audio assets in real time.
“I worked back in the day when you had 64k of memory for music and sound effects,” concludes Murrant. “And what I'm seeing now is truly next generation. Take, for instance, the technical aspects of, ‘Hey, I want to hear this sound filtered because it's on the other side of a door.’ I shouldn't have to be designing it to sound like that. We should be able to do all of this filtering on the fly. And that capability is here now. Am I seeing it in a lot of titles? No. But as people get more and more into the development cycle, you'll see a lot more of that. Also, we're no longer limited by the reverbs that we had on the PS2. We now can create our own reverb set, which is getting close to some of the high-end commercial reverbs that you can buy for Pro Tools. And you're hearing those things being used in the games. The quality aspect is getting closer to film.”
Rob Hanson is a former editor at Mix and Remix magazines.
Processor: multicore Cell processor at 3.2 GHz
Sampling: 44.1/48kHz, 16-bit
Channels: stereo/DD/DTS, up to 7.1
RAM: 256MB system, plus 256 MB dedicated to video
Processor: IBM Broadway CPU
Sampling: 44.1/48kHz, 16-bit
Channels: analog stereo/Pro Logic II
RAM: 88 MB, including 24 MB dedicated to video
Processor: 3 cores at 3.2 GHz
Sampling: 44.1/48kHz, 16-bit
Channels: stereo/DD/DTS up to 5.1
RAM: 512MB system