The New Face of Audio - Mixonline

The New Face of Audio

CHARGING INTO THE WORLD OF VIDEOGAMES
Author:
Publish date:
Gad.gif

Welcome to new monthly column “AudioNext,” which is devoted to making sense of the ever-so-burgeoning categories of digitally distributed music and mobile audio in general. I'm very excited about the prospects of documenting the revolution we're all so fortunate to be a part of and — oh, who am I? Good question, let's start there.

Illustration: Victor Gad

I'm Alexander Brandon, audio director at Midway Home Entertainment in San Diego, Calif., and I'll begin by saying it's a great honor to be writing for this magazine. When I mentioned the gig to a colleague of mine, George Sanger (aka “The Fatman,” whom we'll interview down the road), his response was “Mix? Damn. Nice!”

Now, set the way-back machine for 10 years ago, when I started doing music for a game you might have heard of called Unreal. Unreal Tournament followed (1999), and today, Jesper Kyd (who was recently interviewed for mixonline.com) is remixing the piece I wrote in MOD format (which I'll describe in a future column) for the upcoming Unreal Tournament 2007. I have served on the steering committee of the Interactive Audio Special Interest Group (www.iasig.org) and am now on the board of directors of the Game Audio Network Guild (www.audiogang.org). I also recently finished a stint as an audio columnist for Game Developer magazine (www.gdmag.com) and wrote the second “hands-on” book on game audio, Audio for Games: Planning, Process and Production. I've worked with a number of greats in gaming, people such as Warren Spector and Tim Sweeney, and in Hollywood with entertainers ranging from Reeves Gabrels to The Rock. I've experienced a great deal of magic, success and hardship during that time, and I picked up a great deal of knowledge that I'd like to share with you.

This column will be devoted to all kinds of new media, from cell phone technology to satellite radio, but perhaps the largest “new media” category for entertainment right now is games. Just to whet your appetite, let's talk money. Sales of videogames in the month of April 2006 alone were $699 million, according to the NPD Group. While money is a very influential factor in why we do what we do, the other side of the coin is the art. Games are slowly becoming a new form of not only nonlinear artistic expression, but also dramatic presentation. I say slowly because, yes, everyone watches TV and everyone listens to the radio, but not everyone plays games. Several people I've talked to in the past few months have had their jaws drop and eyes bug out when they hear I record orchestras and direct Hollywood voice talent, or that friends of mine in the biz have organized a concert at the Hollywood Bowl where their works, alongside the theme from Super Mario Bros., have been performed by the Los Angeles Symphony. To say it for the thousandth time: We've come a long way from bleeps and bloops, but we have a long way to go.

It's a brave new world. Film composers from Michael Giacchino to Danny Elfman have written music for games. A-list film sound designer Alan Howarth ran Electronic Arts' sound department for a bit. More than 10 years ago, Christopher Walken starred and provided voice for a videogame, and more recently, James Caan and Marlon Brando have had their voices used in the game version of The Godfather. (See March 2006 Mix.) A number of great game achievements have been chronicled in Mix, but where are the resources and the community that can be tapped to understand games at the source? Where does one go to learn not just about music, but sound effects, engineering, integration and voice-over? Where does one begin to get the info to get a foot in the door? I can help you there. Let's start with a few basic principles, some of which you may find surprising.

Play games. Be passionate and involved in what you produce for. I would never presume to beg for a job writing for M. Night Shyamalan if I told people, “Well, I never really watch movies.” If you're an engineer or a Foley artist or a composer, understand that your realm is nonlinear. Things that happen are dictated not by a SMPTE timeline or a frame, but by a set of game mechanics and design criteria.

Read about what's behind the scenes. The game industry, and the game audio industry in particular, is probably one of the most openly helpful industries I know. There is no end of people willing to help you out and give you critiques on your work, and no end of papers, books, articles (albeit articles mostly on older tech) that can get you a leg up on the game industry's inner workings.

Meet and greet. The places where all the folks who are hiring audio pros and the audio pros themselves for games are here: CES, GDC, Game Audio Conference, Project Bar-B-Q and E3. Give them a “Google.”

Know your hardware. No matter what you may have learned about recording, processing and mixing, whether for straight-ahead music, live sound or feature film, you have to rethink your approach when you begin to work in game audio. First, you're entering a completely nonlinear audio experience, most of which is dependent on the playback hardware's capabilities and limitations. And of the major players, none are the same. Further, because of limitations in real estate, whether on disk or in RAM, audio may have to make compromises. From the very beginning of a project, you might have to determine if, for instance, a door slam is 8-bit while the music plays back at 44.1/16. That said, the next-generation playback devices boast much-improved audio capabilities. Here's what you're looking at in the coming year.

PC
The PC market for games has always been topsy-turvy. At the moment, there is a near monopoly on PC profits by a company called Blizzard Entertainment, which developed the insanely popular World of Warcraft, bringing massively multiplayer online role-playing games (MMORPG) to the masses. In addition, the looming next-gen consoles make it rough on PC titles, but some, such as Half-Life 2, surpass every other game on the market in every way.

On the PC, memory and space are no longer limited by hardware. You could probably get away with about 128 MB of RAM for audio on a next-gen PC first-person shooter, and with lossless compression, that's more than enough to reproduce at least two dozen footstep sounds, a few dozen localized point source and ambient sounds, any number of player sounds and as many hard drive streams as you would need for music and voice-over. Just make sure you get that real estate early on in the development process.

PLAYSTATION 3
PlayStation 3 (PS3) is getting mixed reactions well before its launch. At $599 for a fully decked-out PS3, you could buy both an Xbox 360 and a Nintendo Wii, and following lackluster demos at this year's E3, critics have been skeptical. But they said that about PlayStation 1 and look what happened there. Don't count the current world champ of consoles out yet.

Processor: Cell SPE
Voices: 512
Sampling: 44.1/48kHz, 16-bit
Channels: stereo/DD/DTS
Sound Memory: 256 MB shared = roughly 24 MB max, 8:1 average ratio = 192 MB of sound at any one time.

XBOX 360
The 360 is the “Commodore Amiga” of consoles. For those not familiar with the Amiga, it had the best and most easily accessible hardware capabilities, but didn't end up with the largest roster of games. Out of the gate, the Xbox 360's sales are showing promise, mostly with its ingenious Xbox Live Arcade feature, which allows broadband downloading of smaller titles and earning points so users can compete online with friends.

Processor: 3 cores at 3.2 GHz
Voices: 256
Sampling: 44.1/48kHz, 16-bit
Channels: stereo/DD/DTS
Sound Memory: 512 MB shared = roughly 48 MB max, assuming an 11:1 average ratio with XMA = around 528 MB of sound at any one time.

NINTENDO WII
Nintendo's GameCube was a little ahead of the PS2 and a little behind the Xbox, and the Wii looks to sit in the middle for next gen in a different way. All voices are mixed in mono on a Wii, so one has to do interleaving in real time through two mono buffers, so the use of middleware such as FMOD will be a great help to people seeking (excuse the pun) to do streaming, and it's looking likely that Dolby PLII, once again, will be used instead of Dolby Digital. On the plus side, the memory has increased to the same amount as Xbox; design-wise, the Wii has included a speaker in the controller, which, like the rest of the controller's design, is something of a breakthrough.

Processor: IBM “Broadway”
Voices: 64
Sampling: 44.1/48kHz, 16-bit
Channels: stereo/DPLII
Sound Memory: 512 MB shared = roughly 48 MB max, compression unknown as of this writing.

RAM PERCENTAGES
This is all commercially available information, but where we really get into the nitty gritty for next-gen games is the magic number: 10 percent.

Ten percent is an average that you can expect for RAM usage; therefore, on a PlayStation 3, your average will be 24 MB; on an Xbox 360, 48 MB. These numbers extrapolate well from the PS2 and Xbox, though the Xbox has a little less of an advantage with double the memory rather than four times as much (PS2 with 2 MB, Xbox with 6 MB).

Keep in mind this is an average. An Xbox 360 puzzle game might have even more space to play with, giving room for a mini-GigaStudio yielding incredible adaptive soundtracks, while a high-end Half-Life 2 first-person shooter will have to fight for the 48 MB over such things as membrane shaders and per-pixel lighting.

If you're a producer, programmer, audio director, sound designer or composer, then think about what is realistic. Map memory early in the game and consider the needs of design first. That way, the audio production team can intelligently work within its limits and not create 96kHz/24-bit audio right and left.

The good news is that for the first time, audio has a little breathing room. People were saying that about the PS2, but as with PS1, the limits did not accommodate the rapid increase in expectations by the market. People need real sound now, comparable to films in terms of dynamic range, bit rate and mixing/mastering. And now, at last, we can give it to them with no argument from the development team.

There's a great deal more about game development that we'll cover in future columns. Next up in the digital domain will be another huge emerging digital market: ringtones and mobile audio.

Alexander Brandon is the audio director for Midway Home Entertainment in San Diego, Calif.

ONLINE CHECKPOINTS
Gamaa (www.gamasutra.com): Gamasutra is the sister online site of Game Developer magazine and is chock-full of daily news. In addition, a quick search for audio will yield about 20 features and 30 or so game audio galleries that spotlight individual composers or sound designers.

IA-SIG (www.iasig.org): The Interactive Audio Special Interest Group is an arm of the MIDI Manufacturers Association and specializes in games, Web, portable — just about anything interactive. This group is where you can get a great head start, as volunteers from the industry developed 3-D sound propagation standards that are now in many hardware systems such as Xbox and Xbox 360, as well as Downloadable Sounds (DLS).

G.A.N.G. (www.audiogang.org): If you want to get into game audio, join G.A.N.G. (Game Audio Network Guild), which provides forums for discussions with industry veterans, discounts on software and industry events, scholarships, white papers and news — all rolled into a single source.