Illustration: Hugh D’Andrade
If you’ve been playing (or watching your kids playing) any new videogames recently, you’ve probably noticed that the soundtracks have become incredibly sophisticated — not to mention loud. The loud part isn’t worth discussing much — except please turn it down, for your own sake, especially if you’re playing on headphones — but the sophisticated part is pretty amazing. Although most of the folks reading this have had to deal with complex mixes for recording, broadcast, live sound or film projects, how many have done projects that have to mix themselves on the fly, in real time, in the audience’s living rooms? Well, if you’ve done any game audio in the past couple of years, that’s just what you’ve been doing.
I have to confess that I don’t have much to do with the videogame world these days. In the late ’80s, I did the music for a game that a friend of mine was designing. I had to write several 30-second tunes and turn them into standard MIDI files (which were brand-new at the time). I was asked to do three different versions of each one to accommodate systems that could play two, three or four different musical sounds (that is, individual notes, not files) at a time. Oh yeah — and they had to be able to loop. I don’t know if the game ever got finished.
A few years later, I got a gig creating MIDI and audio files for a fledgling company called Harmonix, which was developing a musical game with some interesting interactive aspects. The company ended up not using any of my music, and it turned out that the game wasn’t terribly successful. But Harmonix has done better since: Last September, Viacom bought it for $175 million. Unfortunately, I have not been paid in stock options.
The fact that Paramount Pictures and MTV’s parent company would throw that kind of money around is just one indication of how serious the game industry has become. So is the fact that within a week of its release, the Sony PlayStation 3 was selling on eBay for up to four times its list price (assuming the sellers actually had them and weren’t planning on shipping bricks). Music budgets on some “AAA” games from major publishers are approaching those of feature films, and more than a few orchestral musicians are finding themselves playing as much on game-scoring sessions as they are on movie soundtracks. Not to mention that orchestras from London to Brazil to New Zealand are putting on concert programs comprising partially or completely game music to attract new audiences. Tommy Tallarico, a prolific California game composer who is one of the prime movers behind Video Games Live, told the BBC recently that these concerts will “revitalize” classical music in the 21st century the way opera did in the 19th. (It did?) I don’t know if listening to a medley of hits from Pong, Mario Brothers and Warcraft is artistically on the same level as The Magic Flute and Tosca, but the audiences seem to be having a good time.
Mike Verrette is a producer for midsize game-development company Iron Lore Entertainment, located in one of the old mills in Maynard, Mass., where the once-mighty Digital Equipment Corporation (DEC) made its home. Verrette is a drummer, a composer and a sound designer, and he was a student of mine some 10 years ago. He cut his teeth in television and commercial production, and then started working in the videogame world about five years ago. He is the founder of the New England chapter of the Game Audio Network Guild. Iron Lore, which he joined in 2004, employs some 38 people, including one full-time composer/sound designer, the position Verrette started in and now oversees.
Iron Lore is best known for Titan Quest, a first-person action game for the PC built around ancient Greek and Egyptian myths; it was released this past June. The expansion pack with new characters and situations, Titan Quest: Immortal Throne, ships next month. The game is published by THQ, a huge outfit that handles titles from Scooby-Doo and the Cyber Chase and The SpongeBob SquarePants Movie to WWE SmackDown! Here Comes the Pain and Full Spectrum Warrior: Ten Hammers.
According to Verrette, the music and sound production for Titan Quest are pretty straightforward in some ways. “It’s almost identical to doing film post,” he says. “We designed a room around Pro Tools, and we have a library of canned effects and a location recorder. One of the spaces we lease in the building is DEC’s old in-house television studio, and it’s nice and quiet so we can do some recording in there. The music is mostly orchestral, so we have a PC running GigaStudio with the Vienna Symphonic Library. The live tracks — percussion, flute and a vocalist — we recorded with Futura Productions, which is in the Sonic Temple, a classical studio in Boston built in an old Masonic temple.
“We outsourced the dialog to a studio in Santa Monica [Calif.] because our publisher has a relationship with a casting director out there,” he continues. “A publisher can bring a lot of that sort of thing to the table: Besides marketing and distribution, they can sometimes provide you with contractors for things like dialog casting or help establish relationships with third parties for licensing agreements. Using the casting director freed up a lot of my time so I could focus on other things.”
Each version of the game has close to two hours of music and uses up to 2,000 sound effects, from footsteps to monster growls. “There isn’t a single sound in the game that we pulled directly out of the library,” Verrette says. “Everything’s layered up in Pro Tools. And the game itself does additional processing.”
One other similarity to traditional sound design is the way a designer decides to use certain kinds of loops. “In a racing game,” Verrette explains, “the engine sound is looped and it changes pitch as the rpms go up. But how do you get it to sound realistic when you shift gears? Like sampling a musical instrument, you can only pitch-shift so much, and at a certain point, you have to change the sample. Same with a machine gun sound — you don’t want to hear the same six bullets all the time.”
But that’s pretty much where the likenesses end. Because a game is constantly changing, the sound environment has to be completely fluid, which means the audio producer has to work closely with the artists and the programmers early in the development process to anticipate what the game’s audio requirements will be. “We spent about a month doing an audio design document before we started building,” Verrette says.
“A game might have lots of different surfaces the characters walk on, like a stone road, a grass field or a bridge,” he continues. “In real life, the footsteps on all those surfaces would sound different. In a film, you just put the right sounds in the right place, but in a game, you don’t know where the player is going to walk, so you have to figure that out in real time. One solution is that the sound person looks at all the levels and uses a tool to designate which areas or zones in the game belong to which footstep effects.
“But a step beyond that, which we’ve incorporated into our development engine, is to build the sounds directly into the ground textures: As the game designer puts down a grass field, the sound is already built into it, so the sound designer doesn’t need to go back and revisit all the levels. Every asset in the game already has sounds attached to it. When the artist creates a sword, the game knows which sounds are associated with that sword: how it sounds when it’s picked up, when it’s swung and whether it hits metal or fur. The idea is to design a system that takes care of itself, more or less.”
Verrette did a demonstration for one of my classes this year in which he showed how the game engine created the sound mix on the fly. Each sound — dialog, music, ambient effects, Foley, etc. — has its own layer and therefore its own audio channel (or two). Background sounds are looped and randomized in terms of timing, pitch and stereo placement. Each element has a timer that determines how frequently it plays while the scene lasts so that the listener hears a non-repeating soundscape.
“Sometimes sounds work a lot differently than you think they will,” Verrette notes. “It’s one of the biggest challenges I found moving from post-production because the mix isn’t static. You can be auditioning your sounds on desktop speakers, but when you get them into a game, they sound completely different. If a sound needs to be totally bass-y, when you put it with everything else it might come up against, it loses its ‘oomph’ or it disappears completely. So sometimes you build sounds that don’t sound great by themselves because you have to take into account everything they might be combined with.
“You also want to make sounds as short as possible so they take up less memory and load fast, and when I first started out, I had a tendency to chop things off short, but I found you lose a lot. I was amazed to find how much the last quarter of the sound helps us to identify it. It’s not just the initial crunch of the footstep that tells us what kind of surface it’s on, but also how it peels off that surface. You can’t sacrifice your reverb tails too much. You do want to make the sounds short, but you still have to sell them as what they’re supposed to be.”
Stitching the music together coherently as scenes change is another major concern, and again, the solution can be counterintutive. “If you want to make the transitions musical,” Verrette explains, “your tendency is to make the loops even numbers of bars and transition on downbeats. But that means that if you want to move quickly, the loops have to be short. And even then, waiting for the downbeat doesn’t always work well because it makes the game feel unresponsive. Instead, we found the best thing to do is to literally crossfade to a different piece of music. When you listen to the transition all by itself as a piece of music, it doesn’t sound as good, but when you’re playing the game, it keys into what the game is doing and makes it feel more responsive. That’s more important than the musicality of the transition.”
Another consideration that post-production mixers don’t have to deal with nearly as much is the delivery platform. Iron Lore’s products so far have just been for Windows PCs, but the company is looking to expand into console development for future products. But even within the PC world, there are many variations that the designers have to worry about. “There’s a lack of standards in hardware,” says Verrette. “You don’t know what the end-user’s going to have. Is he going to be using 2.1, 5.1, 7.1 or headphones? And the sound cards vary: Sound Blaster has a monopoly except for the laptop market — they have onboard chips — but even then, the cards all have different throughput, DSP options and numbers of channels. You have to prioritize the sound so that when the engine runs out of channels, it knows what to drop; the dialog can’t drop out, but maybe the Foley can.
“You have to specify a minimum spec and a recommended spec. This requires a DVD player that runs at ‘x’ speed, this much disk space, this much RAM and this processor speed. And besides the sound cards, the graphics cards make a difference. Plus, you have to worry about what the user is running in the background, like chat clients or antivirus software. You can recommend they turn everything else off, but they often don’t. And then when they have problems, they go online and bitch about how you don’t know how to make a game. The community is very unforgiving — they all know everything,” he says with a laugh.
And, of course, the operating system itself is a big variable. “For the last project, Windows 98 was out of the picture,” Verrette says. “We only supported Windows 2000 and XP. It is always difficult to choose which operating systems you are going to support because you want to make sure you are not alienating large groups of the market that may not be running on the most up-to-date systems. In the future, we will have to look toward supporting Vista, as well.”
Next month, I’ll talk more about the joys and perils of developing for console platforms, and also a bit about the culture of game development and how it affects the people who are in it.
Paul D. Lehrman tries to save his thumbs for typing and playing music. More than 50 of his past columns are now available in book form: The Insider Audio Bathroom Reader is available at
insideraudio.com
and
www.mixbooks.com.
