Since Stan Lee’s “The Avengers #1” comic book was released in September 1963, the superhero team has had their stories told and re-told in TV series, feature films, and games. With each new entertainment iteration, the characters have grown more complex, more dynamic. They’ve evolved and continue to change in subtle ways with each new interpretation of who they are and what they fight for.

The latest Avengers offering is the Square Enix/Crystal Dynamics game for Windows, PlayStation 4, Xbox One, and Stadia (with upcoming releases for Xbox Series X and PS5 next month). The characters’ stories are a mix of MCU and comic canon. But for those not interested so much in character lore, Marvel’s Avengers offers an action-based RPG in which, for instance, you can be the Hulk and do some serious smashing!

Here, the Crystal Dynamics sound team — audio director Jack Grillo, audio lead Chase Ashbaker, senior sound designer, Phil Lamperski, sound designer Morgan Green, and sound designer Brennan Anderson — and Square-Enix audio director Atsushi Suganuma (whose team handled Foley on the cinematics) talk about their approach to designing sound to fit different game-play modes, designing sound for the six playable Avengers characters (including how they incorporated iconic sounds and used creative license to come up with new sound ideas), designing destruction, creating robotic vocals, managing dialogue for 13 different languages, collaborating with Composer Bobby Tahouri, the challenges of recording Foley for the cinematics, the challenges they faced when mixing the game, working with Crystal Dynamics’s updated proprietary Foundation Engine, and more!

Marvel’s Avengers | Launch Trailer

There is a single-player campaign (origin stories for key characters), Avengers Initiative, missions (single and multiplayer), the ability to play missions in parallel with a campaign — so many interconnected ways to play and experience this game world. What did that mean for the sound team?

Jack Grillo (JG): The variety of game modes featured in Marvel’s Avengers created structural and aesthetic challenges, as player expectations change from one mode to the next. But the biggest challenge for us was organizing or managing the audio team’s time. Because this title is just the first instance of a larger, long-term game experience, the entire studio had to experiment and remain flexible throughout development. This means that many of the final elements from other departments only came together recently.

Because this title is just the first instance of a larger, long-term game experience, the entire studio had to experiment and remain flexible throughout development.

In order to manage all of that content, we spread out the ownership and responsibility over broad sound categories. I often worked on the campaign mix or the voice-over pipeline, while Chase [Ashbaker] managed characters and combat. Morgan [Green] was our primary sound designer for cinematics, Brennan [Anderson] owned UI and systems while Phil [Lamperski] created the combat music and warzone ambient systems.

While these assignments allowed the individuals to become experts within their own categories, we also scheduled several rounds of “focus weeks” where the entire department would work together on a single category to catch-up or wrap-up remaining work. This combined method created a lot of momentum for the team and kept us all engaged in each other’s work and the overall product.

 

What were some challenges and solutions you had for getting the most mileage out of your sounds and staying within your allocated audio data budget?

Chase Ashbaker (CA): Much of the real-world common and textural stuff, like rock debris, wood breaks, metal wronks, footsteps, explosions, etc., had to be shared across the board and combined in different ways to make things feel unique.

However, using this base of shared materials and environmental elements left us room to have more unique, one-off source content for our Heroes’ special abilities, super-powers, equipment, and weaponry.
 

There are 81 years of Marvel history to dig through for this game. In terms of sound, how has this helped to inspire or guide your sonic choices and palettes of sounds?

JG: The fact that the Avengers have been featured in so many forms of entertainment over so many years gave us a lot to look into — and a lot to live up to.

… we had the opportunity to design from scratch.

Since the original comics can only feature sound in the reader’s imagination, and because our game isn’t a movie or TV series tie-in, we had the opportunity to design from scratch.

With Marvel’s creative support, our entire game — from the story to character models to music themes to sound design — is unique to this title. It has been a true honor to add our sound design contribution to these beloved characters and to become part of that 81-year plus history.

 

Were you still free to take creative license with character sounds or did you have to adhere to the established palette of sounds for each character?

JG: We based our character sound design approach first on supporting the visuals or mechanics in the gameplay. That said, we created a variation on Iron Man’s iconic “camera whine” sound for charging his repulsors. This is notable because it doesn’t have any visual component in the game (or movies) but during development, it felt missing without it.

This was also our time to add to that pantheon of innovative sound design…

CA: While we were not required to use pre-existing sounds, it was a pleasure to listen to, pull ideas from, and pay homage to the years of incredible sound design that have been used in movies, television shows, and games over the past couple decades. This was also our time to add to that pantheon of innovative sound design, and to create recognizable content of our own to help shape our Marvel’s Avengers gaming universe.

 

---

Popular on A Sound Effect right now - article continues below:

---

There are six playable characters: Black Widow, Capt. America, Hulk, Iron Man, Ms. Marvel, and Thor. Did you delegate the character sounds to different sound designers? If so, who got what character? And did they get to choose the character they wanted to design, or was a character assigned based on a designer’s strengths?

CA: The sound design for each of the core six was a shared labor of love throughout the development process. We did lean towards designer strengths for the bulk of the work, but everyone contributed something towards one or more of the heroes.

…everyone contributed something towards one or more of the heroes.

I spent a lot of time with Thor and Hulk’s combat design. Jack’s early work on Cap’s shield was a good foundation that didn’t change much to the shipped product. Phil did a lot of work on Kamala’s embiggened state. Brennan set up most of the infrastructure and crafted the Foley work for nearly all of the Heroes. Morgan’s design style was a perfect match for much of the high-tech weaponry and gadgets for both Iron Man and Black Widow.

It was very important that the team as a whole felt some level of ownership with these great characters.

 

What were some unique sounds created for each character? What went into creating them?

CA: We spent about a month working on sound design for Thor’s hammer, Mjölnir. At first, we tried recording some objects that we already owned such as a cast-iron skillet or a trailer hitch ball mount, but none of them had a deep resounding tone that we wanted. In the end, we picked up a few solid steel bars from a local steel-works place. We recorded a ton of material with those bars, both suspended for long ring outs and muffled work made by dropping and banging them on the ground. The elusive “THUNK!” took a long time to find, but in the end we’re very happy with those melee impacts.

Hulk had a very special vocal design feature that separated him from the other playable characters. Much of his narrative interaction in the story comes in the form of reactive grunts and roars. To reduce the time in the studio with our Hulk actor, Darren DePaul, and eliminate the need for expensive pickup sessions we used emotive categories, like happy, sad, angry, dismissive, annoyed, etc., and used those categories as variations for a collection of different vocalization types, such as grunts, chuffs, growls, snarls, roars, laughs, etc. By doing this we were able to capture a huge amount of non-verbal reactions to match up with just about every narrative event in the script.

…the base layer of his Foley was a piece of scale mail armor that was made for a cosplay costume.

Brennan Anderson (BA): One of my favorite characters to work on was Thor. Specifically, making his Foley is what stands out to me. We got a lot of materials to record to try and find the sound of his armor. We sent out an email to all of the cosplayers within the company to get various armor pieces they had made, such as chainmail and leather armor. The piece that ended up making the base layer of his Foley was a piece of scale mail armor that was made for a cosplay costume. We used a lot of the recordings from the other pieces we received for some of the costumes on other heroes as well, like Hulk’s gladiator outfit, and some of Cap’s sounds.

Morgan Green (MG): I did a lot of Iron Man’s sound design. I loved his technology-forward sound and I must admit, it was pretty surreal to be designing for such an iconic character. One of the first sounds I created for him was his chest beam weapon. At the end of his portion of the Golden Gate Bridge level, he has a short cinematic where he squares off against a pulsar tank, and the chest beam is the exclamation point to the scene and the perfect place to develop its sound.

For the body I created a few bass layers with an old Reaktor community ensemble “Menace: by noisewreck.” It’s a Reese Bass Machine using three SAW oscillators and dual resonant notch filters — a hidden gem and secret ingredient to a handful of major electronic music releases. In order to bring the sound back from the synthetic realm,

It’s a Reese Bass Machine using three SAW oscillators and dual resonant notch filters…

I created one layer processed from a recording of high-pressure spray on metal, and another layer from a session we did with thick metal piano wire recorded moving through various things, which I distorted and turned into a granular loop.

The end result was four layers of looping sounds combined with some attack elements and then pitch enveloped. The sound never quite felt finished, until Brennan added some slick work on his part to modulate the pitch of the beam at run time depending on if you are aiming the beam or staying on one focused point.

The weapons’ build-up and spin down was a combination of a pre-rendered version of the main sound, which was then pitched and layered with recordings of hard drives, gun reloads, and camera mechanics, and some synth sweeteners. It was a great first sound to tackle with Iron Man because a lot of his palette came from ideas and experiments on the chest beam. The rest of his weapons use different hard drive mechanics, gun reload sounds, camera internals processed, layered, reprocessed, and then pitched and chopped and layered into his future tech weaponry.

 

[tweet_box]Making The Mighty Sound of ‘Marvel’s Avengers’ game[/tweet_box]

What about character dialogue? How many lines were recorded for each character?

JG: The line count for each Hero varies, but it averages out to be around 1,500. The “man in the chair” characters like JARVIS, Hank Pym, or Maria Hill have about 2,000 lines, and our villains have about 500. We also have a number of unnamed AIM (Advanced Idea Mechanics), robot, SHIELD, Inhuman, and Resistance characters along with generic civilians.

Altogether, our total line count is around 20,000 lines and growing. Since this is a live game, we are already writing and recording new characters, and expanding our Hero line count for upcoming content releases.

 

There are 13 different language options for ‘full audio.’ How did you handle the localization (in-studio or outsourced?)

JG: We use the localization services of Square Enix Europe and Japan. They provide translations, recordings, mastering, and language-specific user testing. My job has been to provide all the English performances, implement the localized audio files, and fix bugs — as directed by the localization QA team.

 

What were some challenges and solutions for managing so many lines of dialogue for 13 different languages?

JG: The localization challenges for Marvel’s Avengers are again centered on proper time management. The flexibility needed to experiment and create the game includes the narrative, but localization deadlines are often quite rigid, which is understandable given that their coordination efforts are considerably more complicated than our English recording sessions.

…we took advantage of the live aspect of the game and continued to record in regular, small batches.

When we initially announced the release date delay from May to September 2020, we had already finished the majority of our localization recordings. From that point on, we took advantage of the live aspect of the game and continued to record in regular, small batches. Every few months we collected new lines along with revisions or extensions to existing content to reconcile any story changes. This method is now part of our on-going post-launch content plan.

 

There are some fun, robotic voices. Can you talk about the processing chain — or your approach to processing — on some of those robotic voices?

MG: The robots were a lot of fun to create. The idea was to try to represent the evolution of AIM in both technology and motive. We wanted to use a mix of verbal barks as well as nonverbal expressions, so I focused on crafting the texture and tone to tell that story.

We ended up with a total of seven unique robots with dialogue.

I leaned heavily on old drum ‘n’ bass processing techniques: resampling, micro-edits, frequency splitting, and utilizing long chains of insert effects. The creative dilemma was weighing the pay-off of crafting this level of detail versus simplifying by batch processing, and we already knew this would need to be localized in at least ten languages. With the help of some custom macros and Nuendo’s “rename from list” function it was decided, go BIG.

The end results were three major classes of robot VO, each using a different processing technique for its texture and tone, with the sub-families each getting unique attention in the chains for each character. We ended up with a total of seven unique robots with dialogue.
 

---

Highlights from A Sound Effect - article continues below:

---

The Frenzy class of robots were spectral-morphed with a Reese Bass using Melda’s MMorph. To improve intelligibility, I used the Nuendo feature of “veriaudio pitch correction” to flatten out the dialogue assets on a doubled track. The tracks were then comped together and sent through FabFilter Saturn buss compression and finished off with a little Chorus post-fader to add movement and glue.

The Reaper class of robots’ main nastiness comes from N.I. Razor used as a vocoder, adjusted with PitchDriver, then slammed through N.I. Supercharger.

The Adaptoid class of robots’ next-gen shiny sound came from three layers of dialogue using Absynth FX, NI Reflector, and SaltyGrains. A fourth layer was added using the old reverse reverb trick.

 

The characters really destroy the heck out of the environment, especially Hulk! It must have been really fun to create those destruction sounds. Was that covered in Foley? Sourced effects from libraries? Did you capture any custom field recordings of smashing and bashing things?

CA: There’s a fair amount of library content in this project that has been manipulated and layered to cover the sheer volume of breakable objects, but otherwise, we spent as much time as possible recording our own custom content. Let’s just say there’s some office equipment, already destined for the recycle bin, that met with a gruesome end.

BA: I do remember several recording sessions where Chase and I took a bunch of scrap metal and computer parts, a few monitors and keyboards, and smashed them to pieces!

One of my favorite sessions…was when we went into the woods to record destruction sounds for some forest environmental levels.

There was one take where I tied together two of the monitors with rope and started swinging them around, flipping them over, dragging them on the ground, and just smashing them onto the pavement as hard as I could.

One of my favorite sessions though was when we went into the woods to record destruction sounds for some forest environmental levels. We found a bunch of dead tree branches and took turns cracking and breaking them. At one point, I remember just jumping on one and it made the most satisfying crack I had ever heard. Very cathartic!

Phil Lamperski (PL): We recorded interior and exterior car door, trunk, and hood slams from my aggressively bland Nissan Versa, which I used in the design of Iron Man’s punch impacts. It was fun and we got some really great sounds.

 

Looking at the Foley for the cinematics, what were some of your challenges, and also opportunities to get creative there?

Atsushi Suganuma (AS): One of the challenges we faced for working on Cinematics was that we were required to prepare our best for every Foley session with limited materials and pictures. This is a rather common issue for pretty much all of my past AAA titles that I was involved with. For example, we had limited materials to understand what kind of environment the cinematics were taking place in, which sort of prop, costume, and terrain will be used on the final gameplay, and how we should adapt and re-conform to the picture changes. Luckily, Jack and his team were able to fully support my team with updated pictures and the necessary materials to prepare the best for the sessions.

…those issues and limitations really sparked my creative visions for how my team can achieve top-quality sound in cinematic storytelling with our Foley work.

Speaking of opportunities to be creative, actually, those issues and limitations really sparked my creative visions for how my team can achieve top-quality sound in cinematic storytelling with our Foley work. Our best work came from trying to cover as many Foley elements as we could in the cinematics. We really went into a deeper level of detail to create each individual character’s unique sound in order to support storytelling with our Foley work. You can perhaps notice that on Kamala’s subtle cloth and prop movement, Hulk’s huge body movement, Iron Man’s mech movement, Thor’s hammer and body armor movement, Black Widow’s body armor and prop movement, Cap’s shield, his armor and his prop, Tarleton’s mech movement, Monica’s prop and body movement, and so on.

 

Can you talk about your collaboration with Composer Bobby Tahouri? How did you divide this game into manageable chunks of music? (Did you discuss the score in terms of levels? Were there several cues for each level — like an action cue, a suspense cue, an exploration cue, etc.?)

JG: The overall music for the game was evenly split between the needs of campaign and multiplayer. I directed Bobby’s deliveries for overall theme development, campaign and cinematics, while Phil directed the multiplayer content and implementation.

Shaun Escayg (the game’s Creative Director) was very specific about keeping music minimal during crucial story moments

All of the music was delivered in separate stems based on instrumentation, which gave us the flexibility to create a variety of mixes for each piece.

For the story campaign, we based the work loosely on levels or major events. By the time the cinematics came together, we already had most levels scored so we were able to let the existing feel of the levels help define what would sound best for the cinematics. Shaun Escayg (the game’s Creative Director) was very specific about keeping music minimal during crucial story moments so that we could give more weight and responsibility to the actors. This aesthetic provided dynamics in the mix and in the compositions.

PL: I worked closely with Bobby Tahouri on designing the palette of sounds and instruments used to represent the Avengers.

Each Avenger gets their own suite of nine music tracks, for a total of 54 tracks, specifically composed for dynamic combat scenarios that range from low to high intensity.

Each Avenger gets their own suite of nine music tracks…specifically composed for dynamic combat scenarios…

Our goal was to achieve variety and identity while also having the ability to ramp the music intensity depending on the context and intensity of the fight.

We also had Bobby score custom ambient traversal music specific to the geographical locations featured in multiplayer. This blend between location specific and character-specific music gave us a wonderful palette to work with for scoring the player experience.

 

How were you able to get the most from each music cue?

PL: I had the luxury of editing Bobby’s fantastic music, which allowed me to create a wide variety of mixes used all throughout the game for all sorts of different moods and contexts outside of combat.

This system generates percussion on the fly in the style of the composer according to what the character is doing in combat.

In addition to having a great variety of instruments and tracks, Marvel’s Avengers features the novel Dynamic Percussion System developed by Intelligent Music Systems. This system generates percussion on the fly in the style of the composer according to what the character is doing in combat.

For instance, you’ll hear higher-ranged instruments like shakers play fast rhythms while the player is sprinting away from combat, and then lower and slower aggressive drums kicking in after the player has taken damage from an enemy. Or you may hear the percussion drop out when the player jumps.

The Dynamic Percussion System allows for each character’s unique percussion palette to be extracted and divided up into descriptive categories or groups such as Low, Mid, High, Fast, Slow, Metal, etc. These groups are then arranged and mapped to best support the narrative of the combat.

As the player performs heroic superpowers, finishers, combos and more, the percussion is listening and following along to give the player a real feel for the rhythm and dynamics of the specific combat scene.

The system can be subtle at times, especially during big combats, and tends to shine more in the smaller combats. However, having control over the frequency range, energy, and onset of the percussion gave us a unique way to achieve a cleaner mix without having to entirely sacrifice the percussion.

 

In terms of mix, what were some challenges when mixing the single-player campaign? What about mixing the multi-player mode?

JG: The campaign of Marvel’s Avengers centers around the characters and story and so the mix is entirely anchored by voiceover. We had to make sure VO was clear and understandable in all conditions, even when the on-screen action is filled with overwhelming spectacle.

Story progression or character arc VO plays during cinematics, basic traversal and exploration, dramatic set-pieces, and during full-on combat — with or without companions.

This drove us to make choices with asset mastering, dynamic RMS volume ducking, and momentary shifts in how any given sound category reacts to another, targeting changes in both volume and EQ.

Understanding and digging into our mix hierarchy allowed us to subdivide sound categories into a number of smaller categories, each dynamically reacting to each other.

Arriving at an ideal starting place for the mix was a true challenge and, of course, after that we made use of other tools to exponentially increase, completely ignore, or gently push these sound relationships as needed.

PL: Mixing multiplayer mode was very challenging because of the massive amount of activity happening at once: VO, explosions, weapons, four Heroes, enemies, bosses, music, and more!

What is interesting is that if you are playing a 4 player co-op game every person’s mix will be different and tailored to their point of view.

We needed a way to know who the local player was and then reassign at runtime the sounds attached to the local player to several special local-player mixing busses. This allowed us to highlight the local player as the main focus of the mix.

What is interesting is that if you are playing a 4 player co-op game every person’s mix will be different and tailored to their point of view.

For instance, if you are playing as Iron Man, your repulsor blasts will duck companion characters and enemies, and your VO will be routed differently to sound clearer and fuller. But if you are playing as another character then Iron Man’s repulsors will not duck you, in fact, if you are playing as Black Widow and fire your pistols, those sounds will duck Iron Man’s repulsors on your local machine only. All of this is achieved systemically under the hood.

We also deployed a powerful scaling mix system that listens for the amount of enemies present and then scales different mix values accordingly. For instance, as more enemies appear in the vicinity of the player we can gradually reduce radius size, volumes, reverb sends on multiple mix busses to make for a cleaner mix. As the enemy count goes down, the radius sizes, volumes, and reverb sends gradually return back to their normal fuller sizes. All of this coupled with systemic RMS ducking and our sound group structural hierarchy works together to craft an exciting and dynamic mix!

CA: At a higher level, much of the mix was managed with a series of prioritized mix buses. All the unique game sound groups, i.e. Foley, weapons, dialogue, etc., are routed into these groups where we could use RMS levels for the highest priority groups to duck the lowest priority groups. This ensures that higher priority sounds were always heard. This system was used in both campaign and in the multi-player missions.

…when players use their heroic abilities, we temporarily reduce the volume for…non-essential feedback sounds and really boost the heroic sound design elements and Hero VO to showcase that powerful moment in gameplay.

We then utilized a collection of mixstates to dial in volume relationships between sound groups for dramatic effect. For example, when players use their heroic abilities, we temporarily reduce the volume for ambient SFX, enemy content, and other non-essential feedback sounds and really boost the heroic sound design elements and Hero VO to showcase that powerful moment in gameplay.

During those heroics, we also pull back on the combat music mix and push forward on the interactive percussion content so that there’s a feeling of energy shift in the music drive as well.

Some of these mixstates are persistent throughout the entirety of a level, or perhaps intermittently, like during a combat section, or just a few seconds to punch up a combat finisher.

 

Marvel’s Avengers uses Crystal Dynamics’s proprietary Foundation Engine, which had to be updated in order to build this game. Was there an opportunity to improve the audio aspects of the game engine during the update? If so, what improvements did you make?

PL: Yes, as the game evolved improvements became clearer and necessary in order to support the magnitude of it all. Some of the most challenging problems were in the realm of dynamic dialogue processing and mixing. As mentioned above, new tech was added to dynamically and systemically reroute sound groups depending on whether or not you are the local player. This tech had a massive impact on our approach to mixing and our final mix.

This tech had a massive impact on our approach to mixing and our final mix.

There were also a handful of dialogue processing problems that this solution applied to. For instance, Iron Man’s dialogue is processed differently depending on what costume he is in, and whether his mask is up or down. Also, an NPC’s VO will sound clean or as though it is coming over a radio depending on their distance to the local player.

We also made improvements to the Dynamic Percussion System, where it now has stereo playback, an updated authoring tool, and it’s capable of playing many more pieces than before due to dynamic loading tech. Very early on the project, there were improvements to our audio triggers and ambience systems that streamlined our implementations and saved us a ton of time.

 

What are you most proud of in terms of sound on Marvel’s Avengers?

JG: There are several areas of the game that I’m proud of, but “Testing 1, 2, 3…”, or the space level is my favorite. It’s a dynamic combination of bombastic action and mystery. From start to finish it’s a really satisfying story moment and there is a lot of unique sound design work there.

CA: Oddly enough, some of the enemy sound design is my favorite. The Prime Synthoids with the large staff weapons and Adaptoids have some really cool stuff going on. There is so much attention and great content for the heroes, but the immense variety between the different enemy types, the different elemental powers they wield, and then the different weapon sets they use are really impressive. That is if you can hold off for a few moments before smashing them to bits.

PL: I am most proud of the Golden Gate Bridge level, because I think that it best highlights and captures how the audio team’s collective efforts worked together to create something very impactful, fun, and memorable.

MG: It’s hard to pin down one. But I agree with Phil. I’d stretch it to the whole intro from the A-Day celebration through the end of Golden Gate Bridge. It is just so full of life, story, and cool sound-designed moments that shine and the contrast of the fair with the complete destruction of the bridge. Included in that is Thor’s Intro during A-Day, which came out as one of my favorite sounding scenes. The other being when he is reintroduced much later in the campaign.

BA: I would have to say that I am proud of the systemic work we were able to do in automating the Foley system. We were able to use some of our tools to automatically trigger Foley in many cases instead of having to tag each animation by hand. This saved so much time that let us focus on other things. Fine-tuning that system to work in conjunction with the outfit sound system we put together was incredibly satisfying, and really helped bring the characters to life.

AS: From the Square-Enix Sound team side, I feel truly proud to be able to check out the released game cinematics for which our Foley work ended up becoming one of the major sound elements of the storytelling, especially in those drama scenes. Also, a big shout out to Jack and his team on achieving top-quality in-game mixing; it gives the users a well-crafted gameplay experience. I particularly love the contrast of the big intense action sound in combat, adventuring, and exploring with the very unique superpower sound and very natural, subtle, and dynamic soundscape in the cinematics.

 

A big thanks to Crystal Dynamics audio director Jack Grillo, audio lead Chase Ashbaker, senior sound designer, Phil Lamperski, sound designer Morgan Green, and sound designer Brennan Anderson and to Square-Enix audio director Atsushi Suganuma for giving us a behind-the-scenes look at the sound of Marvel’s Avengers, and to Jennifer Walden for the interview!

 

Please share this:

 

---