Interview by Jennifer Walden, photos courtesy of EA
Squeezing console-sized sound into a handheld gaming device is like Tommy Boy (Chris Farley) trying to put on Richard (David Spade)’s coat.
Audio Lead Viviana Caro was tasked with shrinking down the sound of EA’s Plants vs. Zombies: Battle for Neighborville to fit the Nintendo Switch — this was EA’s first port to the Switch using Frostbite. She says, “We used the existing content from the previously released game on larger platforms (PC, Xbox and PS4) and made it work for the Switch which has less audio capabilities.”
Caro and her compact team — a lead audio coder and an audio artist — had to find ways to optimize the game to make the audio work while maintaining the quality and feel of the game.
And since Caro contributed to the earlier release of PvZ: Battle for Neighborville — working as a senior audio artist who handled sound design and implementation for the ambiances, world sounds, and some gameplay content — she knew how far she could push the sound for the Switch without comprising the soundtrack’s integrity.
Here, Caro talks about the challenges and solutions to trimming down the sound of PvZ: Battle for Neighborville, plus shares expert tips on creating ambiences, and gives advice on how to get started in the game audio industry.
Plants vs. Zombies: Battle for Neighborville Complete Edition – Reveal Trailer – Nintendo Switch
As audio lead on Plants vs. Zombies: Battle for Neighborville‘s port to Nintendo’s Switch, what did your responsibilities entail?
Viviana Caro (VC): This was my first project as a lead. Working with the Plants vs. Zombies franchise was an exceptional opportunity. I really enjoyed combining my skills in doing creative sound design and ambiences, technical audio implementation, and project management for the game.
Audio Lead Viviana Caro
Our main goal was to fit the audio from the larger platforms into the limited memory and space requirements of the Nintendo Switch console while still preserving quality and performance. It was the first game that used Electronic Arts’ proprietary Frostbite game engine technology on the Switch so we knew it was going to be a challenging project. I had to make many creative and technical decisions that balanced the quality of the audio with the limited amount of time we had available to work and the hardware capabilities of the Switch.
I worked closely with the development team, primarily the audio software engineer, as there were a lot of unknowns throughout the project. We were continually discovering and solving issues as we were progressing through the project.
I also provided management support and guidance to the audio artist helping in the project, was involved in task planning, identifying opportunities to do further audio optimization, identifying and fixing bugs, writing documentation, and overall, making sure we finished the product on time, on budget, and with the high level of polish required for the game. This was all at the same time as making the transition to working from home during the pandemic, thankfully PvZ is such an inspiring title to work on.
The game was originally developed for Microsoft Windows, PlayStation 4, and Xbox One. That must have been a real challenge in adapting this game to the Switch…
VC: We needed to find a way to preserve the sonic experience from the original version of the game without compromising the richness of the different audio elements even though we had less than half the memory and CPU resources available on the Switch when compared to the initial launch version of the game. We needed to squeeze all the audio details that were important for gameplay for the characters, weapon layers, voice variations, musical elements, etc., in a way that was transparent in the final result.
…we had less than half the memory and CPU resources available on the Switch when compared to the initial launch version of the game.
We spent a lot of time listening to the game, identifying key sonic elements that needed to be preserved, and categorizing every single sound into a priority level so that we could quickly change the quality threshold at which elements would be played. Making the priority level for all the audio facilitated our decision-making when we had to make sound reductions.
We used the weekly team playtests to do critical listening and to evaluate our next steps on the game optimization. We repeated this process and continuously iterated through all the features of the game many times throughout the project. It was like doing sonic strokes on a canvas, making sure we had the right balance of sounds at all given moments.
How did you optimize the game audio for the smaller Switch platform, while maintaining the quality and feel of the game? Can you walk us through your approach?
VC: When the project started, audio was using 200% of its allotted CPU budget, so we had to find ways to make it work on a reduced capacity. We also had to reduce the audio data size, since the entire game had to fit in an 8GB cartridge, this was around a 50% reduction from the specifications from the previous platforms like the PS4.
We started with the big optimization wins, such as determining proper audio codecs, adjusting individual data compression and quality settings for our audio assets, moving our reverb implementation into the Switch hardware reverb, reducing our number of impulse response (IR) channels, and reducing our output channel count from 7.1 to 5.1.
We spend a good chunk of time focused on optimizing the weapons patches as they’re a huge part of the first-person shooter experience…
As the project moved along, we focused on identifying and optimizing expensive audio patches within the Frostbite engine, costly game locations and assets, while always making sure we were preserving quality and performance for the overall gameplay experience. We spend a good chunk of time focused on optimizing the weapons patches as they’re a huge part of the first-person shooter experience, by removing expensive digital signal processing (DSP) effects such as flangers and filters, as well as simplifying the visual scripting audio patches in Frostbite by removing unnecessary layers, collating similar nodes together, and testing the resulting CPU overhead in the debug output while running the game.
…we adopted the practice of fixing issues as we encountered them rather than leaving them to haunt us later.
Once we had the technical audio resources fit the budget we were allotted for the Switch, we shifted our attention onto the overall audio quality, doing a polish pass for all the levels, reviewing all weapons, listening to the overall mix, and making changes based on our qualitative analysis through in-depth listening.
Finally, we focused on fixing bugs, which were not as many as we initially thought, since we adopted the practice of fixing issues as we encountered them rather than leaving them to haunt us later.
Are there discernable sonic differences between the Switch version and the PC version, let’s say? How do those experiences compare, sonically?
VC: Personally, I do think there are sonic differences between both versions, mainly because the Switch version is a “simplified” version of the original console game. To me, it actually feels less crowded, since there are less sonic elements playing at the same time. We also simply had more time to polish the audio overall so I feel the additional care and attention put into the mix helps as well.
I spent time tuning the low dynamic range version of the mix, focusing on how it sounds through the Switch speakers, by adjusting the compressor settings and making tests to measure game levels on both settings. A common issue with handheld platforms is that the quiet sounds play too low, and the loud sounds play too loud, so finding the right balance of bringing up some of the low sounds, but not too much, so they sound balanced with the loud sounds, isn’t an easy task.
A common issue with handheld platforms is that the quiet sounds play too low, and the loud sounds play too loud…
During combat, the mix dynamically adjusts so that you hear more the weapons and details of the characters close to you while keeping the weapons of characters further away more in the background. That said, we also kept in mind specific important instances where weapons that can be triggered at long distances can be heard, so the player doesn’t lose the sonic warning if they are in danger of being shot. Being able to patch things in Frostbite using visual coding gave me a large amount of control and autonomy over the audio. That helps set the audio of the game apart from other games that don’t have as much ease over the procedural control over the audio.
On the Switch, you won’t hear as much detail as the console in regards to ambiances and world sounds. Most of the world animations such as moving machines, fans, bubbles, etc., were adjusted for the Switch, and the number of ambiance sounds needed to be simplified. You will hear enough to make you feel a particular location, but it will certainly feel more sparse than the console versions of the game.
Being able to patch things in Frostbite using visual coding gave me a large amount of control and autonomy over the audio.
For a handheld device, we needed to work within the capabilities of the machine, and I find it has the right amount of sonic elements in-game. I worked hard with the team to make the audio feel balanced and enjoyable when playing.
Popular on A Sound Effect right now - article continues below:
-
-
-
20 %OFF -
23 %OFF -
17 %OFF
This is the first time that EA’s Frostbite engine has been used for a Nintendo console. What were the creative and technical hurdles this posed for the smaller Switch platform?
VC: We had a smaller team working on this feature, and a year to complete it. As noted, we were given a big challenge to transition to work from home due to the pandemic and still hit our release date.
For this version of the game, we wanted to give players the ability to play on the go, since handheld is the main feature for the Switch. We also focused on mixing the game, providing two different dynamic range versions to choose from: High dynamic range mix for players using headphones or speakers, and Low dynamic range for players using the built-in speakers on the Switch.
Overall, we fit around 95% of the original game into 30% of the hardware power of the Switch.
Overall, we fit around 95% of the original game into 30% of the hardware power of the Switch. Of course this is a big challenge and I’m happy that I was supported by such an amazing team.
We found optimizations that worked well in handheld, while maintaining a good resolution, also adding the options for motion control and touch screen inputs in conjunction with being able to play both online and offline. Since you can play online, this can create a difficulty of knowing exactly what might happen so that you can still have a great mix, so we worked a lot with the quality assurance (QA) team to give feedback for different online scenarios and adjusted the mix accordingly.
Before working on the Switch version of the game, you were also a senior audio artist on the earlier PvZ: Battle for Neighborville release for the PC/Xbox One/PS4. What did your responsibilities on that release entail?
VC: I was responsible for the ambience content creation and implementation for all 14 levels of the game, including world sounds (ambiance sounds you don’t see such as birds and crickets), emitters such as waterfalls, and gameplay content for the quests or missions within the levels.
As far as examples of quests, there is the ‘Treasure Hunt’ quest, where you have to find map pieces, or the ‘I Treasure You’ quest, where you have to save your plant friends from dancing forever inside a boogie trap. These were really enjoyable to create content for since they had such fun themes.
I also helped establish a levels workflow with the other departments, to help improve information flow throughout the team with audio.
As the ambiance point of contact, I was in close communication with the development team. I attended all the level design meetings, brought up possible situations in which audio might be blocked (when we couldn’t continue to work on something), and design questions that will impact audio. I also helped establish a levels workflow with the other departments, to help improve information flow throughout the team with audio. When new audio artists were hired on the team, I also provided Frostbite training to help get them started with the basics of visual scripting.
I designed and implemented 14 levels and 12 Social Hub seasons for each time of the month, including Halloween, Christmas, and Valentine’s day, added 80 new world sounds, and 250 emitters.
GameSoundCon, after Talk Virtual chat with Attendees
At last year’s Game Sound Con, you presented details of your work on game ambiences. Can you share some tips on creating effective ambiences for games?
VC: There are many ways to approach ambiance design and implementation in video games. It really depends on the type of game, console, game engine, time, resources, and so on. I can give some general tips though, which are based on my personal approach when working on PvZ: Battle for Neighborville:
• Before getting started, gather information on specific traits, concept and style of the game. You can look at level briefs, concept art, narrative documents, etc. This information will help you develop a sound palette for your ambiances.
• Think about your audience. Who is going to play this game? What is your audience?
• Design an implementation prototype in your game engine, test it, and make sure you are aware of all the technical capabilities and restrictions you will be working with.
• Maintain regular communication with the dev team, to see their progress on building the levels. Share your progress, raise concerns, make suggestions, request implementation or design changes.
• Make iterations based on levels changes and ask for feedback.
• Make areas sonically different to give them their own personality. Add or remove sounds based on how busy is the area.
• Make sure to leave enough time to do a polish pass.
• Use a digital audio recorder to have a personal ambiance approach — when you record your own sounds, the act of recording your sound makes it unique. The creative decisions you make while recording the sound gives it personality.
• Have fun! Enjoying your work translates into the work that you do.
[tweet_box]Transplanting the Sound of ‘Plants Vs. Zombies: Battle For Neighborville'[/tweet_box]
Why are ambiences an important audio asset for PvZ: Battle for Neighborville?
VC: On this game, ambience sounds are everything else that is not directly related with gameplay. They are divided in two big groups: emitters (you can see the object associated with the sound, like animated props, moving platforms, sparks, water, etc.), and world sounds (just hear it but don’t see it, like birds, weather elements or insects).
Ambiences bring the game to life, making each level sonically unique. Each level is designed with a specific personality, to make the player feel like they’re in a different place, and at the same time all the levels keep a similar feel so that they stay within the game world.
Ambiances are my favorite part of the game to work on since I believe that building soundscapes help create a sense of immersion.
My sound palette was oriented toward realistic sounds with a cartoony twist: strange creature sounds in the woods and jungle, rivers made out of cheese, Halloween pumpkins laughing…
Ambiences are meant to be very subtle, not designed to stand out, so they don’t overpower other game elements. In quiet areas where the player is exploring, I added more detailed elements into the ambiences, since it was an opportunity to have ambiences take more of the foreground of the mix.
Ambiances are my favorite part of the game to work on since I believe that building soundscapes help create a sense of immersion. They provide an emotional element as well, and help the location have a specific feel. They also help keep things together, it is the glue for the game transitions and the quiet moments.
Using physics raycasts to calculate information about the character’s position in the world
On the technical side, working on ambiances is really interesting. I work towards finding a natural balance on how and when the sounds react in combinations and transitioning from one to the next. By tuning levels and attenuation curves, you can hear the ambiance elements work together at different distances. Using reverb helps to make spaces feel bigger or very small.
We used physics raycasts to calculate information about the character’s position in the world (how indoor or outdoor is the player), and with that information we added generic ambience beds and IR reverb that crossfaded between them when needed. This was our base ambience audio layer and we started building all the other elements out of this.
I see ambiances as being similar to the texture that the painting goes on, is it an oil painting on canvas, or like spray paint on a concrete wall? The ambiences help to frame and contextualize the audio feel of the game world just like in the real world.
What was your favorite ambience that you created for PvZ: Battle for Neighborville? What went into it?
VC: It was fun to create 100% original content for the ‘Tourist Trap Island,’ by recording my own sounds and processing them to make them sound cartoony. They have a bit of silliness behind them, but they also have realistic content too.
When you record your own sounds, the act of recording your sound makes it unique. The creative decisions you make while recording the sound helps to give it personality. One of the reasons I like this tropical island ambiance is just because of that.
When you record your own sounds, the act of recording your sound makes it unique.
When I listen to the island sounds, it takes me back to the moment I was recording those sounds. A broken oven clock, a silly voice… those elements processed and combined together are the ingredients for the sounds on this weird tropical island where zombies are building a storm machine. You hear odd monkeys, small creatures hiding behind the vegetation, strange insects, all surrounded by the ocean.
The island is not playing the same sounds in all locations. As you progress through the island, you see more zombie constructions and vegetation is more sparse. You hear fewer creatures, and more weather elements such as wind and rain, the closer you get to the storm machine.
Audio helps to immerse and carry you through the game experience and I was really able to get creative with PvZ as it’s such a fun and creative title.
Caro recording ambience at a Bird Sanctuary
What advice would you give other sound pros or students looking to get into the game audio industry?
VC: My advice would be:
• Learn, practice and explore as much as you can. There are plenty of game audio educational resources available online.
• Give yourself small realistic goals, and make sure you accomplish them
• Connect with other professionals in the industry, attend conferences and do networking.
• Find a mentor
• Always have your resume and portfolio up to date
• Apply for jobs you are interested in, like I did when I applied to work at EA. You never know what’s possible until you give things a try!
Thank you for the interview! I’m always happy to connect with people interested in game audio and the best way to reach me is at my website at VivianaCaro.com.
A big thanks to Viviana Caro for giving us a behind-the-scenes look at the sound of Plants vs. Zombies: Battle for Neighborville and to Jennifer Walden for the interview!
Please share this:
-
-
15 %OFF -
-
-
-
20 %OFF
-
23 %OFF
-
17 %OFF