Game Audio Pipeline Asbjoern Andersen


Whether you're working on large or small game projects, getting your game audio pipeline right is essential for a smooth-running development process - and to help you do that, Mads Vesterager Riddersholm and Jeppe Emil Lindskov have written this useful guide for you:
Written by Mads Vesterager Riddersholm and Jeppe Emil Lindskov
Please share:
 

Motivation

In the last year of our bachelor’s degree, we were both working full time at different game developers in Copenhagen. We started as interns, but we were responsible for most/all audio-related work at these studios. Without previous experience working in the industry, we quickly realized the importance of pipeline development. If you are not thinking carefully about your workflow, how you organize your project, and what tools you need, it will slow progress and cause problems during production. Even in small studios working on small games.

If you are not thinking carefully about your workflow, how you organize your project, and what tools you need, it will slow progress and cause problems during production.

But how should we learn the forbidden arts of game audio pipeline development? The College we attended didn’t provide us with much knowledge on the subject, and while a lot of available information can be found on how to make sound design for video games, not much can be found on the subject of creating an efficient workflow and pipelines.

After a lot of research on the subject, we decided that we wanted to write our own guide on game audio pipeline development. We then managed to get in contact with some of the best and most experienced audio directors and audio professionals in the industry. The thesis is based on the information they provided us, combined with our own experience and curiosity.

In this article, we want to highlight parts of the thesis that we believe can help new people in the industry as well as those trying to find their way into it.

 

Game Audio Pipeline

But what is a pipeline?

In its most basic form, a pipeline is a chain of processes. The chain is arranged so that the output of each element is fed straight into the input of the next. Much like a real physical transport pipeline.

So what does a game audio pipeline look like? The most basic form of game audio pipeline starts with a concept and stops when that concept can be heard by the player in-game. It could look like this:

GameAudioPipeline-1

This model is very simplified. But the fundamental idea is: Create a pipeline which is as frictionless as possible. In return, the pipeline will provide the sound designer with the best possible creative environment — from asset creation, to implementation, to iteration, to finished product.
 

Pre-production

The production of a video game starts with pre-production. This is the initial phase where the team will come together and scope, plan, and conceptualize the project. This is done to prove out pipelines, team dynamics and metrics.

This is also the phase in which you want to include the audio department and start developing a game audio pipeline.

Pre-production usually ends with a demo or prototype of the game. This is also the phase in which you want to include the audio department and start developing a game audio pipeline.

Unfortunately, there is a tendency at some game developers to think of audio as part of post-production. This might be because of economy, lack of experience, or an old echo from film production. No matter the reason, excluding audio from pre-production is going to have consequences for the production and the final product.

There are three major reasons to include the sound designers from day one:

Having the audio pipeline thought through, laid out, and proven at the beginning of a project provides higher planning stability and a more reliable schedule without last-minute surprises.

1. Having sound designers join early makes it possible to define an audio identity for the game. An audio identity is usually designed on multiple factors, like the vision of the game, the concept art and genre. This also goes the other way around. If the concept artist has access to ambience tracks and/or music it will help them build a stronger and more thorough visual identity for the game. Worst-case scenario is the audio identity being defined by somebody other than the audio department. This is likely to create problems down the road.

2. When having sound designers join the pre-production, they are not only there to talk audio. Being an audio specialist, working with video games makes you an expert in making games. Not only are you developing games, you are also passionate about games. Your opinion about anything in a feature is valid. Every person in pre-production is there to understand and reinforce the vision and to share their experiences for the sake of the game.

3. Creating a pipeline is a planning process. Deciding on every aspect as early as possible is going to save time and make the process of creating a video game more transparent for everybody involved. Having the audio pipeline thought through, laid out, and proven at the beginning of a project provides higher planning stability and a more reliable schedule without last-minute surprises. It also gives audio design and audio technology the time to build a trustworthy collaboration.
 

Game audio tools

A very important technical aspect of creating a good pipeline is the creation of tools for the sound designer to use. Tools do not necessarily have to belong in the game engine. Many tools are developed for other parts of the game audio pipeline, like exporting and importing tools between DAW and Middleware.

The main point is to facilitate the flow from what is in the sound designers head all the way to what the player experiences in the game.

In a way, a tool can be seen as the thing in a pipeline that makes it run smoothly from concept to implementation. The main point is to facilitate the flow from what is in the sound designers head all the way to what the player experiences in the game.

The amount and types of tools needed are different, depending on the game and the studio. Tools can be shared across multiple projects, especially if the same Middleware and game engine is being used. But it is important to constantly review and evaluate which tools are needed and how to improve on existing tools. The idea is to always be aware of what works and what doesn’t. If the need for a new tool arises, it gets added to the technical backlog and prioritized depending on how urgent it is. This way a company can be aware of what problems lie ahead and be able to react in time.


Popular on A Sound Effect right now - article continues below:


Trending right now:

  • Cinematic & Trailer Sound Effects Abandoned Oil Tank Play Track 327 sounds included $40

    On a recording trip to the smaller Danish port town Struer, I came across an abandoned empty oil tank. Inside I found two giant wires attached to the floor and the ceiling of the tank, almost like a huge 2 stringed double bass. I’m telling you the reverb in there was longer than the Eiffel Tower on a cloudless day !

    On the outside this was just an empty abandoned building, but it turned out it was not empty at all because it was full of sound and a few dead birds. I’m glad I was curious enough to go inside for a closer investigation and not just pass by.

    After several hours of concentrated recording, it was nice to get my ears back out in dry air. I’m happy there are no open sends to Valhalla in real life but experiencing the eternal resonance on the low frequencies from the wires was absolutely amazing.

    Abandoned Oil Tank was recorded with a LOM basic Ucho stereo pair, a Zoom H6 and the LOM Geofon. All files are in 96 kHz/24 bit and contain the original recordings.

    The result is a unique sound effect library including balloon pop impulse responses from a truly unique location that very recently became even more unique since word says that the oil tank is no more and has abandoned this world.

    327 individual sounds – all tagged with Universal Category System (UCS) metadata.

    www.katrineamsler.com

    20 %
    OFF
    Ends 1713131999
  • Door Sound Effects Noisy Doors Play Track 44 sounds included $10

    This library contains recordings of eleven different types of doors specifically chosen for their interesting sound. I performed all actions with a lot of detail covering “open/close”, “knocks”, “bangs and thuds”, “handle grab and rattle” and when possible also “crackles and squeaks” (all recorded at various intensities)

    50 %
    OFF
    Ends 1711666799
  • Kawaii UI Trailer
  • Foley Sound Effects Pickups and Small Item Sounds Play Track 280 sounds included, 7 mins total $4.16

    Great Audio is Essential

    Essential Audio Pickups and Small Item Sounds contains 280 Foley sounds designed specifically for Video Game Sound Designers who need the usual Pick-Up sounds and SFX for common small items, including: Keys, Coins, Access Card, Pills/Health, Latches, Locks, Chains, Bags, and many many more!

    Features:

    •  280 sounds
    • Bags
    • Coins
    • Keys
    • Pill Bottle
    • Cable Ties
    • Camera
    • Lighter
    • Access Card
    • Chains
    • Locks
    • Clips and Clasps
    • Computer Mouse
    • Boxes
    • Papers
    • Clicks and Ticks
    • Tools
    • Scissors
    • Wood Game Pieces
    • Zippers

    Number of Audio Waves: 280

    Sample rate / bit rate: 44,100 Hz / 16 bit

    75 %
    OFF

Latest releases:

  • Birdsong from the countryside of the Swedish rural region of Värmland.

  • The physical version of Small Rantonen Kantele is one of the oldest, if not the oldest, still playable 5-string kanteles in the world, as it was made in 1770. The kantele was made famous by kantele player Antti Rantonen, who owned it and played it during his career.

    NOTE: FULL Kontakt 6.7.0 or higher required (NOT Kontakt Player)

    Kantele, also called kannel, is a traditional Finnish and Karelian plucked string instrument. Similar instruments include the Estonian kannel, Latvian kokles, Lithuanian kanklės, and Russian gusli.

    Main idea for the virtual kantele was to create an instrument you can play live, almost like a real kantele. We used pedals, velocity layers, and special keyswitches to create a playing experience closer to the real instrument. Strumming is a crucial part of playing the small kantele, so we kept that in the middle of the development from start to finish.

    Samples were recorded using different parts of the kantele, with resonance and without resonance, to give more options for the sound of the instrument. During the sampling the strings were tuned using a diatonic scale (CDEFG). With the virtual instrument it is possible to tune every string independently, so you can achieve the traditional D-tuning and many other tunings.

    We captured long, damped, and dead notes, and harmonics. In addition we recorded percussion with the strings and the body of the instrument. Many other delicious samples are also present, including the sounds of hand damping the strings!

    Below you can view the walkthrough video for the instrument.

  • The physical version of Large Rantonen Kantele is an old, large 30-string concert kantele made in 1902. The kantele was made famous by kantele player Antti Rantonen, who owned it and played it during his career.

    NOTE: FULL Kontakt 6.7.0 or higher required (NOT Kontakt Player)

    Kantele, also called kannel, is a traditional Finnish and Karelian plucked string instrument. Similar instruments include the Estonian kannel, Latvian kokles, Lithuanian kanklės, and Russian gusli.

    Main idea for the virtual kantele was to create an instrument you can play live, almost like a real kantele. As both hands are on the strings when playing a physical kantele, we used pedals, velocity layers, and special keyswitches to create a playing experience closer to the real instrument.

    We recorded long notes, damped notes, harmonics, and mallet hits. For the virtual instrument, we expanded these notes to cover the missing sharps and flats, and even more lower and higher notes. We also added a virtual damping board, two play modes, and a large number of options to help you to create your own sound and playing style.

    Below you can view the walkthrough video for the instrument.

  • Door Sound Effects Noisy Doors Play Track 44 sounds included $10

    This library contains recordings of eleven different types of doors specifically chosen for their interesting sound. I performed all actions with a lot of detail covering “open/close”, “knocks”, “bangs and thuds”, “handle grab and rattle” and when possible also “crackles and squeaks” (all recorded at various intensities)

    50 %
    OFF
    Ends 1711666799
  • Human Sound Effects Zody Play Track 1243+ sounds included, 151 mins total $55

    With abounding character and a penchant for speaking their mind, these little folk always have a lot to say. Wonder and amazement naturally follows.

    Having the honor to be able to observe these tikes in their natural habitats, it became clear how babies share a great deal across their mannerisms as the grow. A longitudinal family of vocalizations and sounds that span the first year of life and maturity, the heart of this library, centers on those personalities and mannerisms changing and taking shape over that period of time. The little ones have been recorded at infancy, 3 months, 6 months, 9 months, and 1 year old. Each of these developmental milestones mark a significant step forward for all babies and at these stages their personalities and expressiveness grow remarkably. It is the constantly evolving language of exploration and examination; always learning. Each vocalization in this library, even within the same stage of development, is unique. From giggles to shouts, questioning to cries, babbles, cooing, and inventive phrases all the way around.

    Additionally, strollers, toys, diaper bags, bottles, pumps, swings, and more have been captured as compliments to represent all of those ubiquitous items in a baby’s world. The strollers were profiled on different surfaces and from different mechanical perspectives. The toys were recorded both in-play and as Foley. Bottles and diapers were used in their normal function and fashion, while other tools and devices of the dedicated and savvy parent were selected for their sound and utility.

    Joyfulness is easy to hear in this library.

Need specific sound effects? Try a search below:


Through our interviews we have found some common guidelines that a good tool should adhere to:

1) The tool should try and ease one or several pain points in the pipeline.

This could be reducing the complexity of a task or reducing the number of clicks needed to perform an action. The first task when designing a tool is to sit down and ask yourself, “What are you trying to archive?”

If the creation of the tool requires help from a programmer, it is essential to focus on the problem you are trying to solve, not necessarily the solution you have in mind.

2) The tool should be able to speak the language of the user and help the user fulfill their creative potential.

It is important that tools are easily understood by the people using them, e.g. making sure UI/UX is intuitive and that features are named accordingly to what they do. Otherwise, it might cause user frustration.

3) The tool should be expandable and be able to be constantly iterated on.

The tool should likewise be granular and easy to update and maintain. Tools will often times be developed early in production, so they need to be flexible, be able to evolve and embrace potential problems that might arise later in the pipeline.

Below we included a playlist of all the tools we made for out bachelor thesis:



Wwise Unity Tools - Sound Painter


Wwise Unity Tools – Sound Painter

 

Organization, transparency and naming convention

When developing your pipeline, it is important to plan out how you want to organize your project from day one. The areas that we have covered so far are all elements of this organization.

Another very important part of pipeline development is to make the workflow as frictionless as possible. An important part of making the pipeline frictionless is to make the pipeline transparent.

An important part of making the pipeline frictionless is to make the pipeline transparent.

To make the pipeline transparent, first of all you need to organize and come to an agreement with both the audio team and everybody else involved in the process of the game audio pipeline on how the pipeline is going to be executed.

This involves documentation, creating a consistent naming convention (which we will talk about shortly) — what format should different types of assets be exported as, what is the hierarchy structure in the Middleware, folders, engine, etc., and what tools are available, what should be built, and how should they be used and integrated into the pipelines of other departments.

Gustav Rathsman from DICE uses the term “One True Source,” which is a great way of describing this need for coherence and transparency throughout a project. He says, “That is a classic problem, where people have different sources of truth and then you end up with something that doesn’t align in the end.”

The keyword here is communication. Communication is absolutely essential to keeping everyone in the loop on what is happening. Having all of the above-mentioned steps laid out as early as possible makes the project and the pipeline way more transparent. It makes it much easier to adapt to changes and when something breaks (and something will break), having a transparent project is going to make the problem solving much easier.

‘Naming conventions’ is a funny buzzword in audio production and game development. Having a naming convention and sticking to it all the way through production is very important. There is no golden solution when deciding upon a naming convention. But a good naming convention should follow these guidelines:

1. Make it very clear what the asset or file is and what it is connected to.

2. Have it appear in sorted order, for easy search and browsing.

An example could look like this:
Category_Subcategory_Name_Layer_VariationNumber

Using this convention in your Middleware hierarchy and/or folder structure is also very beneficial, as you can use it to easily navigate and find the path of certain assets when looking in the profiler.

We also put together a nifty set of tools that you can use to easily name and import assets from Reaper into Wwise:



Reaper Export Workflow


Reaper Export Workflow

 

Conclusion

It is important to emphasize the fact that first of all, there is no right way of building a pipeline. It is rather a question of making the decision on how to do it.

Secondly, there is much more going into the development of a pipeline then what we have been able to present in this article and in our thesis.

When developing a pipeline that is going to stay organized, transparent, and stick to the idea of One True Source, having audio be part of pre-production is vital. It is impossible to prepare for every situation, but having outlined and tested the pipeline is going to provide higher planning stability.

There is no universal answer to a perfect pipeline. A pipeline is developed through experience, trial and error.

That being said, being part of pre-production is also a question of acknowledging everybody working in the game industry as being experts in video games. The knowledge they bring to the production is more than just their field of expertise.

The hardest part of working with a determined pipeline is keeping it consistent if things should change, which inevitably will happen. There is no universal answer to a perfect pipeline. A pipeline is developed through experience, trial and error. That is also why it is important to be aware of how you are working, what steps you need to go through from A to B, and then try to find ways of improving it. Time spend working against the pipeline, is time you could have spent on being creative. In the end, that is what the player experiences.

 

Acknowledgement

We would like to express our sincerest gratitude to Ben Minto, Gustav Rathsman, Guy Somberg, Jonas Breum Jensen, Lydia Andrew, Rob Bridgett and Stephen Hodde. Without your contribution the thesis and this article wouldn’t have been possible.

 

Outro

In this article, we have tried to touch on a couple of important areas of pipeline development. It is impossible to cover all aspects in the length of an article. If you want a bigger insight, please access the thesis via our websites. The thesis also includes a case study using Reaper, Wwise, and Unity, which showcases various workflow and pipeline tools. It might help you contextualize the idea of tool development.

If you have any questions feel free to reach out to us.

Mads Vesterager Riddersholm
https://www.madsriddersholm.com/

https://www.linkedin.com/in/madsriddersholm/

contact@madsriddersholm.com

Jeppe Emil Lindskov
https://www.idosounddesign.com/

https://www.linkedin.com/in/jeppelindskov/

https://twitter.com/idosounddesign

 
A big thanks to Mads Vesterager Riddersholm and Jeppe Emil Lindskov for giving us a look at Game Audio Pipelines!

 

Please share this:


 



 
 
THE WORLD’S EASIEST WAY TO GET INDEPENDENT SOUND EFFECTS:
 
A Sound Effect gives you easy access to an absolutely huge sound effects catalog from a myriad of independent sound creators, all covered by one license agreement - a few highlights:

  • Door Sound Effects Noisy Doors Play Track 44 sounds included $10

    This library contains recordings of eleven different types of doors specifically chosen for their interesting sound. I performed all actions with a lot of detail covering “open/close”, “knocks”, “bangs and thuds”, “handle grab and rattle” and when possible also “crackles and squeaks” (all recorded at various intensities)

    50 %
    OFF
    Ends 1711666799
  • Cinematic & Trailer Sound Effects Abandoned Oil Tank Play Track 327 sounds included $40

    On a recording trip to the smaller Danish port town Struer, I came across an abandoned empty oil tank. Inside I found two giant wires attached to the floor and the ceiling of the tank, almost like a huge 2 stringed double bass. I’m telling you the reverb in there was longer than the Eiffel Tower on a cloudless day !

    On the outside this was just an empty abandoned building, but it turned out it was not empty at all because it was full of sound and a few dead birds. I’m glad I was curious enough to go inside for a closer investigation and not just pass by.

    After several hours of concentrated recording, it was nice to get my ears back out in dry air. I’m happy there are no open sends to Valhalla in real life but experiencing the eternal resonance on the low frequencies from the wires was absolutely amazing.

    Abandoned Oil Tank was recorded with a LOM basic Ucho stereo pair, a Zoom H6 and the LOM Geofon. All files are in 96 kHz/24 bit and contain the original recordings.

    The result is a unique sound effect library including balloon pop impulse responses from a truly unique location that very recently became even more unique since word says that the oil tank is no more and has abandoned this world.

    327 individual sounds – all tagged with Universal Category System (UCS) metadata.

    www.katrineamsler.com

    20 %
    OFF
    Ends 1713131999
  • Magic & Fantasy Sound Effects Magic and Spell Sounds Play Track 682 sounds included, 29 mins total $19

    Magic and Spell Sounds is a custom designed sound library covering many types of magic and spells. From shining in the light to bringing darkness.

    Elements: electricity, energy, fireballs, ice, vines, rock, wind and water. Covering actions such as: blasts, casting, conjuring, impacts, healing and more.

    682 total sounds (422 magic and spell sounds, 260 bonus sounds from Pro Sound Collection).

    ALL sounds from this library Magic and Spell Sounds are included in our full sound library Pro Sound Collection. So if you need more sounds be sure to check it out before purchase.

    24 %
    OFF
Explore the full, unique collection here

Latest sound effects libraries:
 
  • Birdsong from the countryside of the Swedish rural region of Värmland.

  • The physical version of Small Rantonen Kantele is one of the oldest, if not the oldest, still playable 5-string kanteles in the world, as it was made in 1770. The kantele was made famous by kantele player Antti Rantonen, who owned it and played it during his career.

    NOTE: FULL Kontakt 6.7.0 or higher required (NOT Kontakt Player)

    Kantele, also called kannel, is a traditional Finnish and Karelian plucked string instrument. Similar instruments include the Estonian kannel, Latvian kokles, Lithuanian kanklės, and Russian gusli.

    Main idea for the virtual kantele was to create an instrument you can play live, almost like a real kantele. We used pedals, velocity layers, and special keyswitches to create a playing experience closer to the real instrument. Strumming is a crucial part of playing the small kantele, so we kept that in the middle of the development from start to finish.

    Samples were recorded using different parts of the kantele, with resonance and without resonance, to give more options for the sound of the instrument. During the sampling the strings were tuned using a diatonic scale (CDEFG). With the virtual instrument it is possible to tune every string independently, so you can achieve the traditional D-tuning and many other tunings.

    We captured long, damped, and dead notes, and harmonics. In addition we recorded percussion with the strings and the body of the instrument. Many other delicious samples are also present, including the sounds of hand damping the strings!

    Below you can view the walkthrough video for the instrument.

  • The physical version of Large Rantonen Kantele is an old, large 30-string concert kantele made in 1902. The kantele was made famous by kantele player Antti Rantonen, who owned it and played it during his career.

    NOTE: FULL Kontakt 6.7.0 or higher required (NOT Kontakt Player)

    Kantele, also called kannel, is a traditional Finnish and Karelian plucked string instrument. Similar instruments include the Estonian kannel, Latvian kokles, Lithuanian kanklės, and Russian gusli.

    Main idea for the virtual kantele was to create an instrument you can play live, almost like a real kantele. As both hands are on the strings when playing a physical kantele, we used pedals, velocity layers, and special keyswitches to create a playing experience closer to the real instrument.

    We recorded long notes, damped notes, harmonics, and mallet hits. For the virtual instrument, we expanded these notes to cover the missing sharps and flats, and even more lower and higher notes. We also added a virtual damping board, two play modes, and a large number of options to help you to create your own sound and playing style.

    Below you can view the walkthrough video for the instrument.

  • Door Sound Effects Noisy Doors Play Track 44 sounds included $10

    This library contains recordings of eleven different types of doors specifically chosen for their interesting sound. I performed all actions with a lot of detail covering “open/close”, “knocks”, “bangs and thuds”, “handle grab and rattle” and when possible also “crackles and squeaks” (all recorded at various intensities)

    50 %
    OFF
    Ends 1711666799
  • Human Sound Effects Zody Play Track 1243+ sounds included, 151 mins total $55

    With abounding character and a penchant for speaking their mind, these little folk always have a lot to say. Wonder and amazement naturally follows.

    Having the honor to be able to observe these tikes in their natural habitats, it became clear how babies share a great deal across their mannerisms as the grow. A longitudinal family of vocalizations and sounds that span the first year of life and maturity, the heart of this library, centers on those personalities and mannerisms changing and taking shape over that period of time. The little ones have been recorded at infancy, 3 months, 6 months, 9 months, and 1 year old. Each of these developmental milestones mark a significant step forward for all babies and at these stages their personalities and expressiveness grow remarkably. It is the constantly evolving language of exploration and examination; always learning. Each vocalization in this library, even within the same stage of development, is unique. From giggles to shouts, questioning to cries, babbles, cooing, and inventive phrases all the way around.

    Additionally, strollers, toys, diaper bags, bottles, pumps, swings, and more have been captured as compliments to represent all of those ubiquitous items in a baby’s world. The strollers were profiled on different surfaces and from different mechanical perspectives. The toys were recorded both in-play and as Foley. Bottles and diapers were used in their normal function and fashion, while other tools and devices of the dedicated and savvy parent were selected for their sound and utility.

    Joyfulness is easy to hear in this library.


   

Leave a Reply

Your email address will not be published. Required fields are marked *

HTML tags are not allowed.