virtual reality sound Asbjoern Andersen


Alex Riviere has worked with all the major VR platforms - PSVR, Gear VR, Oculus Rift, Google Daydream & HTC Vive - as Audio Director on VR titles EVE Gunjack, and Gunjack 2: End of Shift. And below, he shares his insights on creating audio for VR:
Written by Alex Riviere
Please share:

Lots of my fellow game audio professionals are often asking me the question about the difference working with Virtual Reality compared to flat-screen games except for Binaural positioning. Well, my short answer is that to create an alternate reality that provides a natural listening experience in (gaming) Virtual Environments, our game audio creative approach needs to be adapted, and binaural positioning plays a major role into revisiting it. The long answer is below…
 

Multi-Sensory Integration

With the lack of full-sensory usage in Virtual Reality games (only Vision, Hearing, and Touch through haptics are integrated), the human brain has the power to adapt the focus or the perception of the senses that are integrated. Don’t get me wrong, you won’t become a super-hero with sharpen sense(s) by playing VR games, but the brain can rewire itself through training and learning, to enhance the use of integrated senses, with the goal to use information at its disposal to analyze and interact with the surroundings.

Your smell and taste are not turned off while playing VR games, so those two senses are providing contradictory information between what you see and hear from the game versus what you smell and possibly taste from your physical surrounding environment. Touch (and proprioceptors – sensory receptor which receives stimuli from within the body, especially one that responds to position and movement) would send mixed information between the virtual world and the real world.

The senses that are used need to be reproduced respectfully to immerse the player mentally in the virtual world, combining well-designed and controlled simultaneous visual, auditory, and haptic cues to create a believable Virtual Reality experience. The brain allows to react when subtle sensory signals that might not seems important on their own gets trigger simultaneously, and that’s the power of multisensory integration (MSI).

“A basic tenet of multisensory integration is the ability of one sensory modality to enhance or to suppress information from another sensory modality.”

(Calvert et al., 2004)

A good example would be the “Virtual barber shop” demo released about 10 years ago on Youtube.

Most people experiencing this binaural audio demo for the first time, if mentally immersed (close your eyes while listening to not be distracted by your surroundings, and take a seat), would feel someone is literally behind them touching their scalp, talking to them, and cutting their hairs.
 

Human Evolution & Game Audio

The modern world we’re living in as well as the interactive medias audio language that have been developed for decades have been bringing modern sounds and audio cues to us, and we are constantly analyzing them without even necessarily being conscious of it. Our ears and brain can distinguish thousands of sounds at the same time, being highly informative, sounds are giving us the ability to analyze situations or events and react to it instinctively.

Sounds bring physical and spatial information about objects and environments that our brain can analyze, so we can learn about those sound sources and understand better a situation of objects and events.

In VR, hearing is the only sense able to provide full spatial information going beyond our field of view

Our auditory system provides a lot of information about the world surrounding us. In real life, vision, audition, and our sense of smell provide information that helps us to identify object, situations and navigate in our environment. But in VR, hearing is the only sense able to provide full spatial information going beyond our field of view, including elevation, 360 degrees and depth, allowing us to guide our decisions and behaviors as well as understanding our virtual surroundings.

In other terms, Audio in VR is the only medium able to make you turn your head, grabbing your attention and orientation to possible scripted events, or pre-defined directions and paths.


Popular on A Sound Effect right now - article continues below:


Trending right now:

  • Nature & Countryside Ambiences African Jungle Play Track 80+ sounds included, 460 mins total From: $190

    African Jungle is a huge collection of unique ambiences recorded in Central Africa’s Congo Basin rainforests. To my knowledge, these are the first surround field recordings made in this area. Last year I spent a whole month off the grid, listening to and recording a variety of habitats at all times of day and night, at the height of the rainy season when birds are most vocal. It took me many days of travel to get to these places, from using trains, boats, cars and planes to hiking on foot in the jungle.

  • Foley Sound Effects Pickups and Small Item Sounds Play Track 280 sounds included, 7 mins total $4.16

    Great Audio is Essential

    Essential Audio Pickups and Small Item Sounds contains 280 Foley sounds designed specifically for Video Game Sound Designers who need the usual Pick-Up sounds and SFX for common small items, including: Keys, Coins, Access Card, Pills/Health, Latches, Locks, Chains, Bags, and many many more!

    Features:

    •  280 sounds
    • Bags
    • Coins
    • Keys
    • Pill Bottle
    • Cable Ties
    • Camera
    • Lighter
    • Access Card
    • Chains
    • Locks
    • Clips and Clasps
    • Computer Mouse
    • Boxes
    • Papers
    • Clicks and Ticks
    • Tools
    • Scissors
    • Wood Game Pieces
    • Zippers

    Number of Audio Waves: 280

    Sample rate / bit rate: 44,100 Hz / 16 bit

    75 %
    OFF
  • Game Audio Packs Warfare Sounds Play Track 2492 sounds included $28

    WARFARE SOUNDS is a massive collection of 2492 professionally recorded, edited and designed sound effects.  Including guns, bullet flybys, bullet impacts, silenced guns, explosions, footsteps, Foley, nature ambiances and more!

    WARFARE SOUNDS contains sounds from our other sound packages, including:

    Gun Sound Pack
    Bullet Impact Sounds
    Silenced Gun Sounds
    Explosion Sound Pack
    Footstep and Foley Sounds

    Plus additional content from Pro Sound Collection! This is the ultimate sound package for anyone working on a military type film or game.  Perfect for an FPS or any type of shooter.

    ALL sounds from WARFARE SOUNDS are already included in our full package Pro Sound Collection. So if you need more sounds. Be sure to check it out before purchase.

    28 %
    OFF
  • Bundles Videogame Foley Essentials Vol. I Play Track 4394 sounds included, 134 mins total $60

    Videogame Foley Essentials was created with the goal of assisting all editors and sound designers alike on their newly found videogame ventures, ready to satisfy all your foley needs.

    You’ll find on this library the following folders with clear nomenclature and respective metadata pertaining to various interactions either with environmental objects or the characters you’ll find in the game itself:

    • Buttons & Switches: everything from simple button presses (of all shapes, sizes and materials, including light switches), to plugging-in sounds of plastic devices and simple mechanisms being interacted with.
    • Clothing & Accessories: quick and snappy movement sounds from pants and jackets of diverse materials for all weather conditions; backpacks, pouches, belt buckles, keychains and zippers jiggling; rummaging and dressing-up takes of all the aforementioned clothing pieces and accessories.
    • Doors: door handle one-shots ranging from old/decrepit wooden doors to slick/modern ones; closing and opening sounds of multiple door types; insertion, removal and turning sounds of keys on numerous locks; hinges creaking and screeching.
    • Drawers & Cupboards: all types of drawer sounds based on metallic and wooden surfaces, both with and without bearings; small office cabinet sounds; creaky and quiet cupboards with and without simple lock sounds.
    • Footsteps: numerous one-shots of different shoes on a neutral surface (two shoe types, boots, tap dancing shoes, high heel boots, medium heel shoes, slippers, pool slippers and barefoot), while also including certain ground textures (bark, dirt, dry leaves, dry pine tree leaves, gravel, different plants, sand and wet dirt).
    • Rummaging & Jiggling: textures being handled in several ways, from rummaging/dropping/dragging/handling/jiggling sounds of scrap, wood, metallic fragments, plastic and cardboard boxes, cans, plastic bottles, metal plates and plenty of other materials/interactions.

    These sounds were created with videogame development in mind, meaning all sounds are short and snappy one-shots with dozens of iterations each, capable of being layered, stretched and bent with surprising results. All audio files were mixed at the same volume level, carefully cleaned-up of any unnecessary background noise and artifacts ready to be dropped onto any audio middleware solution such as Wwise or FMOD. The goal is to help the editor/sound designer on their workflow by simply dropping these ready-to-go sound batches on their middleware projects, avoiding auditory repetition and fatigue for the gamer whilst keeping them totally immersed on their upcoming adventures.


Latest releases:

  • COMPATIBLE WITH ANY VIDEO SOFTWARE
    CRAFTED BY INDUSTRY PROFESSIONALS
    INSTANT DIGITAL DOWNLOAD
    100% ROYALTY FREE

    Say hello to our ‘Futuristic Ambiences Sound FX’ pack! It’s full of 75 amazing sound effects. Whether you make movies, edit videos, or just love futuristic stuff, these sounds will take you to cool futuristic places.

    This pack has 75 different sounds that make you feel like you’re in a futuristic city or exploring space. It’s like having sounds from a futuristic movie or a space adventure.

    These effects are great for making scenes in a futuristic world or creating a super cool atmosphere. Dive into futuristic worlds with the ‘Futuristic Ambiences Sound FX’ pack and let your audience experience amazing, out-of-this-world adventures!

    Image
  • Environments & Ambiences Inside Transport Sound FX Play Track 54 sounds included, 40 mins total $14.95
    COMPATIBLE WITH ANY VIDEO SOFTWARE
    CRAFTED BY INDUSTRY PROFESSIONALS
    INSTANT DIGITAL DOWNLOAD
    100% ROYALTY FREE

    Step aboard with our ‘Inside Transport Sound FX’ pack! With 54 incredible sounds, this pack takes you on a journey through various transport settings. It’s part of our Ambiences collection, offering a diverse range of ambient soundscapes from around the world.

    Containing 54 unique sounds, this pack captures the essence of different transport modes, from the rumble of trains to the hum of airplanes and the chatter of bustling terminals. It’s like having your ticket to a sonic voyage across various travel experiences.

    Perfect for filmmakers, video editors, or anyone seeking authentic transport atmospheres, the ‘Inside Transport Sound FX’ pack adds realism and depth to your projects. Come aboard and experience the diverse world of transportation with this captivating collection from our Ambiences series!

    Image
  • Bundles Dark Ambiences Vol. 3 Play Track 12 sounds included, 17 mins total $14

    In this library you will find a dark ambience sounds.
    These sounds are perfect to use in movies, trailers, games, documentaries, etc.

    These sounds were all created by me on virtual synthesizers.

    After working these sounds in post-production, this library comes with 12 dark ambiences.

    18 %
    OFF
  • Environments & Ambiences Wind vol.2 Play Track 137+ sounds included, 340 mins total $31

    Wind in urban and natural environments, recorded all year round. Windy room tones, wind whistling under doors and windows, wind on the boats, in trees, in the city, windy ambiences, lakes, mountains etc. Check the list of sounds for full info. All files are metadata tagged, with detailed description of equipment, mic technique used and position of mics.

     

    Recording equipment used: Sennheisers MKH serie mics, LOM Usi, Primo, Rode mics; Rycote blimps, Sound Devices recorder/mixer, Sony and Tascam hand recorders with LOM Usi mics (in Rycote BBG windshields); Mogami cables, tripods, etc.

    Maximum peak level on all recordings is -5 dBFS (for great dynamic range, great headroom), originally recorded with great headroom keeping in mind and to maintain great dynamic range of this natural phenomena.

     

    21 %
    OFF
  • City Life Sound Effects Netherlands Collection 01 Play Track 500+ sounds included, 1067 mins total $72

    Embark on an auditory journey through the Netherlands with our extensive collection of 359 recordings, encompassing 45GB and 17+ hours of pristine audio. From the serene birdsong at dawn in Texel’s quaint towns to the dynamic ambience of a busy supermarket, each sound tells a story of Dutch life and nature.

    Recorded with the exceptional clarity of the DPA 4006a, Sennheiser mkh 8040 and Sound Devices 744T.
    Discover the gentle rustle of leaves in an autumn forest near Den Haag, the vibrant energy of Amsterdam’s old town, and the soothing harmony of harbors and lakeside parks.

    Whether for film, gaming, or immersive audio experiences, our Netherlands collection offers a rich palette of sounds that capture the essence of this beautiful country.

    10 %
    OFF
Need specific sound effects? Try a search below:


Binaural or Hard Panning Stereo?

To create a sense of presence in new medias (feeling of being physically present in the virtual scene), besides the usual narrative, tactical and strategic immersion, spatial immersion has been added to immersive design rules. Does it mean we need to position every single sound with HRTF processing then? Well, not necessarily.

In VR, you have the choice between using binaural positioning in place of stereo positioning (and we have the choice for any flat screen games as well now, which would likely open the door to new possibilities and discussion for first-person games).

Though less real, stereo positioning creates a stronger sense of directionality

Binaural positioning allows us to create a more realistic experience, as it accounts for real world physics in the calculations. But due to the long history of using only stereo positioning in games, players are accustomed to it, and using stereo positioning may offer certain advantages over binaural positioning. Though less real, stereo positioning creates a stronger sense of directionality, as the sound is hard panned, making it more obvious where the source is positioned, relative to the listener.

Depending on your game, you might want to use different positioning systems for different components.

If your purpose is to completely immerse the player in the world and create a sense of connection with it, you could use the binaural positioning to establish a sense of realness. For game-play feedbacks, you could possibly use hard-panning, as players are more familiar with it, to enable them to focus their attention on the gameplay, creating immersion with the gameplay rather than the environment. The stronger sense of directionality offered by the stereo positioning provides more obvious cues for the player, allowing them to receive stronger audio feedback.

The decision between using stereo positioning and/or binaural positioning should be a creative one, rather than a technical one, and depends on your game needs per component and situation. It’s all about experimentation and choices.

Preferably, your choices should be made prior starting the audio production of your game, as your positioning system(s) will affect your technical pipeline on the implementation front, as well as the creative approach when building assets and implementing them.
 

 

Creative Audio Language

Your creative audio approach when starting to work on a VR game is then affected by (at least) all of the above. You should ask yourself: When to use audio for multi-sensory integration purpose(s)? How human evolution (realistic approach) and game audio language (gameplay feedbacks and players expectation) impact your creative decision and technical choices and pipeline?

But then, what do you do with any extra-diegetic sources? Let’s take a concrete example with Music. From an end-user perspective, people need to be guided to understand what to think of a scene, what’s the context, what to feel, or what’s the setting. Those are some of the primarily Music purposes. It really depends of your game creative approach, so aim for what works best for your game. VR needs to be interactive, and Players actions need to feel meaningful in the game. Adaptive music needs to be think of carefully and creatively to build something that works both for the VR medium, and the game itself.

On the other hand, players have expectation of realism when playing VR game or experiences, and will notice anything done wrong on the soundscape, anything that could take them out of the experience. Contents production value, audio repetition, and the degree of direct or indirect interactivity to be built in Virtual Worlds can be extremely challenging.

You need to decide your intimate zone, the zone where you could interact with object, the zone where you need to add a lot of details in your sounds, both for positioning and dynamic layering matters.

Interactivity is a big topic of itself for audio in VR. You need to decide your intimate zone, the zone where you could interact with object, the zone where you need to add a lot of details in your sounds, both for positioning and dynamic layering matters. Audio can also be used as an input in VR, which once again reinforce the multi-sensory integration with haptic and visuals, and could possibly suggest taste or smell, if the MSI moment is totally mastered. Concrete example, let’s say you could grab a cigarette in the game, take the controller to your mouth, inhale and exhale would trigger respectively the subtle and visceral sound of a cigarette being burned (plus its visual feedback) and the visual cue of the smoke VFX along the sound of your character exhaling, or the direct sound of yourself (from the VR microphone) going through the acoustic rendering of the virtual environment.
 

To sum up

The main rule when doing audio for VR is that…there are no rules! Creative language is currently being developed by pioneers in the industry, both for the VR medium itself, and the type of games that works well for it.

Rome wasn’t built in a day, and as for all great things, VR and VR audio need time to reach its full potential. We are at the beginning of the VR in terms of technology and development, and there is no perfect recipe when working on VR audio.

Audio is not limited to what’s on screen anymore and should play an even bigger role than in flat screen games, serving all gameplay feedbacks, emotional states, engagement, storytelling, spatial immersion, multi-sensory integration, player’s position relative to the game action (and to some extend body balance).
 

A big thanks to Alex Riviere for his insights on VR audio!

 

Please share this:


 

About Alex Riviere:

Alex Riviere is a French born audio artist that have been working on many projects, from record albums (awarded one platinum and one gold record), commercials, documentaries to video-games, including the Gunjack series, Final Fantasy XIV, Call of Duty Online, Transformers: Rising, Civilization Revolution 2 Plus, etc.

A pioneer in VR Audio, Alex has built his experience developing for all major VR platforms (PSVR, Gear VR, Oculus Rift, Google Daydream, HTC Vive) as Audio Director on the VR titles EVE Gunjack, and Gunjack 2: End of Shift.



 
 
THE WORLD’S EASIEST WAY TO GET INDEPENDENT SOUND EFFECTS:
 
A Sound Effect gives you easy access to an absolutely huge sound effects catalog from a myriad of independent sound creators, all covered by one license agreement - a few highlights:

  • Nature & Countryside Ambiences African Jungle Play Track 80+ sounds included, 460 mins total From: $190

    African Jungle is a huge collection of unique ambiences recorded in Central Africa’s Congo Basin rainforests. To my knowledge, these are the first surround field recordings made in this area. Last year I spent a whole month off the grid, listening to and recording a variety of habitats at all times of day and night, at the height of the rainy season when birds are most vocal. It took me many days of travel to get to these places, from using trains, boats, cars and planes to hiking on foot in the jungle.

  • Foley Sound Effects Pickups and Small Item Sounds Play Track 280 sounds included, 7 mins total $4.16

    Great Audio is Essential

    Essential Audio Pickups and Small Item Sounds contains 280 Foley sounds designed specifically for Video Game Sound Designers who need the usual Pick-Up sounds and SFX for common small items, including: Keys, Coins, Access Card, Pills/Health, Latches, Locks, Chains, Bags, and many many more!

    Features:

    •  280 sounds
    • Bags
    • Coins
    • Keys
    • Pill Bottle
    • Cable Ties
    • Camera
    • Lighter
    • Access Card
    • Chains
    • Locks
    • Clips and Clasps
    • Computer Mouse
    • Boxes
    • Papers
    • Clicks and Ticks
    • Tools
    • Scissors
    • Wood Game Pieces
    • Zippers

    Number of Audio Waves: 280

    Sample rate / bit rate: 44,100 Hz / 16 bit

    75 %
    OFF
  • Game Audio Packs Warfare Sounds Play Track 2492 sounds included $28

    WARFARE SOUNDS is a massive collection of 2492 professionally recorded, edited and designed sound effects.  Including guns, bullet flybys, bullet impacts, silenced guns, explosions, footsteps, Foley, nature ambiances and more!

    WARFARE SOUNDS contains sounds from our other sound packages, including:

    Gun Sound Pack
    Bullet Impact Sounds
    Silenced Gun Sounds
    Explosion Sound Pack
    Footstep and Foley Sounds

    Plus additional content from Pro Sound Collection! This is the ultimate sound package for anyone working on a military type film or game.  Perfect for an FPS or any type of shooter.

    ALL sounds from WARFARE SOUNDS are already included in our full package Pro Sound Collection. So if you need more sounds. Be sure to check it out before purchase.

    28 %
    OFF
Explore the full, unique collection here

Latest sound effects libraries:
 
  • COMPATIBLE WITH ANY VIDEO SOFTWARE
    CRAFTED BY INDUSTRY PROFESSIONALS
    INSTANT DIGITAL DOWNLOAD
    100% ROYALTY FREE

    Say hello to our ‘Futuristic Ambiences Sound FX’ pack! It’s full of 75 amazing sound effects. Whether you make movies, edit videos, or just love futuristic stuff, these sounds will take you to cool futuristic places.

    This pack has 75 different sounds that make you feel like you’re in a futuristic city or exploring space. It’s like having sounds from a futuristic movie or a space adventure.

    These effects are great for making scenes in a futuristic world or creating a super cool atmosphere. Dive into futuristic worlds with the ‘Futuristic Ambiences Sound FX’ pack and let your audience experience amazing, out-of-this-world adventures!

    Image
  • Environments & Ambiences Inside Transport Sound FX Play Track 54 sounds included, 40 mins total $14.95
    COMPATIBLE WITH ANY VIDEO SOFTWARE
    CRAFTED BY INDUSTRY PROFESSIONALS
    INSTANT DIGITAL DOWNLOAD
    100% ROYALTY FREE

    Step aboard with our ‘Inside Transport Sound FX’ pack! With 54 incredible sounds, this pack takes you on a journey through various transport settings. It’s part of our Ambiences collection, offering a diverse range of ambient soundscapes from around the world.

    Containing 54 unique sounds, this pack captures the essence of different transport modes, from the rumble of trains to the hum of airplanes and the chatter of bustling terminals. It’s like having your ticket to a sonic voyage across various travel experiences.

    Perfect for filmmakers, video editors, or anyone seeking authentic transport atmospheres, the ‘Inside Transport Sound FX’ pack adds realism and depth to your projects. Come aboard and experience the diverse world of transportation with this captivating collection from our Ambiences series!

    Image
  • Bundles Dark Ambiences Vol. 3 Play Track 12 sounds included, 17 mins total $14

    In this library you will find a dark ambience sounds.
    These sounds are perfect to use in movies, trailers, games, documentaries, etc.

    These sounds were all created by me on virtual synthesizers.

    After working these sounds in post-production, this library comes with 12 dark ambiences.

    18 %
    OFF
  • Environments & Ambiences Wind vol.2 Play Track 137+ sounds included, 340 mins total $31

    Wind in urban and natural environments, recorded all year round. Windy room tones, wind whistling under doors and windows, wind on the boats, in trees, in the city, windy ambiences, lakes, mountains etc. Check the list of sounds for full info. All files are metadata tagged, with detailed description of equipment, mic technique used and position of mics.

     

    Recording equipment used: Sennheisers MKH serie mics, LOM Usi, Primo, Rode mics; Rycote blimps, Sound Devices recorder/mixer, Sony and Tascam hand recorders with LOM Usi mics (in Rycote BBG windshields); Mogami cables, tripods, etc.

    Maximum peak level on all recordings is -5 dBFS (for great dynamic range, great headroom), originally recorded with great headroom keeping in mind and to maintain great dynamic range of this natural phenomena.

     

    21 %
    OFF
  • City Life Sound Effects Netherlands Collection 01 Play Track 500+ sounds included, 1067 mins total $72

    Embark on an auditory journey through the Netherlands with our extensive collection of 359 recordings, encompassing 45GB and 17+ hours of pristine audio. From the serene birdsong at dawn in Texel’s quaint towns to the dynamic ambience of a busy supermarket, each sound tells a story of Dutch life and nature.

    Recorded with the exceptional clarity of the DPA 4006a, Sennheiser mkh 8040 and Sound Devices 744T.
    Discover the gentle rustle of leaves in an autumn forest near Den Haag, the vibrant energy of Amsterdam’s old town, and the soothing harmony of harbors and lakeside parks.

    Whether for film, gaming, or immersive audio experiences, our Netherlands collection offers a rich palette of sounds that capture the essence of this beautiful country.

    10 %
    OFF

   

Leave a Reply

Your email address will not be published. Required fields are marked *

HTML tags are not allowed.