virtual reality sound Asbjoern Andersen


Alex Riviere has worked with all the major VR platforms - PSVR, Gear VR, Oculus Rift, Google Daydream & HTC Vive - as Audio Director on VR titles EVE Gunjack, and Gunjack 2: End of Shift. And below, he shares his insights on creating audio for VR:
Written by Alex Riviere
Please share:

Lots of my fellow game audio professionals are often asking me the question about the difference working with Virtual Reality compared to flat-screen games except for Binaural positioning. Well, my short answer is that to create an alternate reality that provides a natural listening experience in (gaming) Virtual Environments, our game audio creative approach needs to be adapted, and binaural positioning plays a major role into revisiting it. The long answer is below…
 

Multi-Sensory Integration

With the lack of full-sensory usage in Virtual Reality games (only Vision, Hearing, and Touch through haptics are integrated), the human brain has the power to adapt the focus or the perception of the senses that are integrated. Don’t get me wrong, you won’t become a super-hero with sharpen sense(s) by playing VR games, but the brain can rewire itself through training and learning, to enhance the use of integrated senses, with the goal to use information at its disposal to analyze and interact with the surroundings.

Your smell and taste are not turned off while playing VR games, so those two senses are providing contradictory information between what you see and hear from the game versus what you smell and possibly taste from your physical surrounding environment. Touch (and proprioceptors – sensory receptor which receives stimuli from within the body, especially one that responds to position and movement) would send mixed information between the virtual world and the real world.

The senses that are used need to be reproduced respectfully to immerse the player mentally in the virtual world, combining well-designed and controlled simultaneous visual, auditory, and haptic cues to create a believable Virtual Reality experience. The brain allows to react when subtle sensory signals that might not seems important on their own gets trigger simultaneously, and that’s the power of multisensory integration (MSI).

“A basic tenet of multisensory integration is the ability of one sensory modality to enhance or to suppress information from another sensory modality.”

(Calvert et al., 2004)

A good example would be the “Virtual barber shop” demo released about 10 years ago on Youtube.

Most people experiencing this binaural audio demo for the first time, if mentally immersed (close your eyes while listening to not be distracted by your surroundings, and take a seat), would feel someone is literally behind them touching their scalp, talking to them, and cutting their hairs.
 

Human Evolution & Game Audio

The modern world we’re living in as well as the interactive medias audio language that have been developed for decades have been bringing modern sounds and audio cues to us, and we are constantly analyzing them without even necessarily being conscious of it. Our ears and brain can distinguish thousands of sounds at the same time, being highly informative, sounds are giving us the ability to analyze situations or events and react to it instinctively.

Sounds bring physical and spatial information about objects and environments that our brain can analyze, so we can learn about those sound sources and understand better a situation of objects and events.

In VR, hearing is the only sense able to provide full spatial information going beyond our field of view

Our auditory system provides a lot of information about the world surrounding us. In real life, vision, audition, and our sense of smell provide information that helps us to identify object, situations and navigate in our environment. But in VR, hearing is the only sense able to provide full spatial information going beyond our field of view, including elevation, 360 degrees and depth, allowing us to guide our decisions and behaviors as well as understanding our virtual surroundings.

In other terms, Audio in VR is the only medium able to make you turn your head, grabbing your attention and orientation to possible scripted events, or pre-defined directions and paths.
 


Popular on A Sound Effect right now - article continues below:

 

Latest releases:  
  • Destruction & Impact Medieval Battle Play Track 100 sounds included $26

    Medieval Battle contains a wide variety of sounds taken straight from the battlefield. Unleash the full force of your army with effects from a diverse range of medieval weaponry, crowds, horses and battle charges. There are also battle cries, screams and shouts to make the audience feel like they are right there on the frontlines. To bring you the most organic and natural battle sounds, we recorded actors performing in a quiet exterior location. All sounds were recorded in 24Bit 96kHz, allowing for further sonic manipulation.

    If you think your sound collection is in need of some reinforcements then this is the library for you! Medieval Battle is ideal for use in Film, TV and Game projects, and will make a perfect addition to your sound design arsenal.

    Here are the included folders:

    Crowds: Crowd rushes and static battle cries in a natural exterior location make these effects perfect for building the bed of a huge Medieval warzone.

    Female Voices: 4 individual female actors perform multiple vocalisations including screams, grunts and efforts.

    Sword & Shield Hits: A variety of sword and shield hits performed by weapons and combat experts.

    Male Voices: 4 individual male actors perform multiple vocalisations including screams, grunts and efforts.

     

    Add to cart
  • Cars Vehicles Pass-by Play Track 90 sounds included, 23 mins total $15

    Cars, trucks, motorcycles at different speeds on a rural road in the middle of nowhere during summer.
    From left, from right, from both sides, single vehicle or several at the same time.

    Recorded simultaneously with an ORTF and a AB microphone stereo pairs.

    Gear used:
    Sound Devices 788T
    MKH8020 stereo pair
    MKH8040 stereo pair
    Add to cart
  • Destruction & Impact Rocks & Debris Play Track 160 sounds included $40 $30

    Crisp, clean, crunchy and closely mic’d impacts with various types of pebble and dust debris.

    I recorded all these sounds with my brothers during a trip home to Ireland.
    Close to our home (out in the middle of nowhere), there is a little quarry with a lot of sand, dust, stones, rocks and some grass. There was also a broken plough there that gave some nice stone on metal impact sounds.

    Equipment:
    2 Line Audio OM1s
    2 Line Audio CM3s
    Røde NTG3
    Tascam DR100

    25 %
    OFF
    Add to cart
  • Motorcycles BSA M20 1940 Motorcycle Play Track 322 sounds included $249

    Recording of a BSA M20 WW2 motorcycle with a 1 cylinder 500cm3 engine with 4 gears.

    The BSA M20 WW2 motorcycle sound collection features 322 sounds in 12.02 gigabytes of audio. It showcases a British 1940 Birmingham Small Arms Company military bike and the sound of its 1 cylinder 500cm3 engine with 4 gears.

    The sound bundle offers 20 synchronized recordings of onboard, exterior, and performed sound fx. The 7 onboard perspectives record sounds from the engine, exhaust, and onboard locations such as the seat and frame, all of which are also mixed into three pre-rendered tracks with driving at steady RPM, ramps, and gearshifts. The 9 exterior perspectives include turning on and off, departing, arriving, and passing from multiple perspectives and distances from slow to fast speeds. These are paired with performed Foley of handlebar clutch and brake, kickstarting, and more.

    The package includes Pro Tools and Reaper mixing sessions, as well as full metadata embedded in every clip.

    Add to cart
  • Destruction & Impact Sci-Fi Guns Sounds Play Track 146 sounds included $20 $14.99

    Sci-fi Guns Sounds

    146 modern guns sound effects divided into more than 15 different types of guns:

    – Electric Revolver
    – Cannon Gun
    – Blaster Rifle
    – Low Gun
    – Alien Guns
    – Automatic Gun
    – Old Shotgun and Old Cannon Shots
    – Explosive Gun
    – Grenades
    – Compressed Air Gun
    – And many more… Check the Tracklist PDF to see the full list of sounds included in the pack

    Soundcloud Preview
    Tracklist PDF

    More about the pack:

    – Apart from shot sounds, there are reloading & handling sounds

    – There’s also the ”Stab” sound and the ”Hit Marker” sound.

    – There are single-shot sounds but also loops (Rotary cannon)

    – These SFX are suitable for Sci-Fi games but also for warfare, modern warfare ones.

    – Intuitive file naming

    – All you’ll ever need regarding Sci-Fi Gun Sounds

    [Format]
    All files are in:

    WAV – Stereo  44.1 Khz, 16bit
    OGG – Stereo  44.1 Khz, 16bit

    [Use them again & again]

    Use the sound effects over and over, in any of your projects or productions, forever without any additional fees or royalties. Use the SFX in your game, in your trailer, in a Kickstarter campaign, wherever you need to, as much as you want to.

    [Professional audio services]

    For custom music, sound design, sound engineering or any other game audio services, please send a mail to: audioalchemiststore@gmail.com

    25 %
    OFF
    Ends 1579647599
    Add to cart


Need specific sound effects? Try a search below:
 
Want more stories like this? Follow A Sound Effect:
 
                              

Binaural or Hard Panning Stereo?

To create a sense of presence in new medias (feeling of being physically present in the virtual scene), besides the usual narrative, tactical and strategic immersion, spatial immersion has been added to immersive design rules. Does it mean we need to position every single sound with HRTF processing then? Well, not necessarily.

In VR, you have the choice between using binaural positioning in place of stereo positioning (and we have the choice for any flat screen games as well now, which would likely open the door to new possibilities and discussion for first-person games).

Though less real, stereo positioning creates a stronger sense of directionality

Binaural positioning allows us to create a more realistic experience, as it accounts for real world physics in the calculations. But due to the long history of using only stereo positioning in games, players are accustomed to it, and using stereo positioning may offer certain advantages over binaural positioning. Though less real, stereo positioning creates a stronger sense of directionality, as the sound is hard panned, making it more obvious where the source is positioned, relative to the listener.

Depending on your game, you might want to use different positioning systems for different components.

If your purpose is to completely immerse the player in the world and create a sense of connection with it, you could use the binaural positioning to establish a sense of realness. For game-play feedbacks, you could possibly use hard-panning, as players are more familiar with it, to enable them to focus their attention on the gameplay, creating immersion with the gameplay rather than the environment. The stronger sense of directionality offered by the stereo positioning provides more obvious cues for the player, allowing them to receive stronger audio feedback.

The decision between using stereo positioning and/or binaural positioning should be a creative one, rather than a technical one, and depends on your game needs per component and situation. It’s all about experimentation and choices.

Preferably, your choices should be made prior starting the audio production of your game, as your positioning system(s) will affect your technical pipeline on the implementation front, as well as the creative approach when building assets and implementing them.
 

 

Creative Audio Language

Your creative audio approach when starting to work on a VR game is then affected by (at least) all of the above. You should ask yourself: When to use audio for multi-sensory integration purpose(s)? How human evolution (realistic approach) and game audio language (gameplay feedbacks and players expectation) impact your creative decision and technical choices and pipeline?

But then, what do you do with any extra-diegetic sources? Let’s take a concrete example with Music. From an end-user perspective, people need to be guided to understand what to think of a scene, what’s the context, what to feel, or what’s the setting. Those are some of the primarily Music purposes. It really depends of your game creative approach, so aim for what works best for your game. VR needs to be interactive, and Players actions need to feel meaningful in the game. Adaptive music needs to be think of carefully and creatively to build something that works both for the VR medium, and the game itself.

On the other hand, players have expectation of realism when playing VR game or experiences, and will notice anything done wrong on the soundscape, anything that could take them out of the experience. Contents production value, audio repetition, and the degree of direct or indirect interactivity to be built in Virtual Worlds can be extremely challenging.

You need to decide your intimate zone, the zone where you could interact with object, the zone where you need to add a lot of details in your sounds, both for positioning and dynamic layering matters.

Interactivity is a big topic of itself for audio in VR. You need to decide your intimate zone, the zone where you could interact with object, the zone where you need to add a lot of details in your sounds, both for positioning and dynamic layering matters. Audio can also be used as an input in VR, which once again reinforce the multi-sensory integration with haptic and visuals, and could possibly suggest taste or smell, if the MSI moment is totally mastered. Concrete example, let’s say you could grab a cigarette in the game, take the controller to your mouth, inhale and exhale would trigger respectively the subtle and visceral sound of a cigarette being burned (plus its visual feedback) and the visual cue of the smoke VFX along the sound of your character exhaling, or the direct sound of yourself (from the VR microphone) going through the acoustic rendering of the virtual environment.
 

To sum up

The main rule when doing audio for VR is that…there are no rules! Creative language is currently being developed by pioneers in the industry, both for the VR medium itself, and the type of games that works well for it.

Rome wasn’t built in a day, and as for all great things, VR and VR audio need time to reach its full potential. We are at the beginning of the VR in terms of technology and development, and there is no perfect recipe when working on VR audio.

Audio is not limited to what’s on screen anymore and should play an even bigger role than in flat screen games, serving all gameplay feedbacks, emotional states, engagement, storytelling, spatial immersion, multi-sensory integration, player’s position relative to the game action (and to some extend body balance).
 

A big thanks to Alex Riviere for his insights on VR audio!

 

Please share this:


 

About Alex Riviere:

Alex Riviere is a French born audio artist that have been working on many projects, from record albums (awarded one platinum and one gold record), commercials, documentaries to video-games, including the Gunjack series, Final Fantasy XIV, Call of Duty Online, Transformers: Rising, Civilization Revolution 2 Plus, etc.

A pioneer in VR Audio, Alex has built his experience developing for all major VR platforms (PSVR, Gear VR, Oculus Rift, Google Daydream, HTC Vive) as Audio Director on the VR titles EVE Gunjack, and Gunjack 2: End of Shift.


 
 
THE WORLD’S EASIEST WAY TO GET INDEPENDENT SOUND EFFECTS:
 
A Sound Effect gives you easy access to an absolutely huge sound effects catalog from a myriad of independent sound creators, all covered by one license agreement - a few highlights:
 
 
  • Ice Frozen Play Track 809 sounds included $45 $23

    Frozen is a collection of ice impacts, scrapes, debris, slides and more taken during a miserably cold winter in the arctic tundra of Massachusetts. These recordings were meticulously recorded, edited, named, renamed, edited again, re recorded, then re-renamed (you get the point) to give you the best quality ice library you can get.

    The library roughly is broken into two sections, raw and processed. Raw is designed for sound designers to go wild with: stretch, distort, compress, and more to your hearts content. The processed section is for designers and content creators on the go. If you’re on a tight deadline, pulling some sounds that are already going to get you close to where you need to be is imperative. These processed versions play very nicely even when not designing ice.

    49 %
    OFF
    Ends 1580511599
    Add to cart
  • Funny / Comedy / Cartoon Explainer Video Sound Kit Play Track 1222 sounds included, 120 mins total $96 $69

    Skip all the hassle and wasted time trying to find the right sounds. Explainer Video Sound Kit has everything you need to create dynamic sound tracks for video ads, commercials, animations, marketing and explainer videos. Importantly, it’s all royalty-free and an unbeatable time saving value.

    As you know, creators of motion graphics, animations and explainer videos struggle mightily with finding the right sound effects for their projects. Sadly, wasting lots of time and energy. That’s why we created the Explainer Video Sound Kit. It’s an all-in-one power pack that gets your project done fast. 1200 designed sounds, cinematic elements, transitions, Foley, natural and organic sounds and a sweet selection of background music all ready to go.

    Instantly take your production to a new level of engaging sound. Create compelling audio/video experiences easier than ever. Save time and level up your audio skills like a pro with the Explainer Video sound library!

    28 %
    OFF
    Add to cart
  • Destruction & Impact Cinematic Gore Play Track 600+ sounds included $55

    If you need sounds of gore, this SFX collection is truly something special. Delivering more than 600 sounds of blood, guts, bone breaks, sword and knife stabs and cuts, splatters, spurts, flesh tears, impacts – and sawing – its gets you a brutally effective collection of sounds to work with, both raw and designed.

    Add to cart
 
Explore the full, unique collection here

Latest sound effects libraries:
 
  • Destruction & Impact Medieval Battle Play Track 100 sounds included $26

    Medieval Battle contains a wide variety of sounds taken straight from the battlefield. Unleash the full force of your army with effects from a diverse range of medieval weaponry, crowds, horses and battle charges. There are also battle cries, screams and shouts to make the audience feel like they are right there on the frontlines. To bring you the most organic and natural battle sounds, we recorded actors performing in a quiet exterior location. All sounds were recorded in 24Bit 96kHz, allowing for further sonic manipulation.

    If you think your sound collection is in need of some reinforcements then this is the library for you! Medieval Battle is ideal for use in Film, TV and Game projects, and will make a perfect addition to your sound design arsenal.

    Here are the included folders:

    Crowds: Crowd rushes and static battle cries in a natural exterior location make these effects perfect for building the bed of a huge Medieval warzone.

    Female Voices: 4 individual female actors perform multiple vocalisations including screams, grunts and efforts.

    Sword & Shield Hits: A variety of sword and shield hits performed by weapons and combat experts.

    Male Voices: 4 individual male actors perform multiple vocalisations including screams, grunts and efforts.

     

  • Cars Vehicles Pass-by Play Track 90 sounds included, 23 mins total $15

    Cars, trucks, motorcycles at different speeds on a rural road in the middle of nowhere during summer.
    From left, from right, from both sides, single vehicle or several at the same time.

    Recorded simultaneously with an ORTF and a AB microphone stereo pairs.

    Gear used:
    Sound Devices 788T
    MKH8020 stereo pair
    MKH8040 stereo pair
  • Destruction & Impact Rocks & Debris Play Track 160 sounds included $40 $30

    Crisp, clean, crunchy and closely mic’d impacts with various types of pebble and dust debris.

    I recorded all these sounds with my brothers during a trip home to Ireland.
    Close to our home (out in the middle of nowhere), there is a little quarry with a lot of sand, dust, stones, rocks and some grass. There was also a broken plough there that gave some nice stone on metal impact sounds.

    Equipment:
    2 Line Audio OM1s
    2 Line Audio CM3s
    Røde NTG3
    Tascam DR100

    25 %
    OFF
  • Motorcycles BSA M20 1940 Motorcycle Play Track 322 sounds included $249

    Recording of a BSA M20 WW2 motorcycle with a 1 cylinder 500cm3 engine with 4 gears.

    The BSA M20 WW2 motorcycle sound collection features 322 sounds in 12.02 gigabytes of audio. It showcases a British 1940 Birmingham Small Arms Company military bike and the sound of its 1 cylinder 500cm3 engine with 4 gears.

    The sound bundle offers 20 synchronized recordings of onboard, exterior, and performed sound fx. The 7 onboard perspectives record sounds from the engine, exhaust, and onboard locations such as the seat and frame, all of which are also mixed into three pre-rendered tracks with driving at steady RPM, ramps, and gearshifts. The 9 exterior perspectives include turning on and off, departing, arriving, and passing from multiple perspectives and distances from slow to fast speeds. These are paired with performed Foley of handlebar clutch and brake, kickstarting, and more.

    The package includes Pro Tools and Reaper mixing sessions, as well as full metadata embedded in every clip.

  • Destruction & Impact Sci-Fi Guns Sounds Play Track 146 sounds included $20 $14.99

    Sci-fi Guns Sounds

    146 modern guns sound effects divided into more than 15 different types of guns:

    – Electric Revolver
    – Cannon Gun
    – Blaster Rifle
    – Low Gun
    – Alien Guns
    – Automatic Gun
    – Old Shotgun and Old Cannon Shots
    – Explosive Gun
    – Grenades
    – Compressed Air Gun
    – And many more… Check the Tracklist PDF to see the full list of sounds included in the pack

    Soundcloud Preview
    Tracklist PDF

    More about the pack:

    – Apart from shot sounds, there are reloading & handling sounds

    – There’s also the ”Stab” sound and the ”Hit Marker” sound.

    – There are single-shot sounds but also loops (Rotary cannon)

    – These SFX are suitable for Sci-Fi games but also for warfare, modern warfare ones.

    – Intuitive file naming

    – All you’ll ever need regarding Sci-Fi Gun Sounds

    [Format]
    All files are in:

    WAV – Stereo  44.1 Khz, 16bit
    OGG – Stereo  44.1 Khz, 16bit

    [Use them again & again]

    Use the sound effects over and over, in any of your projects or productions, forever without any additional fees or royalties. Use the SFX in your game, in your trailer, in a Kickstarter campaign, wherever you need to, as much as you want to.

    [Professional audio services]

    For custom music, sound design, sound engineering or any other game audio services, please send a mail to: audioalchemiststore@gmail.com

    25 %
    OFF
    Ends 1579647599
 
FOLLOW OR SUBSCRIBE FOR THE LATEST IN FANTASTIC SOUND:
 
                              
 
GET THE MUCH-LOVED A SOUND EFFECT NEWSLETTER:
 
The A Sound Effect newsletter gets you a wealth of exclusive stories and insights
+ free sounds with every issue:
 
Subscribe here for free SFX with every issue

Leave a Reply

Your email address will not be published. Required fields are marked *

HTML tags are not allowed.