virtual reality sound Asbjoern Andersen


Alex Riviere has worked with all the major VR platforms - PSVR, Gear VR, Oculus Rift, Google Daydream & HTC Vive - as Audio Director on VR titles EVE Gunjack, and Gunjack 2: End of Shift. And below, he shares his insights on creating audio for VR:
Written by Alex Riviere
Please share:

Lots of my fellow game audio professionals are often asking me the question about the difference working with Virtual Reality compared to flat-screen games except for Binaural positioning. Well, my short answer is that to create an alternate reality that provides a natural listening experience in (gaming) Virtual Environments, our game audio creative approach needs to be adapted, and binaural positioning plays a major role into revisiting it. The long answer is below…
 

Multi-Sensory Integration

With the lack of full-sensory usage in Virtual Reality games (only Vision, Hearing, and Touch through haptics are integrated), the human brain has the power to adapt the focus or the perception of the senses that are integrated. Don’t get me wrong, you won’t become a super-hero with sharpen sense(s) by playing VR games, but the brain can rewire itself through training and learning, to enhance the use of integrated senses, with the goal to use information at its disposal to analyze and interact with the surroundings.

Your smell and taste are not turned off while playing VR games, so those two senses are providing contradictory information between what you see and hear from the game versus what you smell and possibly taste from your physical surrounding environment. Touch (and proprioceptors – sensory receptor which receives stimuli from within the body, especially one that responds to position and movement) would send mixed information between the virtual world and the real world.

The senses that are used need to be reproduced respectfully to immerse the player mentally in the virtual world, combining well-designed and controlled simultaneous visual, auditory, and haptic cues to create a believable Virtual Reality experience. The brain allows to react when subtle sensory signals that might not seems important on their own gets trigger simultaneously, and that’s the power of multisensory integration (MSI).

“A basic tenet of multisensory integration is the ability of one sensory modality to enhance or to suppress information from another sensory modality.”

(Calvert et al., 2004)

A good example would be the “Virtual barber shop” demo released about 10 years ago on Youtube.

Most people experiencing this binaural audio demo for the first time, if mentally immersed (close your eyes while listening to not be distracted by your surroundings, and take a seat), would feel someone is literally behind them touching their scalp, talking to them, and cutting their hairs.
 

Human Evolution & Game Audio

The modern world we’re living in as well as the interactive medias audio language that have been developed for decades have been bringing modern sounds and audio cues to us, and we are constantly analyzing them without even necessarily being conscious of it. Our ears and brain can distinguish thousands of sounds at the same time, being highly informative, sounds are giving us the ability to analyze situations or events and react to it instinctively.

Sounds bring physical and spatial information about objects and environments that our brain can analyze, so we can learn about those sound sources and understand better a situation of objects and events.

In VR, hearing is the only sense able to provide full spatial information going beyond our field of view

Our auditory system provides a lot of information about the world surrounding us. In real life, vision, audition, and our sense of smell provide information that helps us to identify object, situations and navigate in our environment. But in VR, hearing is the only sense able to provide full spatial information going beyond our field of view, including elevation, 360 degrees and depth, allowing us to guide our decisions and behaviors as well as understanding our virtual surroundings.

In other terms, Audio in VR is the only medium able to make you turn your head, grabbing your attention and orientation to possible scripted events, or pre-defined directions and paths.


Popular on A Sound Effect right now - article continues below:


Trending right now:

  • Cinematic & Trailer Sound Effects Abandoned Oil Tank Play Track 327 sounds included $40

    On a recording trip to the smaller Danish port town Struer, I came across an abandoned empty oil tank. Inside I found two giant wires attached to the floor and the ceiling of the tank, almost like a huge 2 stringed double bass. I’m telling you the reverb in there was longer than the Eiffel Tower on a cloudless day !

    On the outside this was just an empty abandoned building, but it turned out it was not empty at all because it was full of sound and a few dead birds. I’m glad I was curious enough to go inside for a closer investigation and not just pass by.

    After several hours of concentrated recording, it was nice to get my ears back out in dry air. I’m happy there are no open sends to Valhalla in real life but experiencing the eternal resonance on the low frequencies from the wires was absolutely amazing.

    Abandoned Oil Tank was recorded with a LOM basic Ucho stereo pair, a Zoom H6 and the LOM Geofon. All files are in 96 kHz/24 bit and contain the original recordings.

    The result is a unique sound effect library including balloon pop impulse responses from a truly unique location that very recently became even more unique since word says that the oil tank is no more and has abandoned this world.

    327 individual sounds – all tagged with Universal Category System (UCS) metadata.

    www.katrineamsler.com

    20 %
    OFF
    Ends 1713131999
  • Electricity Sound Effects Polarity Play Track 975 sounds included $72

    Polarity delivers more than 950 sounds of electricity, science and technology – captured in several locations around the world, from electricity museums to science labs. About 50% of the library is all about electricity, with various types of Jacob’s Ladders, Tesla Coils, Ruhmkorff lamp and all sorts of impactful bursts of energy.

    Then we go through welders, plasma spheres, 3D printers, starting to cover a more broad technology theme – like old phones, telegraphs, dynamo wheels, rotary dials, whirling watchers, alarm, lab centrifuges, something scientists call a roller and a rocker, servo sounds, neon lights, a wimshurst machine and sparklers.

    Many sounds in this section were captured from vintage equipment, from a 1928’s tram to old telephone switchboards, high voltage levers and control surfaces.

    All content was recorded at 192KHz with a Sanken CO100K, a couple of Sennheiser 8040 and a Neumann 81i, translating into final assets that have plenty of ultrasonic content, ready for the most extreme manipulation.

    Bonus: Two extra libraries included for free:
    This library also includes two additional releases from Mattia Cellotto - for free: Crunch Mode delivers 230 crunchy sounds made with a variety of vegetables, fresh bread, pizza crust and a selection of frozen goods. The Borax Experiment gets you 158 squishy, gory, slimy and gooey sounds.
    20 %
    OFF
  • Door Sound Effects Noisy Doors Play Track 44 sounds included $10

    This library contains recordings of eleven different types of doors specifically chosen for their interesting sound. I performed all actions with a lot of detail covering “open/close”, “knocks”, “bangs and thuds”, “handle grab and rattle” and when possible also “crackles and squeaks” (all recorded at various intensities)

    50 %
    OFF
    Ends 1711666799
  • Foley Sound Effects Skate Play Track 114+ sounds included $15

    A little skateboard recording session of some rolling on surfaces such as Wood, Metal, Concrete and Asphalt. Board manipulations like Wheels spinning, Board impact on the floor, Grinding metal and concrete – and more.

    The library contains 114 files of skateboarding sounds.

    Each sound has been meticulously edited individually. All files were recorded and are delivered in 24bit 96kHz. Broadcast Wave files, all embedded with metadata information for easy import and to ensure a fast and easy workflow.


Latest releases:

  • Trench Rammers is a one-of-a-kind sound library, created by sound designer Barney Oram. It features recordings of two 20th century Trench Rammers, some of the last operating machines of their kind in the world. The library features 48 files in total, capturing the Trench Rammers using a variety of microphones and perspectives, supplied in 192kHz 24bit WAV file format.

    This library contains multiple recordings of two antique Trench Rammers, the Pegson ER5 and the Warsop Benjo, and also features additional recordings of four antique stationary engines, the Amanco Hired Man (1920s), the JAP 1947, the Lister D (1940s) and the Norman T3000.

    Recorded in Leeds, UK, this library features multi-mic coverage and a range of distance positions captured on all included source content. Microphones used include; Sanken, Sennheiser, Neumann and DPA, captured with Sound Devices and Tascam recorders.

    This library includes detailed SoundMiner metadata and utilizes the UCS system for ease of integration into your library.

    Behind the Scenes Video:


    Trench Rammer


  • The cozy natural rhythms of hums, scrapes, splashes and thuds soundtrack the pottery workshop where the earth meets art. Find the true sound of it with Vadi Sound Library.

     

    About Pottery Workshop

     

    84 sounds that are clean, subtle and capture pretty much every object and action of the magic of fire and earth.

    From the rhythmic hum of the spinning potter’s wheel, to the splash of water, the scrape of clay with the tools shaping and smoothing surfaces, listen up for the symphony of creation.

    You will hear the tapping and thud of ceramic pottery, wooden ribs, wire cutters, loop tools, and sponges, all kinds of rummage, and mud actions on the potter’s wheel. The clicking and whirring of mechanisms during firing cycles add a mechanical counterpoint to the workshop’s natural rhythms.

    You will get variations of sounds in different proximity, size, and style of action. This collection is handy due to the clarity and diversity of the raw sounds. They have both Foley and practical usability and room for further sound design.

    You will get intuitive, detailed naming, UCS compatibility and the usual Vadi Sound craft and attention to detail in 84 pristine sounds. Recorded in 24bit-96kHz.WAV format on our favorite Sennheiser MKH 8040 stereo pair and Zoom F6.

     

    Keywords

    Pottery, workshop, electric potter’s wheel, wheel, oven, heat gun, mud, clay, ceramic, glass, wooden, metal, cup, handle, push, pull, move, remove, shape, tap, shake, clank, level, lid, tool, organizer, toolbox, rummage, fiberboard, button, bucket, sponge, plate, Dremel.

     

    What else you may need

    You may also want to check out Drag & Slide for 477 sound files of dragging, sliding, scraping and friction sounds of different objects made of wood, plastic, metal on various surfaces. Our bestseller Crafting & Survival is another good choice to get access to 1000+ survival, gathering, movement and crafting sounds.

  • All files are recorded 32bit, 192 kHz, with Shure KSM 137, Line Audio Omni1, FEL Clippy XLR EM272, Sonorous Objects SO.3 and JrF C-Series Pro+ microphones, Sound Devices MixPre-6 II & Zoom F3 recorders. Library contains wav files of driving, interior and exterior foley, mechanical and electrical sounds. It is also available in UCS.

  • Introducing Devils Bane Trailer, a chilling symphony of horror encapsulated in 533 meticulously crafted sound files, ready to unleash terror upon your audience. Dive into a nightmare realm where every creak, whisper, and shriek is meticulously designed to send shivers down your spine.

    • 533 files
    • 3.5 GB of game audio assets
    • All in 96k 24bit .wav
    • Dark Horror Movie Trailer Sound Effects Library
    17 %
    OFF
  • Birdsong from the countryside of the Swedish rural region of Värmland.

Need specific sound effects? Try a search below:


Binaural or Hard Panning Stereo?

To create a sense of presence in new medias (feeling of being physically present in the virtual scene), besides the usual narrative, tactical and strategic immersion, spatial immersion has been added to immersive design rules. Does it mean we need to position every single sound with HRTF processing then? Well, not necessarily.

In VR, you have the choice between using binaural positioning in place of stereo positioning (and we have the choice for any flat screen games as well now, which would likely open the door to new possibilities and discussion for first-person games).

Though less real, stereo positioning creates a stronger sense of directionality

Binaural positioning allows us to create a more realistic experience, as it accounts for real world physics in the calculations. But due to the long history of using only stereo positioning in games, players are accustomed to it, and using stereo positioning may offer certain advantages over binaural positioning. Though less real, stereo positioning creates a stronger sense of directionality, as the sound is hard panned, making it more obvious where the source is positioned, relative to the listener.

Depending on your game, you might want to use different positioning systems for different components.

If your purpose is to completely immerse the player in the world and create a sense of connection with it, you could use the binaural positioning to establish a sense of realness. For game-play feedbacks, you could possibly use hard-panning, as players are more familiar with it, to enable them to focus their attention on the gameplay, creating immersion with the gameplay rather than the environment. The stronger sense of directionality offered by the stereo positioning provides more obvious cues for the player, allowing them to receive stronger audio feedback.

The decision between using stereo positioning and/or binaural positioning should be a creative one, rather than a technical one, and depends on your game needs per component and situation. It’s all about experimentation and choices.

Preferably, your choices should be made prior starting the audio production of your game, as your positioning system(s) will affect your technical pipeline on the implementation front, as well as the creative approach when building assets and implementing them.
 

 

Creative Audio Language

Your creative audio approach when starting to work on a VR game is then affected by (at least) all of the above. You should ask yourself: When to use audio for multi-sensory integration purpose(s)? How human evolution (realistic approach) and game audio language (gameplay feedbacks and players expectation) impact your creative decision and technical choices and pipeline?

But then, what do you do with any extra-diegetic sources? Let’s take a concrete example with Music. From an end-user perspective, people need to be guided to understand what to think of a scene, what’s the context, what to feel, or what’s the setting. Those are some of the primarily Music purposes. It really depends of your game creative approach, so aim for what works best for your game. VR needs to be interactive, and Players actions need to feel meaningful in the game. Adaptive music needs to be think of carefully and creatively to build something that works both for the VR medium, and the game itself.

On the other hand, players have expectation of realism when playing VR game or experiences, and will notice anything done wrong on the soundscape, anything that could take them out of the experience. Contents production value, audio repetition, and the degree of direct or indirect interactivity to be built in Virtual Worlds can be extremely challenging.

You need to decide your intimate zone, the zone where you could interact with object, the zone where you need to add a lot of details in your sounds, both for positioning and dynamic layering matters.

Interactivity is a big topic of itself for audio in VR. You need to decide your intimate zone, the zone where you could interact with object, the zone where you need to add a lot of details in your sounds, both for positioning and dynamic layering matters. Audio can also be used as an input in VR, which once again reinforce the multi-sensory integration with haptic and visuals, and could possibly suggest taste or smell, if the MSI moment is totally mastered. Concrete example, let’s say you could grab a cigarette in the game, take the controller to your mouth, inhale and exhale would trigger respectively the subtle and visceral sound of a cigarette being burned (plus its visual feedback) and the visual cue of the smoke VFX along the sound of your character exhaling, or the direct sound of yourself (from the VR microphone) going through the acoustic rendering of the virtual environment.
 

To sum up

The main rule when doing audio for VR is that…there are no rules! Creative language is currently being developed by pioneers in the industry, both for the VR medium itself, and the type of games that works well for it.

Rome wasn’t built in a day, and as for all great things, VR and VR audio need time to reach its full potential. We are at the beginning of the VR in terms of technology and development, and there is no perfect recipe when working on VR audio.

Audio is not limited to what’s on screen anymore and should play an even bigger role than in flat screen games, serving all gameplay feedbacks, emotional states, engagement, storytelling, spatial immersion, multi-sensory integration, player’s position relative to the game action (and to some extend body balance).
 

A big thanks to Alex Riviere for his insights on VR audio!

 

Please share this:


 

About Alex Riviere:

Alex Riviere is a French born audio artist that have been working on many projects, from record albums (awarded one platinum and one gold record), commercials, documentaries to video-games, including the Gunjack series, Final Fantasy XIV, Call of Duty Online, Transformers: Rising, Civilization Revolution 2 Plus, etc.

A pioneer in VR Audio, Alex has built his experience developing for all major VR platforms (PSVR, Gear VR, Oculus Rift, Google Daydream, HTC Vive) as Audio Director on the VR titles EVE Gunjack, and Gunjack 2: End of Shift.



 
 
THE WORLD’S EASIEST WAY TO GET INDEPENDENT SOUND EFFECTS:
 
A Sound Effect gives you easy access to an absolutely huge sound effects catalog from a myriad of independent sound creators, all covered by one license agreement - a few highlights:

  • Cinematic & Trailer Sound Effects Abandoned Oil Tank Play Track 327 sounds included $40

    On a recording trip to the smaller Danish port town Struer, I came across an abandoned empty oil tank. Inside I found two giant wires attached to the floor and the ceiling of the tank, almost like a huge 2 stringed double bass. I’m telling you the reverb in there was longer than the Eiffel Tower on a cloudless day !

    On the outside this was just an empty abandoned building, but it turned out it was not empty at all because it was full of sound and a few dead birds. I’m glad I was curious enough to go inside for a closer investigation and not just pass by.

    After several hours of concentrated recording, it was nice to get my ears back out in dry air. I’m happy there are no open sends to Valhalla in real life but experiencing the eternal resonance on the low frequencies from the wires was absolutely amazing.

    Abandoned Oil Tank was recorded with a LOM basic Ucho stereo pair, a Zoom H6 and the LOM Geofon. All files are in 96 kHz/24 bit and contain the original recordings.

    The result is a unique sound effect library including balloon pop impulse responses from a truly unique location that very recently became even more unique since word says that the oil tank is no more and has abandoned this world.

    327 individual sounds – all tagged with Universal Category System (UCS) metadata.

    www.katrineamsler.com

    20 %
    OFF
    Ends 1713131999
  • Electricity Sound Effects Polarity Play Track 975 sounds included $72

    Polarity delivers more than 950 sounds of electricity, science and technology – captured in several locations around the world, from electricity museums to science labs. About 50% of the library is all about electricity, with various types of Jacob’s Ladders, Tesla Coils, Ruhmkorff lamp and all sorts of impactful bursts of energy.

    Then we go through welders, plasma spheres, 3D printers, starting to cover a more broad technology theme – like old phones, telegraphs, dynamo wheels, rotary dials, whirling watchers, alarm, lab centrifuges, something scientists call a roller and a rocker, servo sounds, neon lights, a wimshurst machine and sparklers.

    Many sounds in this section were captured from vintage equipment, from a 1928’s tram to old telephone switchboards, high voltage levers and control surfaces.

    All content was recorded at 192KHz with a Sanken CO100K, a couple of Sennheiser 8040 and a Neumann 81i, translating into final assets that have plenty of ultrasonic content, ready for the most extreme manipulation.

    Bonus: Two extra libraries included for free:
    This library also includes two additional releases from Mattia Cellotto - for free: Crunch Mode delivers 230 crunchy sounds made with a variety of vegetables, fresh bread, pizza crust and a selection of frozen goods. The Borax Experiment gets you 158 squishy, gory, slimy and gooey sounds.
    20 %
    OFF
  • Door Sound Effects Noisy Doors Play Track 44 sounds included $10

    This library contains recordings of eleven different types of doors specifically chosen for their interesting sound. I performed all actions with a lot of detail covering “open/close”, “knocks”, “bangs and thuds”, “handle grab and rattle” and when possible also “crackles and squeaks” (all recorded at various intensities)

    50 %
    OFF
    Ends 1711666799
Explore the full, unique collection here

Latest sound effects libraries:
 
  • Trench Rammers is a one-of-a-kind sound library, created by sound designer Barney Oram. It features recordings of two 20th century Trench Rammers, some of the last operating machines of their kind in the world. The library features 48 files in total, capturing the Trench Rammers using a variety of microphones and perspectives, supplied in 192kHz 24bit WAV file format.

    This library contains multiple recordings of two antique Trench Rammers, the Pegson ER5 and the Warsop Benjo, and also features additional recordings of four antique stationary engines, the Amanco Hired Man (1920s), the JAP 1947, the Lister D (1940s) and the Norman T3000.

    Recorded in Leeds, UK, this library features multi-mic coverage and a range of distance positions captured on all included source content. Microphones used include; Sanken, Sennheiser, Neumann and DPA, captured with Sound Devices and Tascam recorders.

    This library includes detailed SoundMiner metadata and utilizes the UCS system for ease of integration into your library.

    Behind the Scenes Video:


    Trench Rammer


  • The cozy natural rhythms of hums, scrapes, splashes and thuds soundtrack the pottery workshop where the earth meets art. Find the true sound of it with Vadi Sound Library.

     

    About Pottery Workshop

     

    84 sounds that are clean, subtle and capture pretty much every object and action of the magic of fire and earth.

    From the rhythmic hum of the spinning potter’s wheel, to the splash of water, the scrape of clay with the tools shaping and smoothing surfaces, listen up for the symphony of creation.

    You will hear the tapping and thud of ceramic pottery, wooden ribs, wire cutters, loop tools, and sponges, all kinds of rummage, and mud actions on the potter’s wheel. The clicking and whirring of mechanisms during firing cycles add a mechanical counterpoint to the workshop’s natural rhythms.

    You will get variations of sounds in different proximity, size, and style of action. This collection is handy due to the clarity and diversity of the raw sounds. They have both Foley and practical usability and room for further sound design.

    You will get intuitive, detailed naming, UCS compatibility and the usual Vadi Sound craft and attention to detail in 84 pristine sounds. Recorded in 24bit-96kHz.WAV format on our favorite Sennheiser MKH 8040 stereo pair and Zoom F6.

     

    Keywords

    Pottery, workshop, electric potter’s wheel, wheel, oven, heat gun, mud, clay, ceramic, glass, wooden, metal, cup, handle, push, pull, move, remove, shape, tap, shake, clank, level, lid, tool, organizer, toolbox, rummage, fiberboard, button, bucket, sponge, plate, Dremel.

     

    What else you may need

    You may also want to check out Drag & Slide for 477 sound files of dragging, sliding, scraping and friction sounds of different objects made of wood, plastic, metal on various surfaces. Our bestseller Crafting & Survival is another good choice to get access to 1000+ survival, gathering, movement and crafting sounds.

  • All files are recorded 32bit, 192 kHz, with Shure KSM 137, Line Audio Omni1, FEL Clippy XLR EM272, Sonorous Objects SO.3 and JrF C-Series Pro+ microphones, Sound Devices MixPre-6 II & Zoom F3 recorders. Library contains wav files of driving, interior and exterior foley, mechanical and electrical sounds. It is also available in UCS.

  • Introducing Devils Bane Trailer, a chilling symphony of horror encapsulated in 533 meticulously crafted sound files, ready to unleash terror upon your audience. Dive into a nightmare realm where every creak, whisper, and shriek is meticulously designed to send shivers down your spine.

    • 533 files
    • 3.5 GB of game audio assets
    • All in 96k 24bit .wav
    • Dark Horror Movie Trailer Sound Effects Library
    17 %
    OFF
  • Birdsong from the countryside of the Swedish rural region of Värmland.


   

Leave a Reply

Your email address will not be published. Required fields are marked *

HTML tags are not allowed.