Asbjoern Andersen


Virtual reality has been a LONG time coming. I remember reading about it back in the early 90s where it was being hailed as the next big thing – and then 20+ years passed, and nothing much happened, at least in the eye of the general public. However, with devices such as the Oculus Rift nearing completion, it looks like we’re finally getting there.

With this revolutionary new approach to entertainment and immersion just around the corner, what does this mean for the way audio is created and used?

I decided to reach out to Varun Nair for some insights. He’s co-founder of Two Big Ears, an Edinburgh-based company developing audio solutions for games and virtual reality. Here is what he has to say about audio for virtual reality – and the unique possibilities and challenges it offers:

 

Hi Varun, where does virtual reality currently stand in terms of audio?

It is still early days, in terms of technology, workflow and design principles. While the art of designing sound would still be similar (as would the art of image composition, storytelling, etc), virtual reality as a medium needs exploration. In the past two years we have seen some incredible work on the visual front, with thousands of people around the world experimenting with the medium. Comparatively, there’s very little experimentation happening with audio and VR.

The technology is still catching up. Most VR users and developers use headphones, given that it is a personal experience. Binaural audio, which works great over headphones is a perfect fit for VR. Many of the long standing problems with real-time synthesised binaural audio can be overcome thanks to the head and positional tracking sensors and algorithms in VR devices.

Binaural or positional 3D audio makes a huge difference to the virtual reality experience. There was a lot of commercial work around binaural audio in the 90s but it has since retreated into academic and research spheres. Much of our work at Two Big Ears is making such technology (and many more) work really well across game engines, operating systems and hardware devices. It’s not just about having a straight up binaural panner, but also about advanced audio algorithms that can help create a more realistic experience.

From a design perspective, audio for VR is very much an open book. Many of the game design principles (especially those related to FPS games) crumble in virtual reality. I’m curious to see what can be uncovered for audio. For example, I’ve personally found the idea of depicting the scale of objects through sound very intriguing. With binaural audio, the space around the player becomes another parameter to play with. There’s lots that needs to be defined for mixing too. Creating a real-time binaural mix over headphones is very different to mixing on speakers.

How does sound for a 360 degree film work? Nobody is entirely sure! That’s the exciting part!

Outside the scope of games, there’s much work that needs to be done with VR films too. How does sound for a 360 degree film work? Nobody is entirely sure! That’s the exciting part!

 

What are some of the challenges presented by audio in virtual reality?

Designing visuals for VR needs to be treated as a different process (rather than upgrading existing content to support VR) and I’m convinced that the same applies for audio too. A lot of the techniques we take for granted, such as designing sounds for menu screens or HUD elements might not be applicable any more. Quite a few of the VR games are leaning towards user interface elements existing within the virtual space around the player. Could sound be used to draw attention to them?

Another area that has seen almost no discussion is music. How do we treat a background score? Must it be diegetic and exist within the scene? Could it just be a ‘normal’ stereo track in the background? Foley is an interesting area too. Could it be used to help achieve presence in a scene?

Technologically there are challenges too. Game audio needs a real push across the whole stack. We are working hard at it and so are our competitors =) The next few years are going to be an interesting time for audio.
 

As a sound designer for VR, how do you achieve presence and spatialization in your sound design?

For spatialisation we have to rely on technology. Binaural rendering and room modelling makes a massive difference. Although, the spatialisation effect itself can be greatly enhanced by the quality of the sounds. I don’t just mean recording quality, but the timbre and envelope of the sound. As humans, we’re better at localising sounds with higher mid and high frequency content. HDR mixing and ducking can help massively in clearing up a mix.

As for presence — I’m not entirely sure yet. Spatialisation helps, but I feel generative, procedural and dynamic content can greatly improve the experience. I’m personally interested in seeing how input devices for VR evolve. I’m always looking for ways to map sensor data to sound!
 

Battle of the virtual reality platforms
There are several large platforms in development for virtual reality. Best-known is Oculus Rift, a Kickstarter-funded project that ended up being purchased by Facebook. Their latest ‘Crescent Bay’ prototype includes headphones.

Project Morpheus is Sony’s take on virtual reality, announced in March 2014. This platform will be, at least initially, exclusively for the PlayStation 4.

Oculus VR – the company making the Oculus Rift – is also behind the Samsung VR, which offers a novel approach where a Samsung Galaxy Note 4 phone is used as the device display. Final pricing and availability has not been announced for any of the devices.

 

Outside of games, what are some of the areas where you envision VR being used – and how does audio fit into those scenarios?

There’s been a lot of talk lately about education and it is quite easy to see the impact VR can have on it. I also think it can redefine audio post-production, especially with VR movies. We would have to lean towards using game audio tools, but in many ways they can be inadequate for a structured and curated experience.

Being a long time advocator of procedural and generative audio techniques, I think there will be a strong need for those techniques too. Somebody just needs to create the tools that will help us move beyond vehicle, impact, rain and gun-shot models. It is tough work, but it needs getting done. If I had more time in a day, I’d be spending all of it on that!
 


Popular on A Sound Effect right now - article continues below:

 

Latest releases:  
  • Want that sound that’s made between the Optometrist asking if a or b is better? Want to hear what it sounds like when your eyes are being scanned and tested? Hear your glasses being adjusted before you get to wear them? The optometrist equipment library has all of this ready for you to use. This small boutique library covers a range of equipment used everyday to help optometrists figure out how well we see. Weather you’re using the machine noises to cut an optometrist scene or if you want to take the many mechanical sounds and design them into something other worldly it’s all there for you.

    Add to cart
  • This mini -library contains sounds from a reptile shelter. They take in injured, negelcted or abandond reptiles and take care of them. They finance through a small zoo and due to corona had significant loss in income. 50% of the sales of this library will directly be donated to the shelter. For more information check: https://www.reptilien-auffangstation.de/

    The recordings contain sounds from 14 different species. With the focus on reptiles, it also features amphibians, mammals and birds.
    Ever needed a realistic hiss of an Arabian Cobra, the roar of a Nile crocodile, the sweet chirping of a Djungarian hamster, or the whistling sound of a Reticulated Phyton coming out of cold water? Here you go. With two mic positions in 192kHz I followed around the tails of big (Diamondback-) and small (Massasauga-) Rattlesnakes. I even recorded the purr of giant lizards (monitors).

    If you don’t need these specific animals, these growls, sizzling, and calls are a great source sounds for dragons, dinosaurs, monsters and other beasts. Also great to add as a layer in vehicle sound design or as sweeteners to add the noise of a dart frog to your jungle atmo.
    These recordings are hard to come by and my mics were nearly hit by poison, that crystalizes and stays venomous for years to come as well as they got nearly eaten by a crocodile (was very scary indeed with me at the other end of the (way too short) boom pole)

    The best part, the handling with the animals was very thoughtful. No animals were tortured or stressed. They had some great animal voices at the shelter, that didn’t want to speak during the recording session, so we let them sleep and didn’t take any animals away from their hide.

    Full List: Arabian Cobra, Diamondback Rattlesnake, Diamond Dove, Djungarian Hamster, Domestic Chicken, Domestic Goat, Nile Crocodile, Massasauga Rattler, Papua Monitor, Puff Adder, Reticulated Phyton, Ring Tailed Lemur, Yellow-Banded Poison Dart Frog and, well, a fish tank with my hydrophone with some weird electronic sounds from the heating.

    21 %
    OFF
    Ends 1591135199
    Add to cart
  • Horror Giant Pinecones Play Track 500+ sounds included, 44 mins total $79

    • In Giant Pinecones, get a visceral collection of scraped and eviscerated pinecones from the gray pine trees of Northern California. Scrape the razor-sharp hooks of the cone petals and hear guttural scrapes crackling with energy. Hear rolling cones popping and fluttering with rich stuttering tones. Hear fully open cone pedals squeaking with woody vocalizations like supernatural animals and hardy crunches filled with organic grit.
    • This library offers you an extensive collection of sounds from a unique organic sound source. Digger pinecone sounds are incredibly soft and intimate in real life, but when recorded from two inches they morph into a unique wooden sound source brimming with powerful glitchy and stuttering textures.

    2% FOR THE ENVIRONMENT & CARBON NEUTRAL:
    • Two percent of the price of this library is donated to an environmental cause, as an “artist royalty” for the planet!
    • Carbon offset credits were purchased to offset my field recording travel for this library.

    KEY FEATURES:
    • Woody vocalizations
    • Rolling, scraping, and stuttering textures
    • Visceral and guttural scrapes
    • Fluttering and popping textures
    • Rich crunches
    • Chalkboard-like squeaks and squeals
    FILE LIST & METADATA:
    • View larger version or Download CSV
    • A spectrogram is included for each audio file. Double click on the photo to enlarge.
    MORE INFO:
    • Read 40+ testimonials for Thomas Rex Beverly Audio
    • Read my Field Recording Mastering Rules and learn more about how these recordings were mastered.
    • Browse the Library Info Master List to compare specs on all my libraries.
    • Browse the Metadata Master List to search my entire catalog.
    • MD5 and SHA 256 Checksums are included for each zip file in my catalog. Use these hashes to check the integrity of your downloaded files.
    GEAR USED:
    • Sennheiser MKH8040 and MKH30 in MS
    • Sound Devices MixPre-6
    Add to cart
  • Recording of the American 2017 Polaris Ranger EV. Powered by an electric utility vehicle’s 48-volt high-efficiency AC-induction motor.


    The American 2017 Polaris Industries Ranger EV sound collection shares 44 sounds in 3.49 gigabytes of audio. It showcases the sound of an electric utility vehicle’s 48-volt high-efficiency AC-induction motor in 4 channels with 2 custom stereo mixes.

    The sound pack includes 4 synchronized takes of onboard driving. 4 channels of audio capture the engine and onboard perspectives, with 2 custom stereo mixes provided. Performances include driving slow and fast, with steady RPMs and ramps, starting, stopping, and more.

    The package includes Pro Tools and Reaper mixing sessions, full professional metadata, and metadata import files in 7 languages.

    Add to cart
  • A crush on music

    Distortion and saturation play a very important role in music production. From subtle, clean and warm tube or tape saturation to the wildest multiband guitar amp effects: FabFilter Saturn 2 delivers.

    Saturn 2 introduces a host of new features such as a redesigned interface with modulation visualization, new subtle saturation and linear phase processing for mastering, many new distortion styles, and more.


    Warmth, harmonics, color and dynamics

    FabFilter Saturn 2 offers a range of different high quality distortion models, inspired by the vintage sound of tubes, tape, transformers and guitar amps. In addition, you get five creative FX distortion styles to mangle your sounds in weird and unexpected ways.

    With its multiband design and per-band feedback, dynamics, drive, tone and modulation options, Saturn 2 will bring a unique flavor to your music.

    Bring your sounds to life

    Add life and depth to your music using the extensive modulation section. By applying subtle modulation to crossover frequencies, dynamics, band levels or tone controls, great warmth and definition can be achieved.

    With all the XLFOs, EGs, XY controllers/sliders, envelope followers and MIDI sources you will ever need, you get practically unlimited modulation possibilities. Creating new modulation connections could not be easier: just drag and drop. And Saturn 2 visualizes all modulation in real-time to show exactly what’s going on.

    FabFilter goodies

    Finally, FabFilter Saturn 2 contains all the usual FabFilter goodies: perfectly tuned knobs, MIDI Learn, Smart Parameter Interpolation for smooth parameter transitions, interface resizing and full screen mode, support for Avid control surfaces, GPU-powered graphics acceleration, extensive help with interactive help hints, SSE optimization, and much more.

    Add to cart


Need specific sound effects? Try a search below:
 

If sound designers want to get started doing sound for VR, how do they go about it?

It is tough to design for VR without a device — so getting access to one would be a good place to start :) The audio tools exist and are constantly being improved upon. I think it would be great for sound designers to also jump into the video game development process, with game engines like Unreal Engine 4 that have a low price barrier — picking up those skills can help when communicating with the rest of the game development team too.

We definitely need more sound designers breaking new ground, failing, iterating and pushing the medium forward.

Another approach would be to work with the many VR enthusiasts around the world. There are lots of great indie projects around the web and I’m sure they would all value the contribution of a sound designer.

We’ve found that a number of the VR developers around the world care a lot about audio and they are willing to experiment with new technology and techniques.

We definitely need more sound designers breaking new ground, failing, iterating and pushing the medium forward.

 

Please share this:


 

About Varun Nair
Varun has worked on over 400 projects in films, games and advertising and music. He’s the co-founder of Two Big Ears, a company developing advanced audio solutions for games and VR. They offer 3Dception, a unique binaural audio and room modeling solution for game engines and middleware.
 
 
 
THE WORLD’S EASIEST WAY TO GET INDEPENDENT SOUND EFFECTS:
 
A Sound Effect gives you easy access to an absolutely huge sound effects catalog from a myriad of independent sound creators, all covered by one license agreement - a few highlights:
 
 
  • Genres Vintage Anime SFX Play Track 350+ sounds included $69

    The Vintage Anime Sound Effects Library brings all of the excitement of your favorite Japanese animated series to your fingertips. Inspired by classic cartoons from the 80’s and 90’s, these recognizable and versatile sounds will instantly enhance any FX collection. Vintage synths were used to create the auras, beams, mecha blasters, atmospheres, magic spells, guns, sonic blasts and explosives that makeup this pack of over 350+ custom 24bit/96khz .WAV files. Perfect for film, video games, podcasts and any project that could benefit from a power up!

    Both designed sounds and source recordings:
    • Classic anime sfx from the 80’s and 90’s

    • Auras, mecha, beams, blasters, spells, explosives and more! 350+ sounds!

    • Tons of source material for experimentation

    • Expert crafted metadata

    • Vintage Anime PDF

    Add to cart
  • Human Swordfighter Play Track 479 sounds included $25 $12.50

    Swordfighter is a robust package with sharp sounding swords, heaps of variations and all the extras you need to make a fight come alive. Build unique sword swings with various hits, swooshes, schings, different fighter vocals and impacts on various surfaces. All up there are 137 sword sounds, 93 surface impact sounds, 15 knife throwing sounds, 48 swooshes and 180 fighter vocals.

    This version includes two sub-folders: one optimised for a film & TV workflow and the other optimised for video games workflow. Plus a few bonus sounds of a charging army.

    50 %
    OFF
    Add to cart
  • Liquidation is a liquid texture library covering everything from water to slime to fizzes and bubbles. Source material in this library includes wet slimy pasta, giant water balloons, large containers being submerged underwater, leather fizzing in a chemical bath, pool water splashes and more. Plus it was all recorded at 192kHz/24bit so you can really stretch and thrash these sounds – and the vast majority of the sound files are in stereo. Whether you're designing an underwater adventure or an alien autopsy, this library is full of complimentary textures to layer together.

    50 %
    OFF
    Add to cart
 
Explore the full, unique collection here

Latest sound effects libraries:
 
  • Want that sound that’s made between the Optometrist asking if a or b is better? Want to hear what it sounds like when your eyes are being scanned and tested? Hear your glasses being adjusted before you get to wear them? The optometrist equipment library has all of this ready for you to use. This small boutique library covers a range of equipment used everyday to help optometrists figure out how well we see. Weather you’re using the machine noises to cut an optometrist scene or if you want to take the many mechanical sounds and design them into something other worldly it’s all there for you.

  • This mini -library contains sounds from a reptile shelter. They take in injured, negelcted or abandond reptiles and take care of them. They finance through a small zoo and due to corona had significant loss in income. 50% of the sales of this library will directly be donated to the shelter. For more information check: https://www.reptilien-auffangstation.de/

    The recordings contain sounds from 14 different species. With the focus on reptiles, it also features amphibians, mammals and birds.
    Ever needed a realistic hiss of an Arabian Cobra, the roar of a Nile crocodile, the sweet chirping of a Djungarian hamster, or the whistling sound of a Reticulated Phyton coming out of cold water? Here you go. With two mic positions in 192kHz I followed around the tails of big (Diamondback-) and small (Massasauga-) Rattlesnakes. I even recorded the purr of giant lizards (monitors).

    If you don’t need these specific animals, these growls, sizzling, and calls are a great source sounds for dragons, dinosaurs, monsters and other beasts. Also great to add as a layer in vehicle sound design or as sweeteners to add the noise of a dart frog to your jungle atmo.
    These recordings are hard to come by and my mics were nearly hit by poison, that crystalizes and stays venomous for years to come as well as they got nearly eaten by a crocodile (was very scary indeed with me at the other end of the (way too short) boom pole)

    The best part, the handling with the animals was very thoughtful. No animals were tortured or stressed. They had some great animal voices at the shelter, that didn’t want to speak during the recording session, so we let them sleep and didn’t take any animals away from their hide.

    Full List: Arabian Cobra, Diamondback Rattlesnake, Diamond Dove, Djungarian Hamster, Domestic Chicken, Domestic Goat, Nile Crocodile, Massasauga Rattler, Papua Monitor, Puff Adder, Reticulated Phyton, Ring Tailed Lemur, Yellow-Banded Poison Dart Frog and, well, a fish tank with my hydrophone with some weird electronic sounds from the heating.

    21 %
    OFF
    Ends 1591135199
  • Horror Giant Pinecones Play Track 500+ sounds included, 44 mins total $79

    • In Giant Pinecones, get a visceral collection of scraped and eviscerated pinecones from the gray pine trees of Northern California. Scrape the razor-sharp hooks of the cone petals and hear guttural scrapes crackling with energy. Hear rolling cones popping and fluttering with rich stuttering tones. Hear fully open cone pedals squeaking with woody vocalizations like supernatural animals and hardy crunches filled with organic grit.
    • This library offers you an extensive collection of sounds from a unique organic sound source. Digger pinecone sounds are incredibly soft and intimate in real life, but when recorded from two inches they morph into a unique wooden sound source brimming with powerful glitchy and stuttering textures.

    2% FOR THE ENVIRONMENT & CARBON NEUTRAL:
    • Two percent of the price of this library is donated to an environmental cause, as an “artist royalty” for the planet!
    • Carbon offset credits were purchased to offset my field recording travel for this library.

    KEY FEATURES:
    • Woody vocalizations
    • Rolling, scraping, and stuttering textures
    • Visceral and guttural scrapes
    • Fluttering and popping textures
    • Rich crunches
    • Chalkboard-like squeaks and squeals
    FILE LIST & METADATA:
    • View larger version or Download CSV
    • A spectrogram is included for each audio file. Double click on the photo to enlarge.
    MORE INFO:
    • Read 40+ testimonials for Thomas Rex Beverly Audio
    • Read my Field Recording Mastering Rules and learn more about how these recordings were mastered.
    • Browse the Library Info Master List to compare specs on all my libraries.
    • Browse the Metadata Master List to search my entire catalog.
    • MD5 and SHA 256 Checksums are included for each zip file in my catalog. Use these hashes to check the integrity of your downloaded files.
    GEAR USED:
    • Sennheiser MKH8040 and MKH30 in MS
    • Sound Devices MixPre-6
  • Recording of the American 2017 Polaris Ranger EV. Powered by an electric utility vehicle’s 48-volt high-efficiency AC-induction motor.


    The American 2017 Polaris Industries Ranger EV sound collection shares 44 sounds in 3.49 gigabytes of audio. It showcases the sound of an electric utility vehicle’s 48-volt high-efficiency AC-induction motor in 4 channels with 2 custom stereo mixes.

    The sound pack includes 4 synchronized takes of onboard driving. 4 channels of audio capture the engine and onboard perspectives, with 2 custom stereo mixes provided. Performances include driving slow and fast, with steady RPMs and ramps, starting, stopping, and more.

    The package includes Pro Tools and Reaper mixing sessions, full professional metadata, and metadata import files in 7 languages.

  • A crush on music

    Distortion and saturation play a very important role in music production. From subtle, clean and warm tube or tape saturation to the wildest multiband guitar amp effects: FabFilter Saturn 2 delivers.

    Saturn 2 introduces a host of new features such as a redesigned interface with modulation visualization, new subtle saturation and linear phase processing for mastering, many new distortion styles, and more.


    Warmth, harmonics, color and dynamics

    FabFilter Saturn 2 offers a range of different high quality distortion models, inspired by the vintage sound of tubes, tape, transformers and guitar amps. In addition, you get five creative FX distortion styles to mangle your sounds in weird and unexpected ways.

    With its multiband design and per-band feedback, dynamics, drive, tone and modulation options, Saturn 2 will bring a unique flavor to your music.

    Bring your sounds to life

    Add life and depth to your music using the extensive modulation section. By applying subtle modulation to crossover frequencies, dynamics, band levels or tone controls, great warmth and definition can be achieved.

    With all the XLFOs, EGs, XY controllers/sliders, envelope followers and MIDI sources you will ever need, you get practically unlimited modulation possibilities. Creating new modulation connections could not be easier: just drag and drop. And Saturn 2 visualizes all modulation in real-time to show exactly what’s going on.

    FabFilter goodies

    Finally, FabFilter Saturn 2 contains all the usual FabFilter goodies: perfectly tuned knobs, MIDI Learn, Smart Parameter Interpolation for smooth parameter transitions, interface resizing and full screen mode, support for Avid control surfaces, GPU-powered graphics acceleration, extensive help with interactive help hints, SSE optimization, and much more.

 
FOLLOW OR SUBSCRIBE FOR THE LATEST IN FANTASTIC SOUND:
 
                              
 
GET THE MUCH-LOVED A SOUND EFFECT NEWSLETTER:
 
The A Sound Effect newsletter gets you a wealth of exclusive stories and insights
+ free sounds with every issue:
 
Subscribe here for free SFX with every issue

2 thoughts on “Exploring New Sonic Worlds: Sound for Virtual Reality

  1. Speaking of mapping sensor data to sound, you might be interested in the work being done on haptics and haptic-tactile broadcasting, esp. for live virtual reality events where the “feeling” of the remote event can be captured, added to the audio and video VR stream and then decoded and used by the remote VR participant (using the appropriate haptic-tactile hardware such as a platform, vest, gloves, etc.) adding significant presence to the event.

Leave a Reply

Your email address will not be published. Required fields are marked *

HTML tags are not allowed.