Asbjoern Andersen


Virtual reality has been a LONG time coming. I remember reading about it back in the early 90s where it was being hailed as the next big thing – and then 20+ years passed, and nothing much happened, at least in the eye of the general public. However, with devices such as the Oculus Rift nearing completion, it looks like we’re finally getting there.

With this revolutionary new approach to entertainment and immersion just around the corner, what does this mean for the way audio is created and used?

I decided to reach out to Varun Nair for some insights. He’s co-founder of Two Big Ears, an Edinburgh-based company developing audio solutions for games and virtual reality. Here is what he has to say about audio for virtual reality – and the unique possibilities and challenges it offers:

 

Hi Varun, where does virtual reality currently stand in terms of audio?

It is still early days, in terms of technology, workflow and design principles. While the art of designing sound would still be similar (as would the art of image composition, storytelling, etc), virtual reality as a medium needs exploration. In the past two years we have seen some incredible work on the visual front, with thousands of people around the world experimenting with the medium. Comparatively, there’s very little experimentation happening with audio and VR.

The technology is still catching up. Most VR users and developers use headphones, given that it is a personal experience. Binaural audio, which works great over headphones is a perfect fit for VR. Many of the long standing problems with real-time synthesised binaural audio can be overcome thanks to the head and positional tracking sensors and algorithms in VR devices.

Binaural or positional 3D audio makes a huge difference to the virtual reality experience. There was a lot of commercial work around binaural audio in the 90s but it has since retreated into academic and research spheres. Much of our work at Two Big Ears is making such technology (and many more) work really well across game engines, operating systems and hardware devices. It’s not just about having a straight up binaural panner, but also about advanced audio algorithms that can help create a more realistic experience.

From a design perspective, audio for VR is very much an open book. Many of the game design principles (especially those related to FPS games) crumble in virtual reality. I’m curious to see what can be uncovered for audio. For example, I’ve personally found the idea of depicting the scale of objects through sound very intriguing. With binaural audio, the space around the player becomes another parameter to play with. There’s lots that needs to be defined for mixing too. Creating a real-time binaural mix over headphones is very different to mixing on speakers.

How does sound for a 360 degree film work? Nobody is entirely sure! That’s the exciting part!

Outside the scope of games, there’s much work that needs to be done with VR films too. How does sound for a 360 degree film work? Nobody is entirely sure! That’s the exciting part!

 

What are some of the challenges presented by audio in virtual reality?

Designing visuals for VR needs to be treated as a different process (rather than upgrading existing content to support VR) and I’m convinced that the same applies for audio too. A lot of the techniques we take for granted, such as designing sounds for menu screens or HUD elements might not be applicable any more. Quite a few of the VR games are leaning towards user interface elements existing within the virtual space around the player. Could sound be used to draw attention to them?

Another area that has seen almost no discussion is music. How do we treat a background score? Must it be diegetic and exist within the scene? Could it just be a ‘normal’ stereo track in the background? Foley is an interesting area too. Could it be used to help achieve presence in a scene?

Technologically there are challenges too. Game audio needs a real push across the whole stack. We are working hard at it and so are our competitors =) The next few years are going to be an interesting time for audio.
 

As a sound designer for VR, how do you achieve presence and spatialization in your sound design?

For spatialisation we have to rely on technology. Binaural rendering and room modelling makes a massive difference. Although, the spatialisation effect itself can be greatly enhanced by the quality of the sounds. I don’t just mean recording quality, but the timbre and envelope of the sound. As humans, we’re better at localising sounds with higher mid and high frequency content. HDR mixing and ducking can help massively in clearing up a mix.

As for presence — I’m not entirely sure yet. Spatialisation helps, but I feel generative, procedural and dynamic content can greatly improve the experience. I’m personally interested in seeing how input devices for VR evolve. I’m always looking for ways to map sensor data to sound!
 

Battle of the virtual reality platforms
There are several large platforms in development for virtual reality. Best-known is Oculus Rift, a Kickstarter-funded project that ended up being purchased by Facebook. Their latest ‘Crescent Bay’ prototype includes headphones.

Project Morpheus is Sony’s take on virtual reality, announced in March 2014. This platform will be, at least initially, exclusively for the PlayStation 4.

Oculus VR – the company making the Oculus Rift – is also behind the Samsung VR, which offers a novel approach where a Samsung Galaxy Note 4 phone is used as the device display. Final pricing and availability has not been announced for any of the devices.

 

Outside of games, what are some of the areas where you envision VR being used – and how does audio fit into those scenarios?

There’s been a lot of talk lately about education and it is quite easy to see the impact VR can have on it. I also think it can redefine audio post-production, especially with VR movies. We would have to lean towards using game audio tools, but in many ways they can be inadequate for a structured and curated experience.

Being a long time advocator of procedural and generative audio techniques, I think there will be a strong need for those techniques too. Somebody just needs to create the tools that will help us move beyond vehicle, impact, rain and gun-shot models. It is tough work, but it needs getting done. If I had more time in a day, I’d be spending all of it on that!
 


Popular on A Sound Effect right now - article continues below:

 

Latest releases:  
  • Environments Windy Hill Play Track 8 sounds included, 24 mins total $14.99 $10.99

    WINDY HILL is a collection of high quality exterior wind ambiences.
    The library consists of wind gusts through branches, shrub, dry grass, coniferous trees and more.


    Recorded at superb 192 kHz/24 Bit

    2 Super wide AB recordings with a pair of spaced out DPA4060’s

    3 M/S (decoded) with Sennheiser MKH8060 combo + Ambient ATE208

    3 Sony PCM D100 recording

    All SFX have baked-in Soundminer’s meta data.

    RECORDED WITH: Sound Devices MixPre 6 + Sennheiser MKH 8060, ATE208, 2x DPA4060, SONY PCM D100
    EDITED AND MASTERED WITH: RX, Pro Tools, Equality.

    27 %
    OFF
    Ends 1574549999
    Add to cart
  • Environments Transportation – Ambiance Play Track 68 sounds included, 157 mins total $68 $54

    Transportation – Ambiance is a great collection of ambience recordings performed at various transportation scenarios.
    You will find recordings such as a Train car interior, Jumbo jet interior, multiple cars interior under different driving conditions,
    Bus interior, public transportation stations and much more.

    This package includes 68 Samples.
    A total 2h 37m of content.
    Stereo files sampled @ 96Khz / 24bit.

     

    21 %
    OFF
    Ends 1574549999
    Add to cart
  • Destruction & Impact Warehouse Kicks Play Track 50 sounds included $13

    50 powerful industrial kicks tell a story about the role of a man among machines and unprecedented heavy mechanisms. The right kick is 50% success for a track in the true techno style and other experimental sounds. The pack includes oneshots of varying sizes and sonics, with booming subs, persistent tones and reflections. Inspired by illegal raves, parties at abandoned factories and dark basements.

    This compilation is not designed for Deep House, Minimal and EDM regular artists.

    This impacts library can be used as sfx’es for movie, trailers and commercial.

    Add to cart
  • Ambisonics Ambisonic – Transportation Play Track 68 sounds included, 314 mins total $136 $99

    Ambisonic – Transportation is a great collection of ambisonic ambiences recordings performed at various transportation scenarios.
    It is aimed to provide you with great spherical content to wrap your dialogue or main focus content, allowing you to create a conniving and immersing soundtrack.
    You will find recordings such as a Train car interior, Jumbo jet interior, multiple cars interior under different driving conditions,
    Bus interior, public transportation stations and much more.

    This collection is great for post-production, VR/AR interactive sound-design, game developers and any real-time 3D audio engine.
    All files are tagged and categorized for your convenience – supporting multiple tag filtering browsing applications.

    A Sennheiser Ambeo microphone paired with Zoom H8 was used to create this product.

    This package includes 68 Samples – 136 Files.
    A total 2h 37m of content.
    First Order AmbiX B-Format and Stereo @ 96Khz / 24bit.

    Download a Demo here:
    Want to hear an example of the included recordings? Download the B-format Demo Here

    27 %
    OFF
    Ends 1574204399
    Add to cart
  • Foley Footstep Loops II Play Track 663 sounds included $75 $49

    Editing footsteps in audio post-production can be time-consuming. Footstep Loops II is a sound library that delivers a comprehensive kit of footstep sound effects made to ease your daily work.

    The collection contains footstep sounds of various shoes and surfaces, recorded in different paces and edited to continuous but lively 30-second sound loops.

    VARIETY

    The Footstep Loops II Sound Library covers a wide range of different footsteps:

    Barefoot, Socks, Slippers, Flip-Flops, Sneakers on Wood, Sneakers on Concrete, Boots on Wood, Boots on Concrete, Heels on Wood, Heels on Stone;
    Grass, Gravel, Forest, Foliage, Dry Foliage, Stones, Puddle, Mud, Snow;
    Stairs up + down: Wooden Stairs, Metal Stairs, Stone Stairs



    PACE

    Each type of footsteps is available as a set of 13 sound files that represent a range from walking very slowly up to very speedy. Paces are sorted by Footsteps per Minute (FPM):

    Ground Footsteps: from 40 FPM to 160 FPM
    Stairs Footsteps: from 60 FPM to 180 FPM (up) / from 80 FPM to 200 FPM (down)



    LAYERS

    Since all (ground) footstep loops have the same FPM paces, they can be layered easily. E.g. you can add a puddle sound element to sneakers walking on concrete etc.



    CLOTHING

    You can add clothing as a layer to make the movements sound more natural. The sounds of jeans & jacket fit to all ground footsteps. Furthermore, versions with well-balanced clothing sounds of all main footstep loops are already included as ready-to-use files!



    ADDITIONAL ELEMENTS

    Some experimental elements are also included in the library:
    2 layers of floor creaks and one layer that adds the sound of keys in the pocket while walking.



    TIME-COMPRESS

    Paces of the sound loops included in the Footstep Loops II sound library increase in steps of 10 FPM each. If you need a value in between, time-compress the file just a tiny bit – the quality loss is almost inaudible in modern digital audio workstations.



    ONLINE FOOTSTEPS GENERATOR

    To get an impression of what you get with the Footstep Loops II sound library, go HERE and play around with footsteps online.


    • 663 audio files
    • 331 minutes total runtime
    • all files contain meta-data / keywords for easy search
    Video Thumbnail



    All sounds from this library are included in:
    Diversity

    35 %
    OFF
    Ends 1574204399
    Add to cart

Need specific sound effects? Try a search below:
 

If sound designers want to get started doing sound for VR, how do they go about it?

It is tough to design for VR without a device — so getting access to one would be a good place to start :) The audio tools exist and are constantly being improved upon. I think it would be great for sound designers to also jump into the video game development process, with game engines like Unreal Engine 4 that have a low price barrier — picking up those skills can help when communicating with the rest of the game development team too.

We definitely need more sound designers breaking new ground, failing, iterating and pushing the medium forward.

Another approach would be to work with the many VR enthusiasts around the world. There are lots of great indie projects around the web and I’m sure they would all value the contribution of a sound designer.

We’ve found that a number of the VR developers around the world care a lot about audio and they are willing to experiment with new technology and techniques.

We definitely need more sound designers breaking new ground, failing, iterating and pushing the medium forward.

 

Please share this:


 

About Varun Nair
Varun has worked on over 400 projects in films, games and advertising and music. He’s the co-founder of Two Big Ears, a company developing advanced audio solutions for games and VR. They offer 3Dception, a unique binaural audio and room modeling solution for game engines and middleware.
 
 
 
THE WORLD’S EASIEST WAY TO GET INDEPENDENT SOUND EFFECTS:
 
A Sound Effect gives you easy access to an absolutely huge sound effects catalog from a myriad of independent sound creators, all covered by one license agreement - a few highlights:
 
 
  • Destruction & Impact Magic Elements vol.2 Play Track 1438 sounds included $149 $89.40

    Discover the new and complementary opus of Magic Elements sound library!

    ‘Magic Elements’ is a detailed collection of designed & source sounds of organic elements, actions, characters, items, spells & atmospheres in the realm of Magic.
    Volume 2 is complementary to Volume 1 and comes with almost twice the sounds!

    The library is meant to be modular: while being ready-to-use, it delivers useful materials and building blocks that will allow you to create your own sounds.

    • DESIGNED: (772 SOUNDS)
    Volume 2 of this magic saga is comprehensively focused on these 6 elements:
    Energy, Liquid, Foliage, White, Cartoon/Anime, and Generic.

    Each element comes with a variety of spells, actions/movements such as whoosh, impacts, atmospheres, projectiles, along with many variations in intensity and speed.

    All these sounds are carefully designed, edited and meta-tagged.
    The ‘generic’ category contains handy neutral go-to sounds that are useful for any project.

    The ‘cartoon’ will also work well with anime, & manga.

    • SOURCE: (666 SOUNDS)
    The source folder is packed with`many useful files that cover a large spectrum; thoroughly edited and meta-tagged. They will support your creativity with materials such as electricity, liquids, foliage, explosion, whoosh, drones, textures, impacts, chimes, … (file list here)

    Unique & Useful
    This library is made to empower the sound designer’s sonic palette with high quality material but also with unique ‘never heard before’ sound effects.
    A godsend for every sound artist, wether you are looking to start off with a magical toolkit, or you want to enrich your collection with new unique sounds!

    What’s inside:
    Impacts, Whoosh, explosion, fireworks
    Chemical Acid, Bubble, Splash
    Aura, Glyph, Shimmer, Fairy Dust
    Electric Spark, zap, Discharge
    Foliage motion, hit, rustle
    Growing vines, Ent Footstep
    Choir, Harp, Chimes, Bells
    Laboratory, Potion
    Buff, Spell, Hypnotize, Projectile,

    Choose how you want your sounds!
    2 DOWNLOAD OPTIONS PROVIDED:

    • GLUED (Multiple variations of the same sound glued in one file)

    • SEPARATED (Each variations of sound are separated in different files)

    40 %
    OFF
    Ends 1575586799
    Add to cart
  • Sci-Fi Lo-Fi Sci-Fi Play Track 1,000+ sounds included, 431 mins total $30 $15

    Lo-Fi Sci-Fi is a library packed full of characterful, authentic and gritty sounds. It was lovingly created by sound designers Barney Oram and Derek Brown, in homage to classic late 70’s and early 80’s sci-fi movies.
    The collection is comprised of a wide range of sounds, including metal, organics, fire, mechanical, interfaces, creatures, foley and footsteps, abstract, ambience, and much more.

    Lo-Fi Sci-Fi features 620 24bit / 48kHz WAV files in total, including 45 designed sounds and 575 source sounds.

    50 %
    OFF
    Ends 1576105199
    Add to cart
  • Metal Resonant Rusty Door Play Track 100+ sounds included $15

    Recorded at an old bunker constructed near the Chebucto Head lighthouse (Halifax, Nova Scotia) during WWII, now derelict and covered in rust.

    These recordings were a happy accident, discovered on the way to record something else. This large, heavy steel door has been rusting since wartime, and when I noticed it I immediately knew it would have a unique sonic character. When moved, it produces interesting almost musical tones, rich with harmonics and ripe for processing.

    This library includes a variety of sounds generated by interacting with the door in different ways (scraping, punching, and moving at various speeds), and was recorded in mid-side with a Schoeps Mini CMIT + CMC6 MK8 into a Sound Devices 633.

    All sounds are decoded to LR stereo, normalized and processed minimally beyond noise reduction. Files are tagged with metadata in Basehead and stamped in iXML, bext and ID3 formats.

    Add to cart
 
Explore the full, unique collection here

Latest sound effects libraries:
 
  • Environments Windy Hill Play Track 8 sounds included, 24 mins total $14.99 $10.99

    WINDY HILL is a collection of high quality exterior wind ambiences.
    The library consists of wind gusts through branches, shrub, dry grass, coniferous trees and more.


    Recorded at superb 192 kHz/24 Bit

    2 Super wide AB recordings with a pair of spaced out DPA4060’s

    3 M/S (decoded) with Sennheiser MKH8060 combo + Ambient ATE208

    3 Sony PCM D100 recording

    All SFX have baked-in Soundminer’s meta data.

    RECORDED WITH: Sound Devices MixPre 6 + Sennheiser MKH 8060, ATE208, 2x DPA4060, SONY PCM D100
    EDITED AND MASTERED WITH: RX, Pro Tools, Equality.

    27 %
    OFF
    Ends 1574549999
  • Environments Transportation – Ambiance Play Track 68 sounds included, 157 mins total $68 $54

    Transportation – Ambiance is a great collection of ambience recordings performed at various transportation scenarios.
    You will find recordings such as a Train car interior, Jumbo jet interior, multiple cars interior under different driving conditions,
    Bus interior, public transportation stations and much more.

    This package includes 68 Samples.
    A total 2h 37m of content.
    Stereo files sampled @ 96Khz / 24bit.

     

    21 %
    OFF
    Ends 1574549999
  • Destruction & Impact Warehouse Kicks Play Track 50 sounds included $13

    50 powerful industrial kicks tell a story about the role of a man among machines and unprecedented heavy mechanisms. The right kick is 50% success for a track in the true techno style and other experimental sounds. The pack includes oneshots of varying sizes and sonics, with booming subs, persistent tones and reflections. Inspired by illegal raves, parties at abandoned factories and dark basements.

    This compilation is not designed for Deep House, Minimal and EDM regular artists.

    This impacts library can be used as sfx’es for movie, trailers and commercial.

  • Ambisonics Ambisonic – Transportation Play Track 68 sounds included, 314 mins total $136 $99

    Ambisonic – Transportation is a great collection of ambisonic ambiences recordings performed at various transportation scenarios.
    It is aimed to provide you with great spherical content to wrap your dialogue or main focus content, allowing you to create a conniving and immersing soundtrack.
    You will find recordings such as a Train car interior, Jumbo jet interior, multiple cars interior under different driving conditions,
    Bus interior, public transportation stations and much more.

    This collection is great for post-production, VR/AR interactive sound-design, game developers and any real-time 3D audio engine.
    All files are tagged and categorized for your convenience – supporting multiple tag filtering browsing applications.

    A Sennheiser Ambeo microphone paired with Zoom H8 was used to create this product.

    This package includes 68 Samples – 136 Files.
    A total 2h 37m of content.
    First Order AmbiX B-Format and Stereo @ 96Khz / 24bit.

    Download a Demo here:
    Want to hear an example of the included recordings? Download the B-format Demo Here

    27 %
    OFF
    Ends 1574204399
  • Foley Footstep Loops II Play Track 663 sounds included $75 $49

    Editing footsteps in audio post-production can be time-consuming. Footstep Loops II is a sound library that delivers a comprehensive kit of footstep sound effects made to ease your daily work.

    The collection contains footstep sounds of various shoes and surfaces, recorded in different paces and edited to continuous but lively 30-second sound loops.

    VARIETY

    The Footstep Loops II Sound Library covers a wide range of different footsteps:

    Barefoot, Socks, Slippers, Flip-Flops, Sneakers on Wood, Sneakers on Concrete, Boots on Wood, Boots on Concrete, Heels on Wood, Heels on Stone;
    Grass, Gravel, Forest, Foliage, Dry Foliage, Stones, Puddle, Mud, Snow;
    Stairs up + down: Wooden Stairs, Metal Stairs, Stone Stairs



    PACE

    Each type of footsteps is available as a set of 13 sound files that represent a range from walking very slowly up to very speedy. Paces are sorted by Footsteps per Minute (FPM):

    Ground Footsteps: from 40 FPM to 160 FPM
    Stairs Footsteps: from 60 FPM to 180 FPM (up) / from 80 FPM to 200 FPM (down)



    LAYERS

    Since all (ground) footstep loops have the same FPM paces, they can be layered easily. E.g. you can add a puddle sound element to sneakers walking on concrete etc.



    CLOTHING

    You can add clothing as a layer to make the movements sound more natural. The sounds of jeans & jacket fit to all ground footsteps. Furthermore, versions with well-balanced clothing sounds of all main footstep loops are already included as ready-to-use files!



    ADDITIONAL ELEMENTS

    Some experimental elements are also included in the library:
    2 layers of floor creaks and one layer that adds the sound of keys in the pocket while walking.



    TIME-COMPRESS

    Paces of the sound loops included in the Footstep Loops II sound library increase in steps of 10 FPM each. If you need a value in between, time-compress the file just a tiny bit – the quality loss is almost inaudible in modern digital audio workstations.



    ONLINE FOOTSTEPS GENERATOR

    To get an impression of what you get with the Footstep Loops II sound library, go HERE and play around with footsteps online.


    • 663 audio files
    • 331 minutes total runtime
    • all files contain meta-data / keywords for easy search
    Video Thumbnail



    All sounds from this library are included in:
    Diversity

    35 %
    OFF
    Ends 1574204399
 
FOLLOW OR SUBSCRIBE FOR THE LATEST IN FANTASTIC SOUND:
 
                              
 
GET THE MUCH-LOVED A SOUND EFFECT NEWSLETTER:
 
The A Sound Effect newsletter gets you a wealth of exclusive stories and insights
+ free sounds with every issue:
 
Subscribe here for free SFX with every issue

2 thoughts on “Exploring New Sonic Worlds: Sound for Virtual Reality

  1. Speaking of mapping sensor data to sound, you might be interested in the work being done on haptics and haptic-tactile broadcasting, esp. for live virtual reality events where the “feeling” of the remote event can be captured, added to the audio and video VR stream and then decoded and used by the remote VR participant (using the appropriate haptic-tactile hardware such as a platform, vest, gloves, etc.) adding significant presence to the event.

Leave a Reply

Your email address will not be published. Required fields are marked *

HTML tags are not allowed.