Asbjoern Andersen


Virtual reality has been a LONG time coming. I remember reading about it back in the early 90s where it was being hailed as the next big thing – and then 20+ years passed, and nothing much happened, at least in the eye of the general public. However, with devices such as the Oculus Rift nearing completion, it looks like we’re finally getting there.

With this revolutionary new approach to entertainment and immersion just around the corner, what does this mean for the way audio is created and used?

I decided to reach out to Varun Nair for some insights. He’s co-founder of Two Big Ears, an Edinburgh-based company developing audio solutions for games and virtual reality. Here is what he has to say about audio for virtual reality – and the unique possibilities and challenges it offers:

 

Hi Varun, where does virtual reality currently stand in terms of audio?

It is still early days, in terms of technology, workflow and design principles. While the art of designing sound would still be similar (as would the art of image composition, storytelling, etc), virtual reality as a medium needs exploration. In the past two years we have seen some incredible work on the visual front, with thousands of people around the world experimenting with the medium. Comparatively, there’s very little experimentation happening with audio and VR.

The technology is still catching up. Most VR users and developers use headphones, given that it is a personal experience. Binaural audio, which works great over headphones is a perfect fit for VR. Many of the long standing problems with real-time synthesised binaural audio can be overcome thanks to the head and positional tracking sensors and algorithms in VR devices.

Binaural or positional 3D audio makes a huge difference to the virtual reality experience. There was a lot of commercial work around binaural audio in the 90s but it has since retreated into academic and research spheres. Much of our work at Two Big Ears is making such technology (and many more) work really well across game engines, operating systems and hardware devices. It’s not just about having a straight up binaural panner, but also about advanced audio algorithms that can help create a more realistic experience.

From a design perspective, audio for VR is very much an open book. Many of the game design principles (especially those related to FPS games) crumble in virtual reality. I’m curious to see what can be uncovered for audio. For example, I’ve personally found the idea of depicting the scale of objects through sound very intriguing. With binaural audio, the space around the player becomes another parameter to play with. There’s lots that needs to be defined for mixing too. Creating a real-time binaural mix over headphones is very different to mixing on speakers.

How does sound for a 360 degree film work? Nobody is entirely sure! That’s the exciting part!

Outside the scope of games, there’s much work that needs to be done with VR films too. How does sound for a 360 degree film work? Nobody is entirely sure! That’s the exciting part!

 

What are some of the challenges presented by audio in virtual reality?

Designing visuals for VR needs to be treated as a different process (rather than upgrading existing content to support VR) and I’m convinced that the same applies for audio too. A lot of the techniques we take for granted, such as designing sounds for menu screens or HUD elements might not be applicable any more. Quite a few of the VR games are leaning towards user interface elements existing within the virtual space around the player. Could sound be used to draw attention to them?

Another area that has seen almost no discussion is music. How do we treat a background score? Must it be diegetic and exist within the scene? Could it just be a ‘normal’ stereo track in the background? Foley is an interesting area too. Could it be used to help achieve presence in a scene?

Technologically there are challenges too. Game audio needs a real push across the whole stack. We are working hard at it and so are our competitors =) The next few years are going to be an interesting time for audio.
 

As a sound designer for VR, how do you achieve presence and spatialization in your sound design?

For spatialisation we have to rely on technology. Binaural rendering and room modelling makes a massive difference. Although, the spatialisation effect itself can be greatly enhanced by the quality of the sounds. I don’t just mean recording quality, but the timbre and envelope of the sound. As humans, we’re better at localising sounds with higher mid and high frequency content. HDR mixing and ducking can help massively in clearing up a mix.

As for presence — I’m not entirely sure yet. Spatialisation helps, but I feel generative, procedural and dynamic content can greatly improve the experience. I’m personally interested in seeing how input devices for VR evolve. I’m always looking for ways to map sensor data to sound!
 

Battle of the virtual reality platforms
There are several large platforms in development for virtual reality. Best-known is Oculus Rift, a Kickstarter-funded project that ended up being purchased by Facebook. Their latest ‘Crescent Bay’ prototype includes headphones.

Project Morpheus is Sony’s take on virtual reality, announced in March 2014. This platform will be, at least initially, exclusively for the PlayStation 4.

Oculus VR – the company making the Oculus Rift – is also behind the Samsung VR, which offers a novel approach where a Samsung Galaxy Note 4 phone is used as the device display. Final pricing and availability has not been announced for any of the devices.

 

Outside of games, what are some of the areas where you envision VR being used – and how does audio fit into those scenarios?

There’s been a lot of talk lately about education and it is quite easy to see the impact VR can have on it. I also think it can redefine audio post-production, especially with VR movies. We would have to lean towards using game audio tools, but in many ways they can be inadequate for a structured and curated experience.

Being a long time advocator of procedural and generative audio techniques, I think there will be a strong need for those techniques too. Somebody just needs to create the tools that will help us move beyond vehicle, impact, rain and gun-shot models. It is tough work, but it needs getting done. If I had more time in a day, I’d be spending all of it on that!
 


Popular on A Sound Effect right now - article continues below:

 

Latest releases:  
  • 'Organic Lifeform Textures' by Bluezone Corporation is an inspirational sound effect library offering a selection of 99 ( 24 bit / 96 kHz / stereo ) unusual and mysterious sounds : From huge organisms to tiny insects, unexpected groans to invertebrate creatures moving through mud, this SFX library will offer you ultra high quality malleable textures for all your projects. This downloadable sample pack is very usefull for pro editors, film makers, music producers and video game sound designers.

    All sounds were recorded using various sources and processed meticulously using high-end gear. These sound effects have been layered to give you ready-to-use elements. You can easily pitch, modulate, mangle and stretch these sounds to create thousands of variations. 'Organic Lifeform Textures' has been designed to enhance science fiction, mystery, suspense and fantasy video game and scoring projects. Note: The background ambience in the demo is not part of the product but is added free of charge.

    Add to cart
  • Space Cargo Play Track 138 sounds included $22

    Bluezone Corporation presents 'Cargo – Spaceship Sound Effects', a new sci-fi sample library covering a wide range of elements including cockpit and interior ambiences, interface and beep sounds, reactor rumbles, spaceship passby sounds and more. Created with a large selection of high quality recordings and then meticulously layered, this downloadable sample pack will enhance your creative potential with as many as 138 highly usable sounds.

    All sound files are named according to their content and sorted thematically. WAV files are provided as 24 Bit / 96 kHz and sorted in 11 folders. In order to give you ready-to-use sounds for your productions, all samples are royalty-free for all your commercial projects.

    Add to cart
  • Whooshes Cinematic Metal – Titan Play Track Up to 4600 sounds included From: $119 From: $95.20

    CINEMATIC METAL – TITAN pushes big screen sound design beyond its comfort zone. New and unheard HITS, BRAAMS, BOOMS, IMPACTS, STINGERS and much more await you in the comprehensive Construction Kit and devastating Designed edition. Get over 12GB worth of clean, dazzling sound effects – available as individual components as well as layered, processed and ready to use.

    The library is available in two versions & a special bundle:

    CINEMATIC METAL – TITAN CONSTRUCTION KIT

    CINEMATIC METAL – TITAN sounds massive from start to finish. This Construction Kit in particular starts off with an unusual amount of kick. While foley and other, more meticulous applications are certainly possible, the main purpose here is to bring out the big guns and stomp any hint of moderation into the ground.

    LOCK AND LOAD
    Supplement your designs with detailed, high-end metallic sounds. The Construction Kit allows you to build unique, multi-accent effects that not only impress in scale, but also in fidelity, rhythm and character. This library particularly shines in situations, where the visible picture doesn’t necessarily produce the envisioned sound, but warrants its own supernatural emphasis.


    Files: 700 • Sounds: 4200 • Size: 10.9 GB


    CINEMATIC METAL – TITAN DESIGNED:

    Optimized for trailers, action scenes, in-game cinematics and special effects, the Designed edition brings fresh and exciting sounds to the table.

    Discover the force of aggressive, low, soft, processed, clean and tonal HITS, BRAAMS, IMPACTS, SCREECHES, STINGERS and SLAMS.

    DARK AND POWERFUL
    Paint a sense of dread and awe – CINEMATIC METAL – TITAN Designed can evoke fear of the unknown but just as easily kick into rampage mode: Empowered, ready for battle and thirsty for revenge. Find your perfect blend of haunting thriller and jacked, gritty anti-hero.
    Files: 100 • Sounds: 400 • Size: 1.5 GB


    CINEMATIC METAL – TITAN BUNDLE:

    THE BUNDLE – The best of both worlds at a discounted price.
    The Bundle gives you the full sound design power as it contains both – the DESIGNED and the CONSTRUCTION KIT edition at a discounted price.


    Files: 800 • Sounds: 4600 • Size: 12.4 GB
    Included sounds – keywords:

    Braam, impact, rattle, squeak, rumble, clang, crunch, bell, groan, squeak, creak, cymbal, stinger, rusty, gate, container, flam, click, chain, dragging, thwack, bolt, door, train, metallic, iron, sliding, pole, oil drum, scrap, nail, gutter, break, steel, rim, scaffold, crowbar, swell, brass, hook, grate
    20 %
    OFF
    Ends 1563487200
  • Environments New Zealand Ambiences Play Track 27 sounds included, 88 minutes mins total $30

    New Zealand Ambiences is a beautifully crafted ambience library exploring the incredible country and sounds of New Zealand (mainly focusing on the South Island). This library will take you on a sonic adventure exploring the many unique birds and creatures ranging from locations such as: Haast, Queenstown, Lake Paringa, Lake Wakatipu, Te Anau, the Clay Cliffs and Pukaki!

    Recorded at 24 bit/192 kHz you’ll have the flexibility to pitch these ambiences and bird songs to create some truly amazing other worldly atmospheres!

    Add to cart
  • City Life Jamaican Vibrations Vol. 2 Play Track 90 sounds included, 702 mins total From: $30

    Get the sounds and ambiences of Jamaica in this very special sound effects library, featuring almost 12 hours of authentic recordings.

    An Additional Library of Vol 1. the Jamaican Vibrations SFX library includes Walla sounds of Jamaican Patois chatting, urban and village ambiences, high mountain atmospheres, forest sounds, car rides, coffee farm working sounds, wooden house sounds, roomtones, beaches, as well as a luxury hotel visit. So if you're looking for the real sounds of Jamaica, here they are:


Sound Effects Sale

The Sound Effects Summer Sale is now live!

Land huge savings on 100s of excellent sound effects libraries right here

Need specific sound effects? Try a search below:
 

If sound designers want to get started doing sound for VR, how do they go about it?

It is tough to design for VR without a device — so getting access to one would be a good place to start :) The audio tools exist and are constantly being improved upon. I think it would be great for sound designers to also jump into the video game development process, with game engines like Unreal Engine 4 that have a low price barrier — picking up those skills can help when communicating with the rest of the game development team too.

We definitely need more sound designers breaking new ground, failing, iterating and pushing the medium forward.

Another approach would be to work with the many VR enthusiasts around the world. There are lots of great indie projects around the web and I’m sure they would all value the contribution of a sound designer.

We’ve found that a number of the VR developers around the world care a lot about audio and they are willing to experiment with new technology and techniques.

We definitely need more sound designers breaking new ground, failing, iterating and pushing the medium forward.

 

Please share this:


 

About Varun Nair
Varun has worked on over 400 projects in films, games and advertising and music. He’s the co-founder of Two Big Ears, a company developing advanced audio solutions for games and VR. They offer 3Dception, a unique binaural audio and room modeling solution for game engines and middleware.
 
 
 
THE WORLD’S EASIEST WAY TO GET INDEPENDENT SOUND EFFECTS:
 
A Sound Effect gives you easy access to an absolutely huge sound effects catalog from a myriad of independent sound creators, all covered by one license agreement - a few highlights:
 
 
  • Horror Horror Cinema Transitions & Impacts Play Track 100 sounds included, 8 mins total $60 $45

    'Horror Cinema Transitions & Impacts' by Cinematic Sound Design delivers a unique hand-crafted selection of sound effects, designed for Horror films, games, apps, music productions, company logos and more.

    The sounds within this pack have been meticulously designed using both synthesis & recording techniques. In addition to a range of outboard synthesizers, a huge selection of foley sound effects were recorded using Sound Devices 702 field recorder and a pair of Neumann KM184 microphones. All sounds have been put together & processed later through an Apogee Symphony AD/DA.

    25 %
    OFF
    Add to cart
  • The EFX BLACK POWDER Weapons pack is a collection of 19th Century pistol, revolver and rifle weapon sounds.

    The collection includes recordings of 1777 AN IX & Lepage pistols, REMINGTON 1858 & RUGER OLD ARMY revolver, 1777 AN IX, 1822 TBIS, GIBBS, LEE ENFELD, PARKER HALE rifles – as well as a number of salvos from black powder rifles.

    Travel back in time and recreate classic gunfights and shootouts with this collection of vintage weapon sounds.

    Add to cart
  • Mechanical reel2reel Play Track 689 sounds included $20 $10

    Another electromechanical-focused library from SkewSound! Our reel2reel Sound Library features a professional tape machine from the 1980s. We captured all the sounds this little (well, actually quite large and heavy) piece of gear could create.

    DC motors, servos, tape spinning, metal impacts, and more can be found within this library.

    And if you are on the hunt for some huge, analog style mechanical buttons, you should hear what this library has to offer.

    50 %
    OFF
    Add to cart
 
Explore the full, unique collection here

Latest sound effects libraries:
 
  • 'Organic Lifeform Textures' by Bluezone Corporation is an inspirational sound effect library offering a selection of 99 ( 24 bit / 96 kHz / stereo ) unusual and mysterious sounds : From huge organisms to tiny insects, unexpected groans to invertebrate creatures moving through mud, this SFX library will offer you ultra high quality malleable textures for all your projects. This downloadable sample pack is very usefull for pro editors, film makers, music producers and video game sound designers.

    All sounds were recorded using various sources and processed meticulously using high-end gear. These sound effects have been layered to give you ready-to-use elements. You can easily pitch, modulate, mangle and stretch these sounds to create thousands of variations. 'Organic Lifeform Textures' has been designed to enhance science fiction, mystery, suspense and fantasy video game and scoring projects. Note: The background ambience in the demo is not part of the product but is added free of charge.

  • Space Cargo Play Track 138 sounds included $22

    Bluezone Corporation presents 'Cargo – Spaceship Sound Effects', a new sci-fi sample library covering a wide range of elements including cockpit and interior ambiences, interface and beep sounds, reactor rumbles, spaceship passby sounds and more. Created with a large selection of high quality recordings and then meticulously layered, this downloadable sample pack will enhance your creative potential with as many as 138 highly usable sounds.

    All sound files are named according to their content and sorted thematically. WAV files are provided as 24 Bit / 96 kHz and sorted in 11 folders. In order to give you ready-to-use sounds for your productions, all samples are royalty-free for all your commercial projects.

  • Whooshes Cinematic Metal – Titan Play Track Up to 4600 sounds included From: $119 From: $95.20

    CINEMATIC METAL – TITAN pushes big screen sound design beyond its comfort zone. New and unheard HITS, BRAAMS, BOOMS, IMPACTS, STINGERS and much more await you in the comprehensive Construction Kit and devastating Designed edition. Get over 12GB worth of clean, dazzling sound effects – available as individual components as well as layered, processed and ready to use.

    The library is available in two versions & a special bundle:

    CINEMATIC METAL – TITAN CONSTRUCTION KIT

    CINEMATIC METAL – TITAN sounds massive from start to finish. This Construction Kit in particular starts off with an unusual amount of kick. While foley and other, more meticulous applications are certainly possible, the main purpose here is to bring out the big guns and stomp any hint of moderation into the ground.

    LOCK AND LOAD
    Supplement your designs with detailed, high-end metallic sounds. The Construction Kit allows you to build unique, multi-accent effects that not only impress in scale, but also in fidelity, rhythm and character. This library particularly shines in situations, where the visible picture doesn’t necessarily produce the envisioned sound, but warrants its own supernatural emphasis.


    Files: 700 • Sounds: 4200 • Size: 10.9 GB


    CINEMATIC METAL – TITAN DESIGNED:

    Optimized for trailers, action scenes, in-game cinematics and special effects, the Designed edition brings fresh and exciting sounds to the table.

    Discover the force of aggressive, low, soft, processed, clean and tonal HITS, BRAAMS, IMPACTS, SCREECHES, STINGERS and SLAMS.

    DARK AND POWERFUL
    Paint a sense of dread and awe – CINEMATIC METAL – TITAN Designed can evoke fear of the unknown but just as easily kick into rampage mode: Empowered, ready for battle and thirsty for revenge. Find your perfect blend of haunting thriller and jacked, gritty anti-hero.
    Files: 100 • Sounds: 400 • Size: 1.5 GB


    CINEMATIC METAL – TITAN BUNDLE:

    THE BUNDLE – The best of both worlds at a discounted price.
    The Bundle gives you the full sound design power as it contains both – the DESIGNED and the CONSTRUCTION KIT edition at a discounted price.


    Files: 800 • Sounds: 4600 • Size: 12.4 GB
    Included sounds – keywords:

    Braam, impact, rattle, squeak, rumble, clang, crunch, bell, groan, squeak, creak, cymbal, stinger, rusty, gate, container, flam, click, chain, dragging, thwack, bolt, door, train, metallic, iron, sliding, pole, oil drum, scrap, nail, gutter, break, steel, rim, scaffold, crowbar, swell, brass, hook, grate
    20 %
    OFF
    Ends 1563487200
  • Environments New Zealand Ambiences Play Track 27 sounds included, 88 minutes mins total $30

    New Zealand Ambiences is a beautifully crafted ambience library exploring the incredible country and sounds of New Zealand (mainly focusing on the South Island). This library will take you on a sonic adventure exploring the many unique birds and creatures ranging from locations such as: Haast, Queenstown, Lake Paringa, Lake Wakatipu, Te Anau, the Clay Cliffs and Pukaki!

    Recorded at 24 bit/192 kHz you’ll have the flexibility to pitch these ambiences and bird songs to create some truly amazing other worldly atmospheres!

  • City Life Jamaican Vibrations Vol. 2 Play Track 90 sounds included, 702 mins total From: $30

    Get the sounds and ambiences of Jamaica in this very special sound effects library, featuring almost 12 hours of authentic recordings.

    An Additional Library of Vol 1. the Jamaican Vibrations SFX library includes Walla sounds of Jamaican Patois chatting, urban and village ambiences, high mountain atmospheres, forest sounds, car rides, coffee farm working sounds, wooden house sounds, roomtones, beaches, as well as a luxury hotel visit. So if you're looking for the real sounds of Jamaica, here they are:

 
FOLLOW OR SUBSCRIBE FOR THE LATEST IN FANTASTIC SOUND:
 
                              
 
GET THE MUCH-LOVED A SOUND EFFECT NEWSLETTER:
 
The A Sound Effect newsletter gets you a wealth of exclusive stories and insights
+ free sounds with every issue:
 
Subscribe here for free SFX with every issue

2 thoughts on “Exploring New Sonic Worlds: Sound for Virtual Reality

  1. Speaking of mapping sensor data to sound, you might be interested in the work being done on haptics and haptic-tactile broadcasting, esp. for live virtual reality events where the “feeling” of the remote event can be captured, added to the audio and video VR stream and then decoded and used by the remote VR participant (using the appropriate haptic-tactile hardware such as a platform, vest, gloves, etc.) adding significant presence to the event.

Leave a Reply

Your email address will not be published. Required fields are marked *

HTML tags are not allowed.