Asbjoern Andersen


Virtual reality has been a LONG time coming. I remember reading about it back in the early 90s where it was being hailed as the next big thing – and then 20+ years passed, and nothing much happened, at least in the eye of the general public. However, with devices such as the Oculus Rift nearing completion, it looks like we’re finally getting there.

With this revolutionary new approach to entertainment and immersion just around the corner, what does this mean for the way audio is created and used?

I decided to reach out to Varun Nair for some insights. He’s co-founder of Two Big Ears, an Edinburgh-based company developing audio solutions for games and virtual reality. Here is what he has to say about audio for virtual reality – and the unique possibilities and challenges it offers:

 

Hi Varun, where does virtual reality currently stand in terms of audio?

It is still early days, in terms of technology, workflow and design principles. While the art of designing sound would still be similar (as would the art of image composition, storytelling, etc), virtual reality as a medium needs exploration. In the past two years we have seen some incredible work on the visual front, with thousands of people around the world experimenting with the medium. Comparatively, there’s very little experimentation happening with audio and VR.

The technology is still catching up. Most VR users and developers use headphones, given that it is a personal experience. Binaural audio, which works great over headphones is a perfect fit for VR. Many of the long standing problems with real-time synthesised binaural audio can be overcome thanks to the head and positional tracking sensors and algorithms in VR devices.

Binaural or positional 3D audio makes a huge difference to the virtual reality experience. There was a lot of commercial work around binaural audio in the 90s but it has since retreated into academic and research spheres. Much of our work at Two Big Ears is making such technology (and many more) work really well across game engines, operating systems and hardware devices. It’s not just about having a straight up binaural panner, but also about advanced audio algorithms that can help create a more realistic experience.

From a design perspective, audio for VR is very much an open book. Many of the game design principles (especially those related to FPS games) crumble in virtual reality. I’m curious to see what can be uncovered for audio. For example, I’ve personally found the idea of depicting the scale of objects through sound very intriguing. With binaural audio, the space around the player becomes another parameter to play with. There’s lots that needs to be defined for mixing too. Creating a real-time binaural mix over headphones is very different to mixing on speakers.

How does sound for a 360 degree film work? Nobody is entirely sure! That’s the exciting part!

Outside the scope of games, there’s much work that needs to be done with VR films too. How does sound for a 360 degree film work? Nobody is entirely sure! That’s the exciting part!

 

What are some of the challenges presented by audio in virtual reality?

Designing visuals for VR needs to be treated as a different process (rather than upgrading existing content to support VR) and I’m convinced that the same applies for audio too. A lot of the techniques we take for granted, such as designing sounds for menu screens or HUD elements might not be applicable any more. Quite a few of the VR games are leaning towards user interface elements existing within the virtual space around the player. Could sound be used to draw attention to them?

Another area that has seen almost no discussion is music. How do we treat a background score? Must it be diegetic and exist within the scene? Could it just be a ‘normal’ stereo track in the background? Foley is an interesting area too. Could it be used to help achieve presence in a scene?

Technologically there are challenges too. Game audio needs a real push across the whole stack. We are working hard at it and so are our competitors =) The next few years are going to be an interesting time for audio.
 

As a sound designer for VR, how do you achieve presence and spatialization in your sound design?

For spatialisation we have to rely on technology. Binaural rendering and room modelling makes a massive difference. Although, the spatialisation effect itself can be greatly enhanced by the quality of the sounds. I don’t just mean recording quality, but the timbre and envelope of the sound. As humans, we’re better at localising sounds with higher mid and high frequency content. HDR mixing and ducking can help massively in clearing up a mix.

As for presence — I’m not entirely sure yet. Spatialisation helps, but I feel generative, procedural and dynamic content can greatly improve the experience. I’m personally interested in seeing how input devices for VR evolve. I’m always looking for ways to map sensor data to sound!
 

Battle of the virtual reality platforms
There are several large platforms in development for virtual reality. Best-known is Oculus Rift, a Kickstarter-funded project that ended up being purchased by Facebook. Their latest ‘Crescent Bay’ prototype includes headphones.

Project Morpheus is Sony’s take on virtual reality, announced in March 2014. This platform will be, at least initially, exclusively for the PlayStation 4.

Oculus VR – the company making the Oculus Rift – is also behind the Samsung VR, which offers a novel approach where a Samsung Galaxy Note 4 phone is used as the device display. Final pricing and availability has not been announced for any of the devices.

 

Outside of games, what are some of the areas where you envision VR being used – and how does audio fit into those scenarios?

There’s been a lot of talk lately about education and it is quite easy to see the impact VR can have on it. I also think it can redefine audio post-production, especially with VR movies. We would have to lean towards using game audio tools, but in many ways they can be inadequate for a structured and curated experience.

Being a long time advocator of procedural and generative audio techniques, I think there will be a strong need for those techniques too. Somebody just needs to create the tools that will help us move beyond vehicle, impact, rain and gun-shot models. It is tough work, but it needs getting done. If I had more time in a day, I’d be spending all of it on that!
 


Popular on A Sound Effect right now - article continues below:

 

Latest releases:  
  • City Life Shanghai Ambiences Play Track 65 sounds included, 199 mins total $79

    Shanghai is a fascinating and lively city, full of contrast – a mixture of old and new, with towering skyscrapers looming over winding residential lanes, with streets full of rattling old bicycles and electric scooters, with noisy trucks rolling alongside almost silent electric cars.

    The elevated highways that cross the city give certain areas a dense, constant rumble, while the city’s many parks offer a range of sounds, full of people and activity during the day, and generally quite empty at night – offering a diffuse, distant sonic perspective of the environment.

    This library contains many of the everyday sounds of the city, with a variety of on and off-street recordings, interior and exterior walla, park and bird atmospheres, restaurant interiors, traffic, and even a range of metro recordings. While some of this library’s sounds are quite specific to Shanghai, the majority could be used in other large city environments, particularly in but not limited to those cities where Mandarin is the most common language.

    All recordings were made with a pair of Lewitt LCT 540s microphones in an ORTF configuration. These microphones have extremely low self noise and a wide, balanced frequency response, allowing for the detailed capture of a range of sounds, from distant and quiet to close, loud and chaotic, from the deep rumble of engines to the supersonic screeching of brakes. Recorded at a sampling rate of 96kHz, these recordings contain detailed information above 20kHz, expanding the possibilities for sound manipulation by slowing and pitching down the recordings.

    Add to cart
  • World Sounds Airport Soundscapes Play Track 100+ sounds included, 386 mins total $50 $29

    Welcome to A Sound Effect where you can find the largest and most diverse airport sound effects library in the world! This constantly updated library has nearly 6 hours of high-quality recorded sounds from 15 different airports. With over 24GB of professionally edited audio, you'll have the best sounds to work with for your project.

    Featured airports in this library:

    • Athens International Airport Eleftherios Venizelos, Greece
    • Changi Airport Singapore
    • Chiang Mai International Airport, Thailand
    • George Bush Intercontinental Airport Houston, USA
    • Hamad International Airport, Qatar
    • Hong Kong International Airport, Hong Kong
    • Incheon International Airport Seoul, South Korea
    • John F. Kennedy Airport New York City, USA
    • Larnaca International Airport, Cyprus
    • Ngurah Rai International Airport Denpasar, Indonesia
    • Noi Bai International Airport Hanoi, Vietnam
    • Penang International Airport, Malaysia
    • Roberts International Airport Liberia, Costa Rica
    • Suvarnabhumi Airport Bangkok, Thailand
    • Tom Bradley International Airport Los Angeles, USA

    Our library has lush sounds from bustling international airports with thousands of passengers to smaller regional airports with less activity.

    We've captured audio from all areas of these airports. You'll hear sounds from departure halls, arrival halls, inside private lounges, bathrooms, security check with rolling suitcases, beeping sounds from the security scanners, announcements in different languages and passengers waiting at the gates to board the aircraft.

    42 %
    OFF
    Ends 1560981600
    Add to cart
  • City Life Sounds & Ambiences Of London Vol 2 Play Track 101 sounds included, 330 mins total $60 $45

    Following the success of WW Audio's, 'Sound & Ambiences Of London' – here is Volume 2, with even more interesting and useable files for your theatre or film projects.

    Consisting of 101 great, and sometime eclectic recordings from the English capital, and over 10GB in size, this is a library every sound designer needs in their arsenal. It comes with detailed file names plus Soundminer, ID3 v2.3.0 and RIFF INFO metadata embedded.

    In total, this library gets you  5.5 hours of atmospheric London sounds!

    Sounds captured at famous London locations and landmarks, including:

    Bow Bells – Two amazing, over 10 minute recordings from the belfry at St Mary Le Bow Church  • The famous crossing outside Abbey Road Studios, made famous by The Beatles • Stunning overhead plane recordings coming into land at Heathrow airport • Christmas Markets • Crowds outside Buckingham Palace before and after the changing of the guard • Various emergency vehicle sirens • Various markets both internal and external, quiet and busy • Tube train journeys • London Underground ambiences • Various Parks • Great sounding famous London pub ambiences • Train stations • Room tones, including the infamous possible Newgate Prison Cell in the cellar of the Viaduct Tavern Pub • Churches • Crowds in some of London's most popular and obscure streets • General traffic sounds • North, South, East and West London locations • Hotels • London Aquatic Centre • The Royal Exchange • St Bartholomew's Hospital Grounds • Scrooge's Counting House Location • The Tower of London • The River Thames from Execution Dock • Pudding Lane – the location of the start of the Great Fire of London • Various Gardens • Underground car park • Tower Bridge • And much more – check out the complete file list below

    25 %
    OFF
    Ends 1559253600
    Add to cart
  • Environments Singapore City Soundscapes Play Track 122 sounds included, 327 mins total From: $20

    Welcome to A Sound Effect's first ever Singapore Sound Effects library!

    Explore Singapore as you've never heard it before! This massive library includes ambisonic sounds recorded with the new Zoom H3 VR, binaural recordings with the Soundman OKM II Rock Studio and stereo soundscapes recorded with the Zoom F8n, DPA 4060 and several LOM microphones.

    The diversity of Singapore makes this library unique with lush sounds from busy highways, crowded food markets/ hawker centers in many different languages, neighborhood basketball games at night, skateparks, horse races, city rain sounds, the countries super efficient underground train station (MRT), and even loud military fighter jets and helicopters.

    Perhaps you're wanting nature sounds? No problem! This library features sounds from a variety of cicadas recorded at night and birds from the famous Javan Mynah near Mount Faber.

    With nearly six hours of recordings throughout Singapore, we suggest reviewing the metadata and discovering sounds you may not have expected!

    The library is available in 3 variations: A Stereo, AmbiX and special bundle featuring both Stereo and AmbiX files at a big discount.

  • Human The Counting Voice Play Track 167 sounds included $30

    A man with a neutral English accent counts from zero to one hundred. Then counts in hundreds to one thousand. Then counts in thousands to ten thousand. Using the various files you can concatenate the files to read any number between 0-10,999 (and beyond if you fancy some creative editing).

    Useful for all sorts of games, apps and many other projects.

    Add to cart

Need specific sound effects? Try a search below:
 

If sound designers want to get started doing sound for VR, how do they go about it?

It is tough to design for VR without a device — so getting access to one would be a good place to start :) The audio tools exist and are constantly being improved upon. I think it would be great for sound designers to also jump into the video game development process, with game engines like Unreal Engine 4 that have a low price barrier — picking up those skills can help when communicating with the rest of the game development team too.

We definitely need more sound designers breaking new ground, failing, iterating and pushing the medium forward.

Another approach would be to work with the many VR enthusiasts around the world. There are lots of great indie projects around the web and I’m sure they would all value the contribution of a sound designer.

We’ve found that a number of the VR developers around the world care a lot about audio and they are willing to experiment with new technology and techniques.

We definitely need more sound designers breaking new ground, failing, iterating and pushing the medium forward.

 

Please share this:


 

About Varun Nair
Varun has worked on over 400 projects in films, games and advertising and music. He’s the co-founder of Two Big Ears, a company developing advanced audio solutions for games and VR. They offer 3Dception, a unique binaural audio and room modeling solution for game engines and middleware.
 
 
 
THE WORLD’S EASIEST WAY TO GET INDEPENDENT SOUND EFFECTS:
 
A Sound Effect gives you easy access to an absolutely huge sound effects catalog from a myriad of independent sound creators, all covered by one license agreement - a few highlights:
 
 
  • Drones & Moods Blurry Dreams Play Track 203 sounds included $19.99

    BLURRY DREAMS is a collection of 203 unusual, beautiful-sounding, abstract sounds.

    Add a dreamy touch to your next project. Make abstract sounding UIs, implement beautiful and dreamy sounds
    into your next game, setup magical notification on your stream and more. Explore magical granular sounds, abstract evolving drones, anime-like swells and pings and hypnotic transitions.

    Blurry Dreams was created with game developers, sound designers and streamers in mind.

    The collection was created using both hardware and software synths and organic recordings.

    All SFX have baked-in Soundminer metadata.

    RECORDED WITH: Sound Devices MixPre 6 + Senhheiser MKH8060
    EDITED WITH: iZotope RX, Reaktor, Adaptiverb, Modular, FabFilter, Pugment, TC Electronics, Pro Tools.

    Add to cart
  • Looking to blow stuff up? The EFX EXPLOSIONs Collection features a selection of grenade explosion & blast sounds – closeups, distant and muted/muffled explosion recordings included.

    Add to cart
  • Animals & Creatures WINGS Play Track 1444+ sounds included
    Rated 5.00 out of 5
    From: $99

    We are extremely proud to present our first library, WINGS – a one-of-a-kind sound library.

    From tiny insects to small birds, from fairies to dragons, WINGS offers a creative palette with a diverse range of sounds to choose from.

    With over 1400 files (more than 4 GB for the 192 kHz version ) we’re confident you will find the perfect sound.

    When purchasing WINGS you get 2 packs, our Design category that includes 180 files and the Source category that offers more than 1200 sounds. Featuring the very best of our foley sessions.

    All single flaps have been careful edited, allowing for unique speed or rate adjustments.

    Pick your preferred version at the introductory prices below:

 
Explore the full, unique collection here

Latest sound effects libraries:
 
  • City Life Shanghai Ambiences Play Track 65 sounds included, 199 mins total $79

    Shanghai is a fascinating and lively city, full of contrast – a mixture of old and new, with towering skyscrapers looming over winding residential lanes, with streets full of rattling old bicycles and electric scooters, with noisy trucks rolling alongside almost silent electric cars.

    The elevated highways that cross the city give certain areas a dense, constant rumble, while the city’s many parks offer a range of sounds, full of people and activity during the day, and generally quite empty at night – offering a diffuse, distant sonic perspective of the environment.

    This library contains many of the everyday sounds of the city, with a variety of on and off-street recordings, interior and exterior walla, park and bird atmospheres, restaurant interiors, traffic, and even a range of metro recordings. While some of this library’s sounds are quite specific to Shanghai, the majority could be used in other large city environments, particularly in but not limited to those cities where Mandarin is the most common language.

    All recordings were made with a pair of Lewitt LCT 540s microphones in an ORTF configuration. These microphones have extremely low self noise and a wide, balanced frequency response, allowing for the detailed capture of a range of sounds, from distant and quiet to close, loud and chaotic, from the deep rumble of engines to the supersonic screeching of brakes. Recorded at a sampling rate of 96kHz, these recordings contain detailed information above 20kHz, expanding the possibilities for sound manipulation by slowing and pitching down the recordings.

  • World Sounds Airport Soundscapes Play Track 100+ sounds included, 386 mins total $50 $29

    Welcome to A Sound Effect where you can find the largest and most diverse airport sound effects library in the world! This constantly updated library has nearly 6 hours of high-quality recorded sounds from 15 different airports. With over 24GB of professionally edited audio, you'll have the best sounds to work with for your project.

    Featured airports in this library:

    • Athens International Airport Eleftherios Venizelos, Greece
    • Changi Airport Singapore
    • Chiang Mai International Airport, Thailand
    • George Bush Intercontinental Airport Houston, USA
    • Hamad International Airport, Qatar
    • Hong Kong International Airport, Hong Kong
    • Incheon International Airport Seoul, South Korea
    • John F. Kennedy Airport New York City, USA
    • Larnaca International Airport, Cyprus
    • Ngurah Rai International Airport Denpasar, Indonesia
    • Noi Bai International Airport Hanoi, Vietnam
    • Penang International Airport, Malaysia
    • Roberts International Airport Liberia, Costa Rica
    • Suvarnabhumi Airport Bangkok, Thailand
    • Tom Bradley International Airport Los Angeles, USA

    Our library has lush sounds from bustling international airports with thousands of passengers to smaller regional airports with less activity.

    We've captured audio from all areas of these airports. You'll hear sounds from departure halls, arrival halls, inside private lounges, bathrooms, security check with rolling suitcases, beeping sounds from the security scanners, announcements in different languages and passengers waiting at the gates to board the aircraft.

    42 %
    OFF
    Ends 1560981600
  • City Life Sounds & Ambiences Of London Vol 2 Play Track 101 sounds included, 330 mins total $60 $45

    Following the success of WW Audio's, 'Sound & Ambiences Of London' – here is Volume 2, with even more interesting and useable files for your theatre or film projects.

    Consisting of 101 great, and sometime eclectic recordings from the English capital, and over 10GB in size, this is a library every sound designer needs in their arsenal. It comes with detailed file names plus Soundminer, ID3 v2.3.0 and RIFF INFO metadata embedded.

    In total, this library gets you  5.5 hours of atmospheric London sounds!

    Sounds captured at famous London locations and landmarks, including:

    Bow Bells – Two amazing, over 10 minute recordings from the belfry at St Mary Le Bow Church  • The famous crossing outside Abbey Road Studios, made famous by The Beatles • Stunning overhead plane recordings coming into land at Heathrow airport • Christmas Markets • Crowds outside Buckingham Palace before and after the changing of the guard • Various emergency vehicle sirens • Various markets both internal and external, quiet and busy • Tube train journeys • London Underground ambiences • Various Parks • Great sounding famous London pub ambiences • Train stations • Room tones, including the infamous possible Newgate Prison Cell in the cellar of the Viaduct Tavern Pub • Churches • Crowds in some of London's most popular and obscure streets • General traffic sounds • North, South, East and West London locations • Hotels • London Aquatic Centre • The Royal Exchange • St Bartholomew's Hospital Grounds • Scrooge's Counting House Location • The Tower of London • The River Thames from Execution Dock • Pudding Lane – the location of the start of the Great Fire of London • Various Gardens • Underground car park • Tower Bridge • And much more – check out the complete file list below

    25 %
    OFF
    Ends 1559253600
  • Environments Singapore City Soundscapes Play Track 122 sounds included, 327 mins total From: $20

    Welcome to A Sound Effect's first ever Singapore Sound Effects library!

    Explore Singapore as you've never heard it before! This massive library includes ambisonic sounds recorded with the new Zoom H3 VR, binaural recordings with the Soundman OKM II Rock Studio and stereo soundscapes recorded with the Zoom F8n, DPA 4060 and several LOM microphones.

    The diversity of Singapore makes this library unique with lush sounds from busy highways, crowded food markets/ hawker centers in many different languages, neighborhood basketball games at night, skateparks, horse races, city rain sounds, the countries super efficient underground train station (MRT), and even loud military fighter jets and helicopters.

    Perhaps you're wanting nature sounds? No problem! This library features sounds from a variety of cicadas recorded at night and birds from the famous Javan Mynah near Mount Faber.

    With nearly six hours of recordings throughout Singapore, we suggest reviewing the metadata and discovering sounds you may not have expected!

    The library is available in 3 variations: A Stereo, AmbiX and special bundle featuring both Stereo and AmbiX files at a big discount.

  • Human The Counting Voice Play Track 167 sounds included $30

    A man with a neutral English accent counts from zero to one hundred. Then counts in hundreds to one thousand. Then counts in thousands to ten thousand. Using the various files you can concatenate the files to read any number between 0-10,999 (and beyond if you fancy some creative editing).

    Useful for all sorts of games, apps and many other projects.

 
FOLLOW OR SUBSCRIBE FOR THE LATEST IN FANTASTIC SOUND:
 
                              
 
GET THE MUCH-LOVED A SOUND EFFECT NEWSLETTER:
 
The A Sound Effect newsletter gets you a wealth of exclusive stories and insights
+ free sounds with every issue:
 
Subscribe here for free SFX with every issue

2 thoughts on “Exploring New Sonic Worlds: Sound for Virtual Reality

  1. Speaking of mapping sensor data to sound, you might be interested in the work being done on haptics and haptic-tactile broadcasting, esp. for live virtual reality events where the “feeling” of the remote event can be captured, added to the audio and video VR stream and then decoded and used by the remote VR participant (using the appropriate haptic-tactile hardware such as a platform, vest, gloves, etc.) adding significant presence to the event.

Leave a Reply

Your email address will not be published. Required fields are marked *

HTML tags are not allowed.