Asbjoern Andersen


Virtual reality has been a LONG time coming. I remember reading about it back in the early 90s where it was being hailed as the next big thing – and then 20+ years passed, and nothing much happened, at least in the eye of the general public. However, with devices such as the Oculus Rift nearing completion, it looks like we’re finally getting there.

With this revolutionary new approach to entertainment and immersion just around the corner, what does this mean for the way audio is created and used?

I decided to reach out to Varun Nair for some insights. He’s co-founder of Two Big Ears, an Edinburgh-based company developing audio solutions for games and virtual reality. Here is what he has to say about audio for virtual reality – and the unique possibilities and challenges it offers:

 

Hi Varun, where does virtual reality currently stand in terms of audio?

It is still early days, in terms of technology, workflow and design principles. While the art of designing sound would still be similar (as would the art of image composition, storytelling, etc), virtual reality as a medium needs exploration. In the past two years we have seen some incredible work on the visual front, with thousands of people around the world experimenting with the medium. Comparatively, there’s very little experimentation happening with audio and VR.

The technology is still catching up. Most VR users and developers use headphones, given that it is a personal experience. Binaural audio, which works great over headphones is a perfect fit for VR. Many of the long standing problems with real-time synthesised binaural audio can be overcome thanks to the head and positional tracking sensors and algorithms in VR devices.

Binaural or positional 3D audio makes a huge difference to the virtual reality experience. There was a lot of commercial work around binaural audio in the 90s but it has since retreated into academic and research spheres. Much of our work at Two Big Ears is making such technology (and many more) work really well across game engines, operating systems and hardware devices. It’s not just about having a straight up binaural panner, but also about advanced audio algorithms that can help create a more realistic experience.

From a design perspective, audio for VR is very much an open book. Many of the game design principles (especially those related to FPS games) crumble in virtual reality. I’m curious to see what can be uncovered for audio. For example, I’ve personally found the idea of depicting the scale of objects through sound very intriguing. With binaural audio, the space around the player becomes another parameter to play with. There’s lots that needs to be defined for mixing too. Creating a real-time binaural mix over headphones is very different to mixing on speakers.

How does sound for a 360 degree film work? Nobody is entirely sure! That’s the exciting part!

Outside the scope of games, there’s much work that needs to be done with VR films too. How does sound for a 360 degree film work? Nobody is entirely sure! That’s the exciting part!

 

What are some of the challenges presented by audio in virtual reality?

Designing visuals for VR needs to be treated as a different process (rather than upgrading existing content to support VR) and I’m convinced that the same applies for audio too. A lot of the techniques we take for granted, such as designing sounds for menu screens or HUD elements might not be applicable any more. Quite a few of the VR games are leaning towards user interface elements existing within the virtual space around the player. Could sound be used to draw attention to them?

Another area that has seen almost no discussion is music. How do we treat a background score? Must it be diegetic and exist within the scene? Could it just be a ‘normal’ stereo track in the background? Foley is an interesting area too. Could it be used to help achieve presence in a scene?

Technologically there are challenges too. Game audio needs a real push across the whole stack. We are working hard at it and so are our competitors =) The next few years are going to be an interesting time for audio.
 

As a sound designer for VR, how do you achieve presence and spatialization in your sound design?

For spatialisation we have to rely on technology. Binaural rendering and room modelling makes a massive difference. Although, the spatialisation effect itself can be greatly enhanced by the quality of the sounds. I don’t just mean recording quality, but the timbre and envelope of the sound. As humans, we’re better at localising sounds with higher mid and high frequency content. HDR mixing and ducking can help massively in clearing up a mix.

As for presence — I’m not entirely sure yet. Spatialisation helps, but I feel generative, procedural and dynamic content can greatly improve the experience. I’m personally interested in seeing how input devices for VR evolve. I’m always looking for ways to map sensor data to sound!
 

Battle of the virtual reality platforms
There are several large platforms in development for virtual reality. Best-known is Oculus Rift, a Kickstarter-funded project that ended up being purchased by Facebook. Their latest ‘Crescent Bay’ prototype includes headphones.

Project Morpheus is Sony’s take on virtual reality, announced in March 2014. This platform will be, at least initially, exclusively for the PlayStation 4.

Oculus VR – the company making the Oculus Rift – is also behind the Samsung VR, which offers a novel approach where a Samsung Galaxy Note 4 phone is used as the device display. Final pricing and availability has not been announced for any of the devices.

 

Outside of games, what are some of the areas where you envision VR being used – and how does audio fit into those scenarios?

There’s been a lot of talk lately about education and it is quite easy to see the impact VR can have on it. I also think it can redefine audio post-production, especially with VR movies. We would have to lean towards using game audio tools, but in many ways they can be inadequate for a structured and curated experience.

Being a long time advocator of procedural and generative audio techniques, I think there will be a strong need for those techniques too. Somebody just needs to create the tools that will help us move beyond vehicle, impact, rain and gun-shot models. It is tough work, but it needs getting done. If I had more time in a day, I’d be spending all of it on that!
 


Popular on A Sound Effect right now - article continues below:

 
  • Horror Distorted Perception Play Track 146 sounds included, 44 mins total $30.25 $9.68 incl. vat

    Need to create a tense atmosphere to induce madness, craziness and fear in your Horror or Sci-fi project? DISTORTED PERCEPTION is the library for you.

    Abnormal melodies, strange sound movements, transitions and distorted cinematic impacts are some of the things you will find in this collection; sounds with a distorted, saturated and tense feeling to grab people’s attention and provoke fear, tension and suspense in them.

    Yu can also use the sounds to enhance musical productions or create some nasty and crazy musical moments, thanks to a nice selection of percussive elements. The library contains two musical compositions using only the sounds in this library (using the percussive sounds with some of the atmospheres and transitions). Apart of being a great library for Horror or Sci-fi games and movies, this collection works well for trailers too.

    Sounds included:
    Atmospheric elements: Abnormal melodies, feedback and tense drones • Cinematic basses • Saturated bass drops • Distorted cinematic impacts • Sci-fi impacts • Sound movements • Noises • Percussive elements: Drum kicks, distorted drum kicks, hi-hats, snare drums, movements and sequences • Transitions

    68 %
    OFF
    Ends 1519862400
  • Fire Designed Fire Play Track 255 sounds included, 34 mins total $42.35 $30.25 incl. vat

    Designed Fire explores the further sonic realms of a powerful element. It features special effects, ambiences, swooshes, whooshes, fly-bys, drones, textures, glitches and more, that range from heavily designed to basic elements.

    Created in collaboration with Bend Audio Design.

    Designed Fire highlights:

    255 24bit / 96kHz WAV files / Meta-tagged (Soundminer)
    89 Swooshes / Wooshes
    75 Short elements
    29 Impacts
    28 Winds
    34 Textures

    29 %
    OFF
    Ends 1519862400
  • Drones & Moods Aberrant Drones Play Track 79 sounds included, 409 mins total
    5 out of 5
    $60.49 incl. vat

    Discover nearly 7 hours of individually flavoured, hauntingly musical and ever-evolving drones, packaged neatly into 79 of the kind of audio files that you can never have enough of.

    In many ways, Aberrant Drones is the sister library of the Augmentation Elements library, as it provides you with the creative tools required to augment your sound design.

    Whilst the Augmentation Elements library arms you with tools such as whooshes, risers and stings, allowing you to sweeten specific elements of your sound design, the Aberrant Drones library allows you to augment the mood of an entire scene. Each audio file is populated with richly descriptive SoundMiner metadata, making it easier than ever before to sift through hours of abstract drones.

    This is why the metadata is usually described in terms of feeling and mood, like so:

    Drone,Sci-Fi,Abstract,Cacophonous,Glowing,Impending Doom
    Drone,Sci-Fi,Abstract,Textured,Vibrating,Oscillating,Anxiety

    Augment your creative tool-box with Aberrant Drones, and transform your sound design work!


Latest releases:  
  • COUNTRYSIDE WINTER – Ambisonics brings you 15 ambisonic atmospheres recorded during different times of a day in winter: early morning, morning, afternoon, evening and night.

    All these recordings have been done in Sierra Morena, Andalusia (Spain). Being winter, birds tend to be much quiet and insects are almost gone, so most of the ambiences are pretty silent. You will hear mainly birds such as sparrows, doves and woodpeckers. At night dogs and roosters make an appearance.

    Recordings have been done with a Sennheiser Ambeo + Zoom F8. All files are in 96 kHz/24 bit and meta-tagged.

    About this library:

    This Ambisonics sound effects library contains the original recordings in A-Format (no processing whatsoever) and in B-format (AmbiX). By using Harpex, 5.1 (LRCLfeLsRs) and stereo (ORTF) mixdowns have been also added. Note that the Lfe channel is mute. It hasn't been removed to follow the SMPTE  path order.

    The Ambisonics version contains all the formats. Stereo version only includes the stereo files.

    Soundcloud demo plays the stereo version.

    20 %
    OFF
    Ends 1520553600
  • Fire Designed Fire Play Track 255 sounds included, 34 mins total

    Designed Fire explores the further sonic realms of a powerful element. It features special effects, ambiences, swooshes, whooshes, fly-bys, drones, textures, glitches and more, that range from heavily designed to basic elements.

    Created in collaboration with Bend Audio Design.

    Designed Fire highlights:

    255 24bit / 96kHz WAV files / Meta-tagged (Soundminer)
    89 Swooshes / Wooshes
    75 Short elements
    29 Impacts
    28 Winds
    34 Textures

    29 %
    OFF
    Ends 1519862400
  • Ever wonder how game developers get those awesome signature sounds for their games? How do sound designers and audio engineers come up with fantastic user interface and game play sound effects? Well… this is it: Puzzle Game Sound Effects Library. 550+ industry defining game ready sound effects. Full of spunk, sparkly tones and bubbly frequencies, Puzzle Game gives you access to the game sound assets your users are expecting.

    The Puzzle Game SFX library features sounds such as:

    • Game Accents and Achievements
    • Bonus Sounds, Breaks and Collects
    • Connects, Drops, Items
    • Magic, Menu, and UI
    • Merge, Tile Movement
    • Gear and Lever Switches and Turns
    • Notifications and Alert Sounds
    • Upgrades, Poofs, and Disappears
    • Organic UI, Static Hits
    + many more
  • This library is a collection of various curtains and blind sounds. For both curtains and blinds, we recorded various materials mounted in a number of ways for a diverse, well rounded selection of unique sounding effects for an often generalized household sound.Each track includes multiple movements. The recordings are mono and unprocessed, so they sound natural and can be easily placed in your stereo field.

    All tracks recorded on the Zoom H5 using the Neumann KMR81

    20 %
    OFF
  • STUTTER MATTER delivers a fresh sound set of syncopated hi-tech stutters and ultra-triggered sound effects. Whether you work as movie trailer or game sound designer, editor, soundtrack composer or experimental musician STUTTER MATTER will add a touch of sound dementia to create a more diverse listening experience.

    If you are looking for brutal syncopated bursts, head spinning dynamics shifts and industrial dyslexia this sound pack deserves a listen for anyone seeking out interesting sound design.

    Categories:

    • Stutter Chopper
    • Stutter Filter
    • Stutter Reverse
    • Stutter Woosh
    • Stutter Sequences and Glitches
    • Unprocessed/Raw Stutter elements
    + Bonus folder – 40 Hi-Tech Hits


Need specific sound effects? Try a search below:
 

If sound designers want to get started doing sound for VR, how do they go about it?

It is tough to design for VR without a device — so getting access to one would be a good place to start :) The audio tools exist and are constantly being improved upon. I think it would be great for sound designers to also jump into the video game development process, with game engines like Unreal Engine 4 that have a low price barrier — picking up those skills can help when communicating with the rest of the game development team too.

We definitely need more sound designers breaking new ground, failing, iterating and pushing the medium forward.

Another approach would be to work with the many VR enthusiasts around the world. There are lots of great indie projects around the web and I’m sure they would all value the contribution of a sound designer.

We’ve found that a number of the VR developers around the world care a lot about audio and they are willing to experiment with new technology and techniques.

We definitely need more sound designers breaking new ground, failing, iterating and pushing the medium forward.

 

Please share this:


 

About Varun Nair
Varun has worked on over 400 projects in films, games and advertising and music. He’s the co-founder of Two Big Ears, a company developing advanced audio solutions for games and VR. They offer 3Dception, a unique binaural audio and room modeling solution for game engines and middleware.
 
 
 
THE WORLD’S EASIEST WAY TO GET INDEPENDENT SOUND EFFECTS:
 
A Sound Effect gives you easy access to an absolutely huge sound effects catalog from a myriad of independent sound creators, all covered by one license agreement - a few highlights:
 
  • Horror Distorted Perception Play Track 146 sounds included, 44 mins total

    Need to create a tense atmosphere to induce madness, craziness and fear in your Horror or Sci-fi project? DISTORTED PERCEPTION is the library for you.

    Abnormal melodies, strange sound movements, transitions and distorted cinematic impacts are some of the things you will find in this collection; sounds with a distorted, saturated and tense feeling to grab people’s attention and provoke fear, tension and suspense in them.

    Yu can also use the sounds to enhance musical productions or create some nasty and crazy musical moments, thanks to a nice selection of percussive elements. The library contains two musical compositions using only the sounds in this library (using the percussive sounds with some of the atmospheres and transitions). Apart of being a great library for Horror or Sci-fi games and movies, this collection works well for trailers too.

    Sounds included:
    Atmospheric elements: Abnormal melodies, feedback and tense drones • Cinematic basses • Saturated bass drops • Distorted cinematic impacts • Sci-fi impacts • Sound movements • Noises • Percussive elements: Drum kicks, distorted drum kicks, hi-hats, snare drums, movements and sequences • Transitions

    68 %
    OFF
    Ends 1519862400
  • Fire Designed Fire Play Track 255 sounds included, 34 mins total

    Designed Fire explores the further sonic realms of a powerful element. It features special effects, ambiences, swooshes, whooshes, fly-bys, drones, textures, glitches and more, that range from heavily designed to basic elements.

    Created in collaboration with Bend Audio Design.

    Designed Fire highlights:

    255 24bit / 96kHz WAV files / Meta-tagged (Soundminer)
    89 Swooshes / Wooshes
    75 Short elements
    29 Impacts
    28 Winds
    34 Textures

    29 %
    OFF
    Ends 1519862400
  • Knights, the Cavalry, the Three Musketeers, Cowboys, Indians, Arnold Schwarzenegger in TRUE LIES: The cool guys sit on horses.

    Bring these animals to vibrant life with our extensive HORSES SFX library.

    You get 9 GB of high quality and royalty free SFX of countless horse voices, a mass of Foley sounds, pass-bys and steps in various gaits and different surfaces.

    The Horses library just received a 4.8/5 rating at The Audio Spotlight!

    Horses SFX library highlight:

    • Library ships in 96/192 kHz, 24-bit High Definition Audio
    • 9 GB of MOVEMENTS, HORSE VOICES & FOLEY
    • Playable KONTAKT 5 instrument for horse steps
    • Effective workflow: well-grounded Soundminer Metadata

Explore the full, unique collection here
 
 
FOLLOW OR SUBSCRIBE FOR THE LATEST IN FANTASTIC SOUND:
 
                              
 
GET THE MUCH-LOVED A SOUND EFFECT NEWSLETTER + 40 FREE SFX:
Just enter your details below to get the newsletter and free sound effects (soundlist):

2 thoughts on “Exploring New Sonic Worlds: Sound for Virtual Reality

  1. Speaking of mapping sensor data to sound, you might be interested in the work being done on haptics and haptic-tactile broadcasting, esp. for live virtual reality events where the “feeling” of the remote event can be captured, added to the audio and video VR stream and then decoded and used by the remote VR participant (using the appropriate haptic-tactile hardware such as a platform, vest, gloves, etc.) adding significant presence to the event.

Leave a Reply

Your email address will not be published. Required fields are marked *

HTML tags are not allowed.