Pro Tools for Virtual Reality Asbjoern Andersen


The emergence of virtual reality (VR) means a lot of changes for the way content is created and used. And in this special interview, AVID’s Tony Cariddi shares his insights on what the future holds for creatives using Pro Tools for VR content creation:


Written by Jennifer Walden



New features come with every Pro Tools upgrade, but you always get that one feature that makes you say, “How did I work without this?!” Take Offline Bounce for example. Every Pro Tools user was thanking the audio gurus at AVID for bringing that timesaver into their workflow. I mean really, bouncing stems in real time? Insanity! Now, on the cusp of VR virtually becoming a daily reality, content creators are going to want to work with their favorite audio artists instead of having to choose from the small pool of talent wielding the prototype tools for VR production and post production.

Content creators are going to want to work with their favorite audio artists instead of having to choose from the small pool of talent wielding the prototype tools for VR production and post production.

While many third-party companies have been perfecting their VR workflow solutions, AVID has been busily preparing their Pro Tools platform to be the place where these third-party tools can work together, in an environment that’s comfortable for most audio post pros. We’re all waiting for that Pro Tools upgrade that will make us say, “How did I ever work with VR content without this?!” Tony Cariddi, AVID‘s ‎Director of Product and Solutions Marketing, generously shares some insight into Pro Tools upcoming VR workflow solutions.
 

Audio post pros who are familiar with linear projects, like films or series, may not know what they’re up against in terms of handling post sound on virtual reality projects. Can you explain some of the differences in the Pro Tools workflow for a linear project vs. a VR project?

In the simplest, most fundamental way, in virtual reality you have the viewer interacting with the content rather than just passively watching it or listening to it. Tony Cariddi, AVID‘s ‎Director of Product and Solutions Marketing So that sets up the expectation that the content is going to respond to the interaction in a natural way. For example, when the viewer turns his/her head the expectation is that not only does the picture follow, but the sound is going to follow too. That creates a lot of new challenges.
You are dealing with a lot of new variables. Obviously with the surround and immersive mixes that our customers do today using Dolby Atmos or other multichannel surround formats, they have sounds respond to camera angles and things like that, but in virtual reality it has to be much more fluid. You also have new challenges when dealing with stereo field information, which doesn’t really respond so well with traditional panning in virtual reality. When someone turns his/her head, what do you do with a stereo source? So virtual reality opens up a lot of opportunity for new technology to create natural sounding results, and it also sets up a lot of questions aesthetically, just like surround did.
 

What’s been the biggest challenge so far in designing tools that specifically address VR workflows?

We are looking into support for Ambisonics. We have really strong tools for surround and automation, really advanced tools that give mixers a lot of control over that. Outside of that, as it pertains specifically to virtual reality, most of our efforts at this point are to make sure that our SDKs — our Alliance Partner tools, are strong and exactly what the development community needs in order to integrate their tools into Pro Tools. For almost 30 years we’ve had a very vibrant developer community that has made plug-ins for Pro Tools, and plug-ins for Media Composer.

One of the huge challenges in the industry is that there are so many different tools that it becomes very difficult for the users, whether it’s the facility manager or the actual engineer, to integrate all of these things together

And now more than ever, one of the huge challenges in the industry is that there are so many different tools that it becomes very difficult for the users, whether it’s the facility manager or the actual engineer, to integrate all of these things together. There’s no shortage of companies or individuals that are creating amazing tools to solve these problems. But what they don’t do is make it easy for the user to integrate all of these tools together and make them work together. So a lot of our focus over the past few years has been to develop a really strong platform upon which they can easily integrate these tools. So when a user needs to work on a virtual reality project, we make sure it is super easy to get the plug-in, or get the add-on, and simply start using the professional tools that they’re familiar with, and that it gives them compatibility with the rest of the industry.
 

Are there any specific solutions/tools that AVID is developing for Pro Tools to help audio post pros design and mix for VR?

We’ve been working on Dolby Atmos integration and that’s going to be a big boon for anyone working in virtual reality because you can have control over specific objects across a 64-channel speaker matrix. That gives you a lot of flexibility.

We’ve been working on Dolby Atmos integration and that’s going to be a big boon for anyone working in virtual reality

In addition, we’re looking at supporting Ambisonics natively in Pro Tools. We’re also working closely with gaming developers to ensure better workflow across Pro Tools, middleware, and gaming platforms. Otherwise, we are working to make sure that we have a solid platform to support the third-party developer tools, and making sure those can be integrated into Pro Tools.
 

In the future does AVID see themselves developing reverbs or compressors or EQs that are designed specifically for the virtual reality environment?

I think that’s possible but I can’t speculate on what will happen. But, historically we have developed a core set of tools for new formats as they have evolved and became more prevalent. At the same time, we’ve always ensured strong support for the third-party ecosystem, which is usually more agile to produce these tools because they are smaller companies and they can really focus-in on one single thing. So I can’t give a definitive statement about our future development but I think that it is possible that we may develop something like that in the future.

With Pro Tools, we have an extremely mature platform and strong development tools for third-parties. As the challenges of VR workflows become clearer, to us and to the market, we will need to keep our SDK and Alliance Partner tools updated to make sure that they can hook-in in the right way. So we are continuing to evolve that platform to make sure that the development community has everything that they need to create and integrate the best tools possible.
 


Popular on A Sound Effect right now - article continues below:

 

Latest releases:  
  • Mechanical Gearbox Play Track 3551 sounds included, 279 mins total $149.99

    We've ventured to obscure boutiques, prop houses and vintage shops to capture mechanical contraptions from around the world. Ranging from bizarre creations, to steampunk gadgetry, gizmos and machines, GEARBOX clocks in at over 10 GB of high definition, precision mastered sounds spanning across 2987 construction kit sounds and 584 designed sounds.

    GEARBOX equips Sound Designers with a literal toolbox of mechanical gadgetry. Ranging from tiny to huge, GEARBOX's machines and gizmos provide coverage for interacts, mechanism, machine or device in your scene or game.

    INTRODUCING BUILDING BLOCKS

    In addition to CONSTRUCTION KIT and DESIGNED SOUND content, GEARBOX features BUILDING BLOCKS. This category of sound consists of designed phrases and oneshots utilized for our designed machinery, empowering Sound Designers with maximum flexibility when trying to get that particular phrase from an existing DESIGNED SOUND. GEARBOX features over 468 BUILDING BLOCKS ranging from levers, hits, grinds, snaps, and more.

    Video Thumbnail
    Add to cart
  • This is a unique bicycle library that captures this characteristic bike in clean, quiet, nicely performed true exterior rides. Including multiple perspectives, speeds and actions. From fast passbys on asphalt to slow onboard recordings and smooth stops.

    The UglyBike is a typical old bicycle that’s working fine, but needs some TLC. It is a bicycle that’s just average, a little rattle a gentle scrape, a bike that everyone has had but got traded in for a newer one. A story of unrequited love.. :)

    Speeds and actions:
    Three speeds. Departures from slow, medium to fast getaways. Arrivals from slow stops with gently squeaking handbrakes to heavy stuttering skids.

    Five perspectives:
    1. Onboard Front: captures the whirring tire and surface sound.
    2. Onboard Pedal: nice overall combination of pedaling, crank creaks, chain rattle, tire and surface sounds.
    3. Onboard Rear: close up sound of the rear axle, with chain, sprocket and switching of gear.
    4. Tracking shot: mono recording of the passby, keeping the bike in focus while passing by.
    5. Static XY shot: stereo recording of the passby that emphasizes speed.

    Overview of perspectives and mic placement:

    Onboard recordings are 2-3 minutes long depending on speed. Higher speeds > shorter duration.
    All 3 onboard mics are edited in sync with one another to make layering easy.
    All Passbys, Arrivals and Departures move from Left to Right.

    Metadata & Markers:
    Because we know how important metadata is for your sound libraries we have created a consistent and intuitive description method. This allows you to find the sound you need easily, whether you work in a database like Soundminer/Basehead/PT Workspace work, or a Exporer/Finder window.

    However, we are aware that some people have different needs for different purposes, so we’ve created a Metadata Reference Guide that explains the structure. And because we’ve automated the metadata proces, you can be confident that a ‘find & replace’ command will always replace all instances.

    Download our Metadata Reference Guide

    Download complete metadata PDF

    If you have any questions about this, contact us!

    Additionally, we added Markers to some wave files, so specific sound events are easy to spot in Soundminer or other database apps.

    Need more?
    The UglyBike library is part of the complete ‘City Bicycles’ library package available at www.frickandtraa.com. It consists of all 4 bicycles and includes additional surfaces and extras ranging from one-off  bicycle passes captured in the city and bounces and rattles. The extra bicycles surfaces and additional effects are also available seperately here on ‘a Sound Effect’. If you’ve bought a single library and want to upgrade to the full package, contact us for a reduced price on the complete City Bicycles library. Every part of City Bicycles that you paid for will get you an extra reduction on the full package.

    Video Thumbnail
    Responses:

    344 AUDIO:City Bicycles has a plethora of content, for a great price. The perfect balance between a great concept, great presentation and outstanding execution, lands them an almost perfect score of 4.9..

    The Audio Spotlight: City Bicycles is worth getting if you are in need of great sounding and well edited bicycle sounds.

    Watch a video created by Zdravko Djordjevic.

    Video Thumbnail

     

    Add to cart
  • Environments Museums & Galleries Play Track 272 sounds included, 800 mins total $100 $80

    This library features a wide range of recordings from various museums and galleries, each differentiated by the nuances of their size and space. All recordings feature pristine echos, walla and movement. The library includes stereo & 5.0 recordings from:

    • War Museums
    • History Museums
    • State Museums
    • Science Museums
    • Art Galleries
    • Photography Galleries
    • State Galleries

    All sounds were recorded using a stereo pair of DPA 4060s, DPA 5100, Sound Devices Mix-Pre 6 and Sound Devices 788T.

    20 %
    OFF
    Ends 1556056800
    Add to cart
  • The American M5 High Speed Tractor includes over 20 gigabytes of recordings of a WWII US military vehicle with a Continental 6572 six-cylinder petrol engine with 207 horsepower. 188 sound fx document a full suite of performances from M5, also known as versions M5A1, M5A2, M5A3 and M5A4.

    The performances include starting, idling, departing, arriving, and passing by from 6 exterior perspectives at slow, medium, and fast speeds. 10 additional perspectives feature motor, interior, exhaust, tracks, and other locations that capture idles, driving, and steady RPMs from onboard the tractor.

    Includes extensive Soundminer metadata.

    Add to cart
  • Cars Volvo 242 DL 1975 Play Track 364 sounds included $249

    The Volvo 242 sound fx collection includes 271 sounds in 13.51 gigabytes of audio. The 242 is a DL 1975 version of the car, also known as models 240, 244, and 245. It features 25 takes of recordings from the Swedish vehicle and its 4-cylinder B20 A, 82 horsepower engine.

    16 synchronized perspectives capture both onboard and exterior performances. Eight onboard perspectives (12 channels, including 4 in AMBEO) recorded driving at steady RPMs, with gearshifts, and ramps using microphones mounted in the engine, interior, and exhaust. Eight other exterior perspectives (18 channels) showcase driving at fast, medium, and slow speeds approaching, departing, and passing by. There are also steadies in neutral, blips, and performed effects, as well as an Altiverb impulse response.

    All clips have 18 fields of Soundminer, BWAV, and MacOS Finder metadata.

    Add to cart

Need specific sound effects? Try a search below:
 

Are there different workflow requirements for virtual reality projects vs. augmented reality projects?

With augmented reality, you typically think of imagery and graphics overlaying reality. With virtual reality, the entire environment is fabricated. With augmented reality, you have an overlay of interactive production material on top of the real world. It’s usually more of a graphic-based medium compared to virtual reality, which require more sound information. So we have augmented reality tools, such as Orad’s RealSet, which are used in the sports production world — producers can superimpose information on the screen, follow players, put three-dimensional objects in a three-dimensional space on existing footage that you can interact with. So we have a very strong set of tools on that side with RealSet and a handful of other products in that category.
 

When it comes designing, mixing, and print mastering for VR, what solutions will AVID/Pro Tools offer audio post pros that will allow them to make a final mix and then get that back into the VR project?

You can do that today. There was a company called Two Big Ears that was acquired by Facebook in the spring. Their product is called Facebook 360 Spatial Workstation, and it enables users to wear a headset and do the mix within Pro Tools.

It enables users to wear a headset and do the mix within Pro Tools

It handles all the final encoding for you and syncs it back to the final video for you when you are done. So you can do the mix while you’re in that immersive format without having to go back and forth. So that technology is here today and it is only going to get better and better. (For more info, you can check out Facebook 360 Spatial Workstation)
 


Travel to Yosemite National Park in VR, thanks to this National Geograhic presentation, created using the Facebook 360 Spatial Workstation suite

 

For many audio post pros, Pro Tools is their defacto DAW. They’re comfortable designing and mixing on that platform…

People doing this work are typically seasoned professionals and they’re going to want to work in a familiar environment they are comfortable with, so that’s what is great about Pro Tools. You have this immersive VR format and emerging technology, and instead of having to utilize completely new tools in a new workstation you aren’t familiar with and don’t have years of muscle memory to take advantage of, you can work in a way that’s familiar and with the same tools.
 

How does Dolby Atmos for VR differ from the Dolby Atmos tools for linear projects?

The tools they are going to manifest in Pro Tools will be pretty similar. Atmos gives you a lot of flexibility to place objects in very distinct places across a 64-speaker matrix.

If you’re in a small room, or living room, or a giant Dolby-spec theater, those mixes will translate better than they ever have before

That scales really nicely; if you’re in a small room, or living room, or a giant Dolby-spec theater, those mixes will translate better than they ever have before. The basis for this technology will impact VR production just the same.You have a lot of objects to work with and move, so the Dolby tools will translate to that format very nicely.
 

Where would AVID like to be with their sound for VR tools 5 years from now?

We try to stay very close to what our customers are up against, to their challenges as they evolve. That evolution is driven by economic factors, and the health of the industry. Sometimes there are technological breakthroughs that help to advance that. But what we want to do is make sure that we continue to stay close to the customers. We want to make sure that as these relatively new workflows are evolving, that we are seeing where the hang-ups are, where the pitfalls are, and find out where we can increase efficiency and enable creativity.

How can we deliver tools that make their jobs faster, especially for tasks that are redundant and menial? How can we speed that up so that our customers can focus on getting a great sounding mix, and getting very creative results? Beyond that, we want to make sure that we are able to support our really strong third-party Alliance Partners who are usually on the forefront of solving a lot of our customers’ challenges too. We want to stay close and partner with our customers, and also with the developers, so that we can create the best workflow experience possible.

A big thanks to Tony Cariddi for this look to the future of VR sound in Pro Tools – and to Jennifer Walden for doing the interview! Learn much more about Pro Tools here

 

Please share this:


 


 
 
THE WORLD’S EASIEST WAY TO GET INDEPENDENT SOUND EFFECTS:
 
A Sound Effect gives you easy access to an absolutely huge sound effects catalog from a myriad of independent sound creators, all covered by one license agreement - a few highlights:
 
 
  • Destruction & Impact Bodyfall vol 1-4 Play Track 929 - 3804+ sounds included From: $35

    Bodyfall is an exhaustive multi-volume sound library, designed in collaboration with prize-winning French Foley artist Florian Fabre, and recorded in the famous Hiventy foley studio.

    Each volume features 2 different falling surfaces. On each of them, sounds of different parts of the human body:

    Chest (simulated with 4 textures of distinct densities, labeled M1,M2,M3 and M4), feet, knees, and hands, have been separated.

    All recordings were made from 3 distances (close, mid, distant) with 3 strength levels (hard, medium, soft). Finally, this huge toolbox provides you with infinite combinations to make your own and unique bodyfalls.

    Bodyfall vol 1: Generic / concrete & Hard Metal:

    Files included: 972 .WAV, stereo & mono files , 24 bit / 96 kHz (416 MB)
    Bodyfall vol 2: Wood / Rustic & Metal Grid:

    Files included: 974 .WAV, stereo & mono files, 24 bit / 96 kHz (438 MB)
    Bodyfall vol 3: Dirt & Wood / Hardfloor:

    Files included: 929 .WAV, stereo & mono files, 24 bit / 96 kHz (433 MB). No distant files for Dirt soft impacts.
    Bodyfall vol 4: Metal / Composite & Wood / Parquet:

    Files included: 929 .WAV, stereo & mono files, 24 bit / 96 kHz (433 MB)
    Bodyfall Bundle:

    All files & surface variations from vol 1-4 included
  • Destruction & Impact Bullet Impacts Play Track 320 sounds included
    Rated 4.00 out of 5
    $35

    Prepare for impact! This EFX Bullet Impact collection features a huge number of impacts into cars, metal, walls, water, body impacts, as well as passbys, ricochets and underwater passbys.

    A must-have for for actual bullet and combat sounds – and for adding oomph to many other types of impact sounds too!

    Add to cart
  • Introducing Artillery, a new powerful sound library covering a wide range of elements including cannon shots, electric systems, mechanical parts, distant artillery barrage, impacts, whooshes, grenade launchers and more.

    The sounds are organized in the following categories:

    • Artillery: Falling rubble, explosion impacts
    • Beeps
    • Grenade Launchers
    • Howitzer: Electric System Background
    • Howitzer: Falling rubble, explosion impacts
    • Howitzer: Mechanical Parts Handling
    • Howitzer: Shot Metallic Parts
    • Howitzer: Shot Distant Explosions
    • Shell Trajectories

    Add to cart
 
Explore the full, unique collection here

Latest sound effects libraries:
 
  • Mechanical Gearbox Play Track 3551 sounds included, 279 mins total $149.99

    We've ventured to obscure boutiques, prop houses and vintage shops to capture mechanical contraptions from around the world. Ranging from bizarre creations, to steampunk gadgetry, gizmos and machines, GEARBOX clocks in at over 10 GB of high definition, precision mastered sounds spanning across 2987 construction kit sounds and 584 designed sounds.

    GEARBOX equips Sound Designers with a literal toolbox of mechanical gadgetry. Ranging from tiny to huge, GEARBOX's machines and gizmos provide coverage for interacts, mechanism, machine or device in your scene or game.

    INTRODUCING BUILDING BLOCKS

    In addition to CONSTRUCTION KIT and DESIGNED SOUND content, GEARBOX features BUILDING BLOCKS. This category of sound consists of designed phrases and oneshots utilized for our designed machinery, empowering Sound Designers with maximum flexibility when trying to get that particular phrase from an existing DESIGNED SOUND. GEARBOX features over 468 BUILDING BLOCKS ranging from levers, hits, grinds, snaps, and more.

    Video Thumbnail
  • This is a unique bicycle library that captures this characteristic bike in clean, quiet, nicely performed true exterior rides. Including multiple perspectives, speeds and actions. From fast passbys on asphalt to slow onboard recordings and smooth stops.

    The UglyBike is a typical old bicycle that’s working fine, but needs some TLC. It is a bicycle that’s just average, a little rattle a gentle scrape, a bike that everyone has had but got traded in for a newer one. A story of unrequited love.. :)

    Speeds and actions:
    Three speeds. Departures from slow, medium to fast getaways. Arrivals from slow stops with gently squeaking handbrakes to heavy stuttering skids.

    Five perspectives:
    1. Onboard Front: captures the whirring tire and surface sound.
    2. Onboard Pedal: nice overall combination of pedaling, crank creaks, chain rattle, tire and surface sounds.
    3. Onboard Rear: close up sound of the rear axle, with chain, sprocket and switching of gear.
    4. Tracking shot: mono recording of the passby, keeping the bike in focus while passing by.
    5. Static XY shot: stereo recording of the passby that emphasizes speed.

    Overview of perspectives and mic placement:

    Onboard recordings are 2-3 minutes long depending on speed. Higher speeds > shorter duration.
    All 3 onboard mics are edited in sync with one another to make layering easy.
    All Passbys, Arrivals and Departures move from Left to Right.

    Metadata & Markers:
    Because we know how important metadata is for your sound libraries we have created a consistent and intuitive description method. This allows you to find the sound you need easily, whether you work in a database like Soundminer/Basehead/PT Workspace work, or a Exporer/Finder window.

    However, we are aware that some people have different needs for different purposes, so we’ve created a Metadata Reference Guide that explains the structure. And because we’ve automated the metadata proces, you can be confident that a ‘find & replace’ command will always replace all instances.

    Download our Metadata Reference Guide

    Download complete metadata PDF

    If you have any questions about this, contact us!

    Additionally, we added Markers to some wave files, so specific sound events are easy to spot in Soundminer or other database apps.

    Need more?
    The UglyBike library is part of the complete ‘City Bicycles’ library package available at www.frickandtraa.com. It consists of all 4 bicycles and includes additional surfaces and extras ranging from one-off  bicycle passes captured in the city and bounces and rattles. The extra bicycles surfaces and additional effects are also available seperately here on ‘a Sound Effect’. If you’ve bought a single library and want to upgrade to the full package, contact us for a reduced price on the complete City Bicycles library. Every part of City Bicycles that you paid for will get you an extra reduction on the full package.

    Video Thumbnail
    Responses:

    344 AUDIO:City Bicycles has a plethora of content, for a great price. The perfect balance between a great concept, great presentation and outstanding execution, lands them an almost perfect score of 4.9..

    The Audio Spotlight: City Bicycles is worth getting if you are in need of great sounding and well edited bicycle sounds.

    Watch a video created by Zdravko Djordjevic.

    Video Thumbnail

     

  • Environments Museums & Galleries Play Track 272 sounds included, 800 mins total $100 $80

    This library features a wide range of recordings from various museums and galleries, each differentiated by the nuances of their size and space. All recordings feature pristine echos, walla and movement. The library includes stereo & 5.0 recordings from:

    • War Museums
    • History Museums
    • State Museums
    • Science Museums
    • Art Galleries
    • Photography Galleries
    • State Galleries

    All sounds were recorded using a stereo pair of DPA 4060s, DPA 5100, Sound Devices Mix-Pre 6 and Sound Devices 788T.

    20 %
    OFF
    Ends 1556056800
  • The American M5 High Speed Tractor includes over 20 gigabytes of recordings of a WWII US military vehicle with a Continental 6572 six-cylinder petrol engine with 207 horsepower. 188 sound fx document a full suite of performances from M5, also known as versions M5A1, M5A2, M5A3 and M5A4.

    The performances include starting, idling, departing, arriving, and passing by from 6 exterior perspectives at slow, medium, and fast speeds. 10 additional perspectives feature motor, interior, exhaust, tracks, and other locations that capture idles, driving, and steady RPMs from onboard the tractor.

    Includes extensive Soundminer metadata.

  • Cars Volvo 242 DL 1975 Play Track 364 sounds included $249

    The Volvo 242 sound fx collection includes 271 sounds in 13.51 gigabytes of audio. The 242 is a DL 1975 version of the car, also known as models 240, 244, and 245. It features 25 takes of recordings from the Swedish vehicle and its 4-cylinder B20 A, 82 horsepower engine.

    16 synchronized perspectives capture both onboard and exterior performances. Eight onboard perspectives (12 channels, including 4 in AMBEO) recorded driving at steady RPMs, with gearshifts, and ramps using microphones mounted in the engine, interior, and exhaust. Eight other exterior perspectives (18 channels) showcase driving at fast, medium, and slow speeds approaching, departing, and passing by. There are also steadies in neutral, blips, and performed effects, as well as an Altiverb impulse response.

    All clips have 18 fields of Soundminer, BWAV, and MacOS Finder metadata.

 
FOLLOW OR SUBSCRIBE FOR THE LATEST IN FANTASTIC SOUND:
 
                              
 
GET THE MUCH-LOVED A SOUND EFFECT NEWSLETTER:
 
The A Sound Effect newsletter gets you a wealth of exclusive stories and insights
+ free sounds with every issue:
 
Subscribe here for free SFX with every issue

One thought on “What the future holds for Pro Tools and VR – a Q&A with AVID’s Tony Cariddi:

  1. Interesting read. Their product team taking Ambisonics seriously is a late yet great start. More tracks are already being recorded and mixed in Ambisonics format, especially in Higher Order Ambisonics (HOA), but can’t be easily loaded and processed at the moment as Pro Tools HD limits only 8 channels per track. I believe this has significantly hindered many sound engineers and plugin developers from utilizing Ambisonics. Looking forward to hearing updates on HOA.

    Food for thought. Will Pro Tools update/improve interpolation features for automation any sooner? Angle is one of the most critical parameters for interactive sound to fully implement head rotation and spatial positioning in VR, but it’s not properly addressed by the current version. i.e. if the object doesn’t fall under min and max range, its angle value gets distorted right when it passes 360 and 0 degree :(

Leave a Reply

Your email address will not be published. Required fields are marked *