Some of you have probably already have heard of Sound Particles, a strange 3D audio software that is being used a lot on epic Hollywood movies (full list here). But what is Sound Particles?
Wouldn’t it be nice if we could use particle systems with sound? Creating thousands of small sounds that together create the illusion of something great
12 years ago, I realized that the most interesting visual effects I was seeing in movies were using particle systems – a well-known CGI technique where thousands (or even millions) of small points create the illusion of fire, dust, smoke, fairy dust, desert storms, etc. And I thought “wouldn’t it be nice if we could do the same thing with sound? Creating thousands of small sounds that together create the illusion of something great”. But that was just an idea, like so many other we have in our lives, and time went by. In 2012, I had finished my PhD and since no one was using particle systems for audio, as a computer nerd that I am (this face doesn’t fool anyone!), I decided to create my own particle system software for audio, and that was the beginning of Sound Particles…
To those who don’t want to read the entire article to know what Sound Particles is able to do, let us see a small example – Imagine that you want to create the sound of a battlefield: You open your DAW, and then you start adding sounds – an explosion here, another explosion over there, some impacts, some screams, guns, etc. Probably, after several hours you may end-up with 50 sounds playing at the same time.
[tweet_box]Sound Particles: Imagine – and create – thousands of sounds around you:[/tweet_box]
With Sound Particles you can say: give me 10 000 particles (sound sources) spread over 1 square mile, go to your sound library and pick up 200 war-related sounds (each particle will randomly reproduce one of those files), add some random movements and random audio effects, and render the result in 7.1 – everything in a few minutes.
Particle Systems
Particle Systems is a technique widely used in Visual Effects (VFX) and Computer Graphics (CGI) that was originally created by William Reeves, at Lucasfilm, for “Star Trek II: The Wrath of Khan” (yes! Lucasfilm also works for other movies besides Star Wars). And since then, this technique is used every time you see CGI fire, rain, smoke, dust.
A particle system will create and manage every single particle, having only a few parameters for the user to control the desired effect
Instead of having an animator to animate each rain drop or each grain of smoke or dust, a software component (particle system) will create and manage every single particle, having only a few parameters for the user to control the desired effect.
This approach has 3 big advantages:
Productivity – the animator doesn’t need to worry about every single element. By controlling the parameters of the system, you can control everything, independently of having 100 particles or 100 000 particles.
Scale – Even if you had a big animation team, you were still limited to the amount of things that you can animate manually (for instance, there is a scene on the last “Hunger Games” movie with 250 million particles).
Organic results – even if you have the time to animate all elements of the scene (e.g. rain), you will probably not end-up with a good final result – the animator will animate a few rain drops, and then duplicate or copy/paste them, and the final result will lack something… some true randomness.
Sound Particles
Sound Particles is a 3D CGI-like standalone software for audio post production – imagine Maya or Blender, but for sound – you start with an empty space; instead of 3D models, you position sounds on the 3D space; and instead of virtual cameras, you use virtual microphones that capture the overall sound of things. For instance, in the following image, each point represents a particle (a sound source) and the arrows represent a 5.1 microphone (which can also be animated, moving thru the scene).
The virtual microphone concept also makes the system agnostic in terms of output
You will always need at least one sound source (to have some sound) and at least one virtual microphone (to be able to capture the sound of the scene). Actually, the concept of the virtual microphone is quite interesting – not only you are able to get the distance attenuation, propagation delays, and Doppler, but this concept also makes the system agnostic in terms of output. You can simply change the mic format, and everything is re-render for the new format.
Particles Group
When you use a particle group, there are several things that you can do…
• Number of particles – how many sound sources do you want to use? 20? 10000?
• Original Sound Files – which audio files do you want to feed the particles? If you import a single audio file, all thhe particles will play exactly the same. If you import several files (10, 500, …), each particle will randomly select one of the available audio files, and reproduce the file from start to finish.
• Position – Where do you want to create the particles? Inside a box, on a circle line, on the surface of a sphere? For instance, if you want to recreate a stadium, probably the shape of a stadium could be a good starting point.
• Movement – Sometimes you want particles to stay still, but sometimes you want particles (sounds) to move around (rotating around, moving left-to-right, random directions, random velocities, etc.).
• Audio Modifiers – If you have 100 particles and you import only 5 files, then each audio file will be reproduced by more than one particle, which is not a bad thing, but it could be even better if we could create variations of the original files, to get diversity. That is the goal of audio modifiers, random audio effects that slightly change the sound of each particle. This could be simple things like random gains (some particles play louder, others softer), random delays (some particles start playing later on), random filters (random EQ in each particle), random pitch-shifter, or even granular synthesis (instead of reproducing the entire audio file, each particle only reproduces a small random time slot of the audio).
Besides particle groups, you can also use a particle emitter, an object that creates particles at a constant rate (e.g. creating 20 particles per second, during 30 seconds).
Microphones
Regarding virtual microphones, you can select the traditional mono (omni, cardioid, fig-of-eight), stereo pairs (XY, MS, ORTF, Decca Tree), Ambisonics (up to 6th order), multichannel (5.1, 7.1, Dolby Atmos 9.1 bed, 11.1/13.1, 12.0), or even create your own custom configuration. For instance, last year, there was a play in NYC, with actor Ed Harris, where rain had a significant role, and they used Sound Particles with a custom 20 channel virtual microphone, where each channel corresponded to the exact position of the 20 speakers of the theatre, allowing a perfect 3D reproduction of the virtual scene. And even if they wanted to replicate that in a different room, with a completely different speaker layout, they would only need to change the mic settings and re-render everything again.
Other applications
Although most people use Sound Particles with its particle systems features, you can actually use it with single individual sounds. For instance, many people love its Doppler.
Also, a lot of people are using it for VR, due to the ability to import videos (including 360º videos) and actually position and animate sounds on top of the image, by simply dragging the position of the sound with the sound (Sound Particles was probably the first software to allow this). And, you can even use key-frame animation (automation) to animate everything.
Epic, but detailed
When you think on the concept of using thousands of sounds, most of you would automatically think “that will be a big noise-like mess, right?”. Well, it depends… Most of the time, you want both that sense of many things happening, but you also want sound detail. Imagine that battlefield scene…
Most of the time, you want both that sense of many things happening, but you also want sound detail
On a true battlefield, you will have thousands of sounds (1000 soldiers, each one doing several sounds with their weapons, boots, talking, impacts, etc.). But if you play 200 war-related sounds on your DAW, you will have a very bad experience – a noise-like experience. So, what is the difference? Well, on the battlefield, 99% of those thousands sounds will be highly attenuated, since they are happening at a distance. And then you have only some few sounds actually happening close to you – thousands of attenuated sounds to give the sense of epic scale, and a few sounds close to you to give you detail.
The same happens with Sound Particles. If you create 100 particles positioned on a circle around you, by default, all sounds will have the same amplitude, and probably will end up with a big noise-like sound (sometimes, you want noise-like sounds). But if you use a rectangle (spreading sounds over a rectangle with 1-mile length), you get the same natural behaviour that you get from the actual field. Even with 10 000 particles there, doesn’t matter where you place the microphone, you will always have only a few particles near the microphone.
In conclusion
The most interesting thing about creating Sound Particles is that when people start to understand its concept, their imagination starts to sparkle with ideas (“what would be the sound if I …”).
For me, the current version is only the starting point (I confess I have a list of 200 additional features I want to add in the future, but days only have 24 hours). And if you ever need anything, don’t hesitate to email us… really!
PS – The software is free for schools, teachers and students