How to make your games blind accessible with game audio – an essential guide by Máté Moldován

My name is Máté Moldován, sound designer, composer and audio implementer for video games. I wrote this paper to provide game audio professionals a document that summarises currently available resources on how to make games accessible for blind players with the use of dynamic audio.

As this topic comes up more and more often in game development articles, this post breaks down the individual components of audio in games that can serve accessibility functions. I wrote this as part of a university dissertation research project with the involvement of two blind game accessibility consultants, Brandon Cole and LaMoine Williams, and 26 survey respondents who helped me find out more about the playing habits of Visually Impaired (VI) and blind gamers. Furthermore, academic resources and personal gameplay experiences were used in the creation of this document.

By the end, you should have a deeper understanding of the type of sounds and audio behaviour systems necessary to be included in a game to improve the gaming experience of VI and blind players.

All information conveyed in this document is a summary of my University Honours Dissertation essay and my Psychology of Sound essay, that are available for reading in case someone wishes to dig deeper to have a broader understanding of this subject.

Additionally, if you wish to view this document in a more complete format, you can find the Notion version, which contains more insights and examples. Please feel free to share this document among game developers and anyone who’s interested, and let me know if you wish to contribute so this document can be as up to date and useful as it can possibly be!

Why should you care about blind accessibility?

Video games have always given people an escape from everyday struggles. Players are able to relax better, improve their cognitive skills and problem solving capabilities, and get inspired by, in fact, become motivated to do task after task when playing a game. Today’s game stores have all kinds of genres available to meet everyone’s needs: action, adventure, action-adventure, role playing (RPG), action RPG, simulation, strategy, puzzle, sports and the list goes on and on. It’s basically limitless, and everyone can find their favourite.

The visuals of these new worlds and characters in the latest games are astonishingly realistic. While awesome graphics can be achieved using available free game engines (e.g. Unreal 4 and 5, Unity, CryEngine), not all gamers can benefit from the realistic visuals.

A blind or visually-impaired person relies less on the visuals and more on the auditory and haptic feedback of their environment. Because games wish to place the player into their own virtual world, for a blind player to be part of it, games need to try different techniques and systems to make the behaviour of audio more realistic and helpful, acting as a navigational tool to improve the player’s immersive experience.

Let’s see how this has been done in recent games and how the enhanced audio behaviours improve the overall gameplay

Sound Categories

Before you create sound assets for the game you’re working on, you will likely have a spotting session (either alone or with the directors/lead developers) to get the vibe of the game. This is a perfect opportunity to specify the different categories of sounds the game needs and start thinking of how these categories can serve the gameplay experience of visually impaired players.

The most common categories are dialogue, sound effects, ambience and music:

Dialogues exists mostly to drive the narrative and enhance immersion in role playing. It is advised to design dialogues with a mindset that it won’t be annoying for neither sighted or blind players, thus “accessibility dialogues” can be included in the game by default. Additionally, further guiding dialogue lines can be implemented in the game with an on/off toggle in the game’s menu.

Sound Effects (and UI) give real-time auditory feedback of user actions, mostly to point out interactable game objects. These functions are suggested to be set with an on/off toggle in the game’s menu.

Ambience creates a bed for all other in-world sounds, to improve immersion with the game world. Well-designed, detailed and interactive ambiences are suggested to be included in the game by default.

Music supports the narrative and gives a dynamic feel to the game, improving gameflow and immersion. Well-designed, dramatic or playful but definitely interactive music is suggested to be included in the game by default.

These sound categories help us separate the individual tasks of making a game more accessible for blind players. To further break down these categories, you will need to define the game mechanics that will be used the most by the player.

Dialogues

Navigation/Traversal

Dialogues in games are generally not designed to be used as an accessibility tool, although in many cases character dialogues can increase the immersion and navigation of a VI player by explaining the situation the player is set within. NPCs, loudspeaker announcements, creatures (and many more) can suggest directions to the player and can be included in the game by default. Also, game states, such as fight/roaming, can be supported with pre- and post-fight dialogue cues (e.g. “Alright, let’s do this!” / “Pheww, they almost got me there!”). If the states change less often, they can be separated with short in-game cinematics (e.g. The Last of Us Part 2) that set the scene with dialogues.

Before the player can move around in the game world, they will enter it through the game’s menu, which means that a blind player will need information to navigate both the menu and the game world. Mortal Kombat 11 is a great example for accessible menu navigation, as the game has its own dedicated narrated menu, which makes navigation for a blind player a much simpler task than ever before.

Having a dedicated space for accessibility option in the game’s menu that is easily accessible (just like in FIFA21) is a huge help for VI players. Even better is when the game starts with the accessibility settings when it’s launched for the first time.

The Accessibility menu pops up when player launches It Takes Two for the first time

Puzzle/Objective Hints

If the game has puzzles or the player needs to complete objectives to finish the game, these tasks will need to exhibit smart hints to actually tell the player what needs to be done. Dialogues are a quick way to suggest hints in puzzle levels, as well as scenes that are not supported with other sounds, or in not-too-obvious situations. The dialogue can be the main character talking in their head, a narrator/storyteller (interactive audio description), a conversation with another in-game character/NPC (the Uncharted series and Red Dead Redemption 2 are both great examples) or a text-to-speech function.

Combat/Quick Reflex Hints

If the core game mechanic is based on shooting and fighting, then additional audible combat/quick reflex hints are needed. Well-timed warning shouts can also be useful in games that have multiple supporting characters (JRPG, Multiplayer Shooters, etc.), or by a chatty main character (e.g. Spiderman, Uncharted series). These can also be confusing if the fight scene is already well populated with sound effects and loud music, thus timing, volume, a healthy priority system for your sounds, and a reasonable ducking/sidechaining system are important to be done right, so the execution sounds sound as natural as possible.

Further notes for dialogue

• When text-to-speech is active in the game, it has to be ducked down by any in-world dialogue. The in-world dialogues are considered a higher priority than the text-to-speech function.
• When a pop-up training/tutorial message is placed on screen, do not remove it until the suggested buttons have been pressed. When using a side character’s dialogue to train the player, have the character repeat the training lines (using more variations to break down cognitive obstacles and to avoid repetition, such as in RDR2 or It Takes Two), until the required action has been done by the player. This way, you ensure that the training material was understood.
• UI narration is one of the most important voice aspects that allows blind gamers to play a game. (Not only the menu characters need to be recognised, but the gameplay UI characters as well. The player needs to know how many bullets they have, how much health they have, their special powers, etc…).
• Make all characters recognisable by speech synthesiser applications (whether it’s the built in system for PS or Xbox, or PC apps like Mars Vision, UAP, Twilio, and further Amazon apps).
• Hire companies (or professional voice actors) to do your game’s audio descriptions (a few providers: Vocal Eyes, WGBH, Deluxe, plus here you can find further info about Descriptive Video Service in general).
• If possible, have a separate dedicated button on the keyboard or controller to toggle text-to-speech on/off (or make all buttons assignable and create an option for the text-to-speech toggle).

Sound Effects

Navigation/Traversal

Sound effects have a big role in traversal accessibility. Functional audio cues, such as a ledge guard to prevent falling – along with audio and vibration feedback, enhanced listen modes to highlight nearby enemies and objectives within a radius of the player, and other useful traversal and combat-related cues – can drastically increase the accessibility of any game. These are especially important in navigation assistance, both in-game and in the game’s menu. For menu navigation, speech and UI narration is preferred; however, sound effects can still carry cognitive information, such as a sound for toggling something on or going deeper in the menu (e.g. Playstation’s menu UI sounds), and the reversed version of the same sound when toggling something off or coming back out from the menu.

For in-game navigation, the most commonly used audio cue is a sonar-style sound to mark the objective’s echolocation in the direction of the next objective or correct path (with 2D stereo panning or 3D object-based sounds). If there is an obstacle that prevents the player’s smooth navigation, it needs to be marked with additional audio cues to advise the player, such as in TLOU2.

When the enhanced listen mode is triggered, a sonar sound sweeps through the area around the player and nearby enemies become more audible, filtering out all other ambient sounds.

Players can audibly scan for and automatically navigate to enemies in The Last of Us Part 2

When the player is close to an interactable/collectible object, using an attenuated audio cue that loops until it has been collected or interacted with is currently the best available system to point the player to the direction of the required object. For 3D object-based sounds, you can get creative by attaching a long amplitude to the distance parameter, so that the closer you get the sharper the sound becomes, highlighting that the player is getting closer to the object. TLOU2 uses a transient-less, soft looping UI beep, and once you are close enough, the loop event stops and plays the same sound but with its transient. If the player leaves the trigger-box radius and the objective wasn't fulfilled, the subtle beep starts again.

For other objects along a pathway (like a door, a narrow squeeze-through, a ledge, or in the water), the UI audio cue can be accompanied with a sound that reminds the player of the interactable object (e.g. a UI “Heads Up!” sound is followed by a door creak when the player approaches an unlocked door, or it plays the rumble of a lock if it is locked). For further details about the creation and correct use of action indicator sounds, check out my findings on iconosonics.

If a button needs to be held to achieve the next target, the suggested action sound effect can have a reverb or delay tail, or a longer sustain. For example, if pressing O = crouch or holding O = prone, then if the situation needs the player to prone, the crouch sound cue can have a longer tail, which will cognitively suggest that the same button needs to be pressed longer, like the sound.

To ensure the player receives audible feedback about whether their action was successful or not, confirmation sounds need to play when the job is done or when the next target is reached (suggesting “Good job!” *shoulder tap*). Nudging sounds can be played after cut scenes and in-game cinematics when the player needs to be told that an action is required (like pressing forward or if the player needs to press a specific button).

In the following video, after Ellie reads a letter, the game tells players with a subtle ‘woop woop’ sound at 51:36 that they are now good to move on.

A good example of accidental player navigation accessibility can be found in Gears 5 in which there’s a fabricator in Horde mode that needs to be protected by the player. When using the fabricator’s locational ping, it will play back in a pitch that changes based on the player’s distance. The closer the player gets to the fabricator, the higher the ping’s pitch becomes.

Also, in Hades majority of the game-world objects are 3D spatially-positioned sounds: enemies make noises while fighting, and music and sound effects are ducked down when dialogues are active. After killing enemies, collectibles play a looping heartbeat sound until the player collects it.

Puzzle/Objective Hints

As explained with nearby interactable/collectible objects, objective hints can play looping cues or one-shot-style sounds. These sounds need to work closely with the game mechanics and cognitive logic, and they can be 2D UI sounds or 3D objects, depending on the style of the game.

These hints should be separate from the traversal sound family, as they suggest more quiz-like activities. They don't necessarily fall under the accessibility category, as they are used in many games (like the Uncharted series), but while a developer usually triggers these puzzle hint sounds after a period of time or when a number of wrong actions have been performed, the sounds can be triggered immediately or after the first missed action when the accessibility function is turned on. This is a difficult function to get right, because the developers still want to challenge the player, and the VI player also wants to be challenged, thus it is important for the developer to bear in mind that there are probable elements of the puzzle/objective that need to be explained to the player so they fully understand what is expected from them.

Combat/Quick Reflex Hints

Just like in the dialogue cues, these combat sound effects can also be confusing if the fight scene is already well populated with in-world sound effects and loud music, thus timing is important to be done right. These sound effects need to be distinguished from other combat sounds so as to have a 'heads up!' kind of feel to them to cut through the already busy mix. Therefore, it is probably wise to keep these hints 2D stereo panned or mono.

The other approach to this is to attach the sound to a button logically, like a dodge button that needs to be pressed at the perfect moment. In these cases, if the dodge button (in a fight state) is the same as the crouch/prone (in a roaming state), then a more aggressive version of the traversal button sound can be played, as we are just borrowing the button sound from the travel function that is more important in a combat scene.

Aim assist

Highlighting enemies when aiming can also be shown with an audio cue. Based on my research, TLOU2 so far has the best audible aim assist system, which uses a constant looping pitch that turns on while aiming, then the pitch goes up (as well as its volume) when an enemy is in the crosshairs, similar to a metal detector.

The aim assist audio cue plays when Ellie locks onto an enemy at 9:34

In recent shooter games, bullets that hit the target get an extra layer on top of the original gun firing sound: a short white noise or high frequency sound that works as a feedback for successful bullet hits. There are multiple ways of achieving this; it can be a switch that is attached to the scoring system (and selects the high frequency sample when a target is hit) or a volume parameter placed on the high frequency bullet sounds that only opens up when a target has been hit. It can also be coded directly in the game engine to separate the empty and successful bullet hits with two separate events. Additionally, when an enemy is killed, a dedicated “kill stinger” can be played to give feedback to the player of the successful attack (such as in Far Cry 5).

Punches, kicks and other fight moves should also sound different when they reach their target, as opposed to when they hit the air. Mortal Kombat 11's most appreciated "accessibility feature" (apart from the narrated menu) is the fact that every single fight move has a distinct sound. This allows VI players to compete and beat their sighted friends – therefore, they can enjoy the game together!

Music

Navigation/Traversal

Certain places can be separated by music states. For instance, let’s say there's a beach somewhere, then there's a mountain, then a village; these can all be separated by using music states. Additional layers of music can be spatialised (3D object based), thereby suggesting path openings. For instance, music was designed and implemented very cleverly as part of a warning signal in the game A Plague Tale: Innocence (2019). When the player is roaming in the game, the music is generally rhythmless and behaves more like a calm ambience bed to match the stealth nature of the game. As the player moves closer to enemies, certain instrument layers of the music change drastically, and rhythmic cello and percussion become gradually louder and more frequent in the mix. In a short time, the music that was used as a narrative colouring tool instantly becomes a built-in accessibility audio cue that helps the navigation of both sighted and VI players in the game world. You can also suggest new objectives by luring the player closer by spatialising the instrumentation (e.g. distant drums aligned with the music's BPM).

Other examples of clever compositions are the rising arpeggio in the movie Up! when the balloons lift the house and a falling arpeggio when this house is falling back to the ground. They’re all situation dependent, but no one should underestimate the power of well-composed (and implemented) music, a.k.a. ‘Mickey Mousing’, and its effect as a narrative tool (Rayman Legends does it very well, and it makes the gameplay super fun).

Puzzle/Objective Hints

Just like in the sound effects section, music can suggest player actions and motivate them to make decisions in a quiz-like environment. Just as how Hans Zimmer explains the question/answer method in his masterclass, these compositional techniques can serve an incredibly helpful and entertaining function in games as long as they are well implemented. And of course, depending on the game state, if the player is in a puzzle state, the right music (such as the below track from Mass Effect 2) can inspire and stimulate players to do puzzles with joy.

Combat/Quick Reflex Hints

The majority of action-driven, shooter and stealth games change their music between three main states: combat, roaming and cutscenes. In Hades (2020), the music changes between two main states – roaming and combat – however, there is a third state where there is no (or very quiet) music presence, as the game channels the players’ focus onto the game’s story using dialogues.

Switching between musical states can also be made more interesting with the use of transition tracks. It is advised to use variations in the music layers/instruments, which can randomly repeat to keep the music interesting.

Ambience

Navigation/Traversal

In a 3D game, if sound effects are done well with attention to detail, they already provide an environment that is easier to navigate. Spatial rooms, convolution reverb, accurate amounts of reverb on the dialogues and audio objects – all these things together can greatly help in making the game environment easier to navigate. A beach for instance will have water sound, seagulls, boats and perhaps people, while a cave will have a long reverb, echo, slow dripping water but general quietness.

The environment in games can be designed to behave like a navigational tool, just like wind, a river, an echo in a cave, or a little creature that the player needs to follow. By using splines, developers can set a track for a game object to travel on. A spline is an ideal tool to carry the sound emitter in space.

Puzzle/Objective Hints

The accurate use of reverb and filtering can play a big role in puzzle solving if the level design takes into account sound spaces. For instance, in the game Superliminal (2019), when an item grows huge, its sound becomes lower and heavier as opposed to when it’s small and higher pitched. When these sound effects are routed through the room’s space, the reverb and echo change with the size of the object, thereby suggesting its size.

Combat/Quick Reflex Hints

In games where the environment can be used in combat scenes, for instance to drown an enemy in water or push them off a ledge, the environment can suggest and represent these abilities with sound, so the player can cognitively connect the sounds with the action required.

Create Your Own

It is clear that there is no universal way of making games more accessible for the visually impaired. In fact, the use and implementation of accessibility audio cues are so diverse that it was advisable to separate the subject into four main categories.

The use of dialogues and text-to-speech synthesisers, or at least designing the in-game text to be compatible with screen reader applications, are found to be the most important aspect of video game accessibility for the visually impaired. If UI narration was implemented in future games, game tutorials would become easier to understand for VI players.

By using well-localized audio cues with accurate convolution reverb, attenuation curves and spatial information, navigation can be made much simpler for VI players. The ‘pingable’ sonar sound was found to be the most effective navigational tool within the sound effects category.

And by using dynamic music, a game is able to communicate the game’s state and narrative with players. Moreover, it was shown that music can also be used to support game mechanic-related messages with the use of musical stingers and warnings to help player navigation.

The majority of indie game developers are simply not aware that there are existing regulations in the game industry that their game needs to comply with. By creating a shorter, summarized version of my findings, I hope information conveyed in this article can be shared among game developers and game audio professionals to develop games with accessibility features included with the aim to support VI players gaming experience in the future. And remember: these are only guidelines and existing techniques to encourage sound designers in the industry to come up with their own solution of creating accessible audio systems.

Accessibility Consultants

Brandon Cole

• Twitter
• Twitch
• Youtube
• Web Site

LaMoine Williams

• Twitter
• Youtube
• Web Site

Literature and useful links

• The Last of Us Part II accessibility settings
• CVAA guidelines relating to audio (21st Century Communications and Video Accessibility Act
• Full list of the Game Accessibility Guidelines
• About game accessibility generally
• Accessibility on a budget
• Audio Description organizing principles
• What’s it like to play games when you’re colour-blind
• Accessibility in gaming should be the rule, not the exception
• Accessibility and “difficulty” aren’t the same thing

A big thanks to Máté Moldován for his helpful report about how sound designers can make their games more blind accessible! You can follow him on Twitter, LinkedIn, or visit his website.

Please share this: