Harmonizing Music and Sound Effects using FMOD Studio

A recent trend in game audio is blending the sound effects with the music, ensuring that a sound harmonizes with the score at any given moment. A recent example would be Super Mario Odyssey.

When Mario rides the power lines pictured above, the sound of the electrical current follows the chord progression of that world's music. Using just a little bit of code, FMOD Studio allows its users to achieve this effect.

I employed this technique in my own project, Alatheia, which I will be referring to for this guide. In the game, there are collectibles that mark the path to the end of each level. Being that the player picks these up quite often, I wanted to add a layer of interest. I opted to make the collectible sound a harp tone from the current chord of the background music.

Picking up collectibles in Alatheia

To set this up in FMOD Studio, create your music event first. Once that is functional, create an empty event for the collectible, and a parameter named something like "music_time". Make the range of this parameter the duration of your music event (in seconds). Since we don't know when the player will pick up a collectible, we don't know what sound to play without some information. If we knew where the playhead was on the music event timeline, we could use that information to decide what note to play.

In the FMOD Studio API, there is a function called getTimelinePosition. As the name implies, it will return the current playback position of the event passed to the function. Using this method, we can ascertain where the song is, and that will help us build our collectible event.

Music event from Alatheia

Corresponding collectible event for the above music track

The units for the parameter represent the timeline position of the music event. When the player picks up a collectible, getTimelinePosition is called to determine where in the song the music event playback cursor is. Then, that value is fed to the Music Time parameter, pictured above. Now, the correct multi-instrument has been selected by the parameter. When the collectible sound event is played, FMOD will pick a sound from that multi-instrument, which in this case is a random note from the current chord.

The pseudo-code for the process is as follows:

1. getTimelinePosition("your music event")

2. setParameterValue("music time parameter")

3. Play("your collectible sound")

The syntax for getTimelinePosition can be project-dependent, but here is a link to the integration details: https://www.fmod.com/docs/api/content/generated/FMOD_Studio_EventInstance_GetTimelinePosition.html