Music games have been popular for some time. One of the most common ways that those games utilize music as game mechanics is to design the gameplay based on existing music composition, which in many cases is synchronizing player actions with music rhythm. Games such as Guitar Hero or the Taiko series took this approach. In those games, players often see a bunch of bars or icons moving along the screen based on music beats, and the gameplay is limited to pressing right buttons when corresponding bars reach the end of their lane. Such design indeed takes advantage of music rhythm but at the same time constrains the player agency. Nowadays as music and audio are playing more important and diverse roles in media content, what are some new ways to design game mechanics based on music?
Guitar Hero (2005-2015)
Taiko No Tatsuji (2001-2019)
Common Types of Audio-based Gameplay
Before focusing our lens on music, let’s take a look at audio in general as gameplay. Aaron Oldenburg, a professor at University of Baltimore, wrote a journal article discussing intersections of music, sound arts, and video game designs. Through looking at the history and development of those three fields, Oldenburg suggests that common types of audio-based gameplay are either built around pre-composed music pieces or built to allow players to generate “dynamic music” with their actions, leaving other forms of sound under-explored in the field of games. As he then proceeds into analyzing a few recent experimental audio games, Oldenburg suggests new possibilities in utilizing sounds as means to unfold game space to visually impaired audiences, in creating chance-based composition with gameplay, and in simulating sounds in games without reproducing them, etc.
Oldenburg mentioned a wide variety of audio-based games and projects that range from PaRappa the Rapper, which follows the conventional rhythm game design of pressing buttons, to more recent experimental games such as Optic Echo, which uses audio as the input and visualizes echolocation. When looking at music-based gameplay, however, some approaches–such as using existing compositions and generate “dynamic music”–could be more applicable than others. Certainly, one can build experimental experiences around visualization or using sound as the input, but this approach doesn’t take advantage of what’s unique to music as opposed to general audio—it’s structure (rhythm, harmony) and flow (melody, progression).
PaRappa The Rapper (1996)
Optic Echo (2011)
Unconventional Music Games
While most music games would utilize rhythm as mechanics, they are a number of them who did it in a way that doesn’t constrain player agency to merely “pressing the right button at the right time”.
| 140
140 is a minimalistic platform game that creates synesthesia with electronic music as players proceed through levels. In each of its level, the platforms, obstacles, and traps change their colors, positions, and even shapes along with the rhythm of the background music. The player will have to synchronize their actions with the rhythm pattern to be able to proceed in a level.
Created by Jeppe Carlsen who directed gameplay for the well-known Playdead’s Limbo, 140 is a successful example to combine music rhythm with mechanics of a platform game. While players have the freedom to move their character within the game space, the strategy to solve puzzles or to get over traps and obstacles is to some extent defined by the music rhythm. Such mechanic therefore creates a balance between player agency and the significance of music rhythm: the rhythm defines your strategy but not your movement.
140 (2013)
Klang (2016)
| Klang
Klang is an innovative rhythm game that aim to push the genre out of the convention of merely pressing right buttons at the right time. Set in a virtual world of music and beats, players play as a hero who uses two tuning forks as weapons and navigates through space while eliminating enemies according to background music rhythm.
Similar to the design of 140, in Klang there are traps in each stage that change along with music. Players have to time jumping and moving with the beat pattern to be able to proceed. During the combat phase, there will be enemies floating around the character. In that phase, enemies would stay at the same relative positions to the player no matter where the player goes. The enemies shoot projectiles according to the music beat, and players will have to deflect the projectiles back to enemies by pressing corresponding direction buttons when they come close.
Though Klang aims to push rhythm games beyond pressing buttons and it does well-incorporated the action aspect of platform games, the fact that the battle phase is constrained to accurate button press still resembles the old convention. With their combat UI design resembling the look of conventional rhythm games, the mechanics of Klang somehow feel like an arbitrary combination of rhythm and action.
| Patapon
Patapon is a Japanese rhythm action game where players command an army of soldiers by playing different drum patterns. Players can tap four different types of drum beats, and depending on the order and the combination of those beats they would be able to commend their army to do things such as charging, dodging, retreat, and attack, etc. While players have absolute freedom in terms of when to give those commends, the closer they align their drum beats with the background rhythm they better their army will perform the commend.
Although the player actions in Patapon still take the form of accurate button press, the player agency is not constrained by the music rhythm because the gameplay lies heavily on when and what commends to perform in different situations in addition to accurate button-pressing. The strategy and decision-making added depth to the rhythm-based actions.
Patapon (2007)
Direction to Experiment
As a huge fan of action games and a musician, those unconventional music games inspired me to explore and experiment with combining music game mechanics with action game elements. What are some other interesting ways of making rhythm-action mechanics? And, while most music game tend to utilize rhythm, how would one use other qualities such as melody or harmony to build gameplay?
| A Platformer about Harmony and Music Chords
When I first started making games two years ago, I experimented with combing music chords and harmony with platformer mechanics. I built a prototype called Monic in which players choose the right ability to shoot at enemies based on the music chords they hear. In that world, every type of enemy and the player ability have their own sound–sometimes it’s just one note and sometimes it’s a chord. Players will only be able to kill an enemy with the ability that has the note which harmonizes with the chords of the enemy.
Different from identifying music rhythm, recognizing chords and harmony has a much higher bar–sometimes it’ll take a few years of training for a person to be able to hear the nuances. Therefore, a game built around that wouldn’t be so playable. When making Monic, one thing I did was to add color cues that corresponds to each chord, so that players can rely on the colors to make their choices even when they fail to hear the nuances.
In the end, this prototype became a game that can be played purely by matching the colors. And music chords lost their significance in the gameplay. If I were to further iterate on this concept, the use of color cues could be limited to only a few levels where players can rely on them to learn about hearing the nuances. The game could slowly progress into the stage where colors are removed and players have to really rely on hearing.
Summary
To summarize my point, while music rhythm is very over-utilized as a game mechanic, there have been innovative ways to break it out of the convention of accurate button press. Beyond rhythm, there are also many other qualities of music and sound that are under-explored as game mechanics. While they might be inherently more difficult than rhythm to get good at recognizing, the integration could be designed to softly introduce these music concepts without overwhelming the player. And if designed properly, these games could even have the potential for players to learn about those music concepts and to strengthen their music instinct.