Pure Data

This post is about the tool OFFBEAT uses to generate and manipulate audio. If you’re making a platformer, every game engine has ready-made tools for handling collision and gravity. But it turns out, there aren’t any out-of-the-box solutions if you want to make a game about simulated playable synthesizers. Very early in the life of OFFBEAT (before it was even called that), I had to figure out how we were actually going to make the sounds. You might think that the giants of game audio middleware would be our friends here, and while it is true that your FMODs and WWISEs have built-in instruments and effects, none of them had the flexibility and breadth for what we had in mind. And at the same time, building an entire audio engine from first principles seemed like a frightening time-sink of a proposition.

So I asked myself - who else has done it? We’re certainly not the first people to put a playable synthesiser in a Unity game. And by blessed coincidence, around this time, I learned that it was the 10-year anniversary of a wonderful game called FRACT OSC. It was this video (plus a zoom call with the dev, who was super generous with his time - cheers dude!) that cued me into Pure Data.

Now, don’t panic. I know that the name makes it sound like some kinda heinous spreadsheet tool, or an illegal brain-drug from a cyberpunk dystopia, or a youtube channel about optimising your conversion funnel. What it actually is is an idiosyncratic, complex, somewhat academic tool for realtime audio processing and synthesis. It’s the holy grail if you want to make your computer go “beep” in lots of highly specific ways. The real power of this system is that you can implement virtually any synthesis technique - sample-based, granular, FM, subtractive, Karplus-Strong, wavetable, whatever - as well as any effects you dream up. It can also record audio and manipulate prerecorded sound. The whole sandwich, basically.

The core tool was created by genius luminary Miller Puckette, who was also involved in the creation of the equally-arcane Max/MSP. However, Max/MSP, Pure Data is free and open-source and has an equally open-source Unity integration. It’s incredibly powerful. We’re using it to build what basically amounts to a full DAW with a suite of instruments, but people use it to make all kinds of things - VSTs, art installations, data visualisers.

In practice, you can think of it as a visual scripting tool. Or, if you’re that way inclined, you can think of it as a modular synth with control signals and audio signals, but where the modules are all very small and specific. Here’s a look at one of ours:

This is the core of the in-game Monosynth. It uses a standard subtractive synthesis architecture. The main signal path runs from top-to-bottom along the left hand edge of the image. A MIDI note value comes in at the top, which is fed to both the “pd voice” and “pd sub” nodes, which in turn generate musical tones. Those tones are mixed together by the pair of “*~” nodes, which I use all over the project to manipulate audio volume. From there, they’re passed into the “bob~” node, which is an emulation of a filter from a very early Moog synth, and it tells you a lot about the developers of Pure Data that this is one of the nodes included in the basic set of tools. And after that, the sound is sent into a final stage where an envelope generator sculpts individual notes. All that gubbins on the right is receiving and manipulating control data from elsewhere in Pure Data, or from Unity itself. That’s how the various dials and sliders in the game interact with the system. It’s also where most of the complexity lies, not only with our game’s implementation, but also with designing synthesizers in general.

Pure Data also has a fantastically active and generous community. It’s surprising to find so much activity surrounding a piece of obscure software that’s almost 30 years old, but the core tool is still in somewhat active development, as far as I can tell. It has a busy subreddit, and people are posting new tutorials on youtube all the time. While I wrote nearly all the patches in OFFBEAT from scratch, there are a few places where I’ve been able to save headaches by using patches shared by members of the community. The gnome, for example, is built around a wavetable player that I found in this repo. I’m also extremely indebted to the pd-tutorial guide by Johannes Kreidler, and make extensive use of the architectures described in the book.

Due to the nature of how the Unity wrapper works, all our Pure Data patches and samples are accessible in OFFBEAT’s project files, meaning curious players can freely explore and modify them. Hell, it’s also easy to find and modify the game’s save file. We’re completely cool with all of that. It’s true to the spirit of the game. My only real reservation is that some of the pd files are pretty untidy, but for that I can only apologise. It’s particularly true the first ones I made. The envelope generator in the Monosynth is something I hope I never have to touch again.

Anyway, I suppose the real agenda with this post is just to give a little insight into how the game’s audio works on a technical level, and to actively bless and endorse players who might want to dig into the source files themselves. Pure Data is great, and I expect I’ll be using it for all sorts of future projects, music-based or not.

The next post will be something less nerdy, I promise.