Using music as a timing source for demos
This is part 7 of the JavaScript MOD Player series. If you haven't read the first part, here it is: Generating sound in modern Web Audio API
When it comes to the demoscene productions, the real-time effects realized by a programmer, the graphics and the music are all very impressive separately, but the real magic happens when they are combined together. The music is often the glue that holds the whole production together, and a demo with good timing is always a joy to watch. Often, the music is used as a timing source for the demo, and events in the music are used to trigger transitions and effects.
Subscribing to every row
The first subscription I will implement is to be able to subscribe to every row. For some demos that do a lot of music-triggered effects, this will be very useful. To implement this, the worklet thread will notify the main UI thread for each row, and the main UI thread will be responsible for actually calling the callback functions. Communication between the two threads is done using the MessagePort, which I already use for starting the music. I will take this opportunity to first refactor the code, and create a new class called ModPlayer, which implements the UI thread side of the MOD player.
import { loadMod } from './loader.js';
export class ModPlayer {
constructor() {
this.mod = null;
this.audio = null;
this.worklet = null;
this.playing = false;
}
async load(url) {
if (this.worklet) this.unload();
if (this.playing) this.stop();
this.mod = await loadMod(url);
this.audio = new AudioContext();
await this.audio.audioWorklet.addModule('player-worklet.js');
this.worklet = new AudioWorkletNode(this.audio, 'player-worklet');
this.worklet.connect(this.audio.destination);
}
unload() {
if (this.playing) this.stop();
this.worklet.disconnect();
this.audio.close();
this.mod = null;
this.audio = null;
this.worklet = null;
}
async play() {
if (this.playing) return;
if (!this.worklet) return;
this.audio.resume();
this.worklet.port.postMessage({
type: 'play',
mod: this.mod,
sampleRate: this.audio.sampleRate
});
this.playing = true;
}
async stop() {
if (!this.playing) return;
this.worklet.port.postMessage({
type: 'stop'
});
this.playing = false;
}
}import { ModPlayer } from './player.js';
const modPlayer = new ModPlayer();
const loadButton = document.getElementById('load');
const playButton = document.getElementById('play');
const stopButton = document.getElementById('stop');
const unloadButton = document.getElementById('unload');
loadButton.addEventListener('click', async () => {
// Load Elekfunk from api.modarchive.org
const url = 'https://api.modarchive.org/downloads.php?moduleid=41529';
await modPlayer.load(url);
});
playButton.addEventListener('click', async () => {
await modPlayer.play();
});
stopButton.addEventListener('click', async () => {
await modPlayer.stop();
});
unloadButton.addEventListener('click', () => {
modPlayer.unload();
});<!DOCTYPE html>
<html>
<head>
<title>JS Mod Player by Anders Marzi Tornblad</title>
</head>
<body>
<button id="load">Load</button>
<button id="play">Play</button>
<button id="stop">Stop</button>
<button id="unload">Unload</button>
<script type="module" src="main.js"></script>
</body>
</html>After this refactoring, it's time to add the event subscription mechanism. The worklet thread will send a message to the main UI thread for each row, and the main UI thread will call the registered callback functions.
class PlayerWorklet extends AudioWorkletProcessor {
constructor() {
super();
// ...
this.publishRow = false;
}
onmessage(e) {
switch (e.data.type) {
// ...
case 'enableRowSubscription':
this.publishRow = true;
break;
case 'disableRowSubscription':
this.publishRow = false;
break;
}
}
// ...
nextRow() {
// ...
if (this.publishRow) {
this.port.postMessage({
type: 'row',
position: this.position,
rowIndex: this.rowIndex,
});
}
}export class ModPlayer {
constructor() {
// ...
this.rowCallbacks = [];
}
async load(url) {
// ...
this.worklet.port.onmessage = this.onmessage.bind(this);
}
onmessage(event) {
const { data } = event;
switch (data.type) {
case 'row':
for (let callback of this.rowCallbacks) {
callback(data.position, data.rowIndex);
}
break;
}
}
watchRows(callback) {
this.worklet.port.postMessage({
type: 'enableRowSubscription'
});
this.rowCallbacks.push(callback);
}To use this feature, the user can call watchRows(callback) and pass in a callback function that will be called for each row. I have updated the index.html and main.js files to demonstrate this feature.
<!DOCTYPE html>
<html>
<head>
<title>JS Mod Player by Anders Marzi Tornblad</title>
</head>
<body>
<button id="load">Load</button>
<button id="play">Play</button>
<button id="stop">Stop</button>
<button id="unload">Unload</button>
<br>
<button id="watchRows">Watch rows</button>
<br>
<input type="text" id="output" size="40">
<script type="module" src="main.js"></script>
</body>
</html>// ...
// Store the new button and the output element
const watchRowsButton = document.getElementById('watchRows');
const output = document.getElementById('output');
// ...
// Add a click handler to the new button
watchRowsButton.addEventListener('click', () => {
// Call the watchRows method and pass in a callback function
modPlayer.watchRows((position, rowIndex) => {
// Update the output element with the current position and row index
output.value = 'Position: ' + position + ', row: ' + rowIndex;
});
});Subscribing to known points in time
The next step is to be able to subscribe to known points in time. With this feature, a demo coder can subscribe to a combination of position and row, and a callback will be called when the music reaches that point. For this, I will need to store the callbacks in a dictionary, where the key is a string that represents the position and row. Because there can be multiple callbacks for the same position and row, the values of the dictionary will be an array of callbacks.
export class ModPlayer {
constructor() {
// ...
this.singleCallbacks = { };
}
onmessage(event) {
const { data } = event;
switch (data.type) {
case 'row':
// Call all the general row callbacks
for (let callback of this.rowCallbacks) {
callback(data.position, data.rowIndex);
}
// Call all the single row callbacks
const key = data.position + ':' + data.rowIndex;
if (this.singleCallbacks[key]) {
for (let callback of this.singleCallbacks[key]) {
callback(data.position, data.rowIndex);
}
}
break;
}
}
watch(position, row, callback) {
this.worklet.port.postMessage({
type: 'enableRowSubscription'
});
// Store the callback in a dictionary
const key = position + ':' + row;
// There can be multiple callbacks for the same position and row
// so we store them in an array
if (!this.singleCallbacks[key]) {
this.singleCallbacks[key] = [];
}
// Add the callback to the array
this.singleCallbacks[key].push(callback);
}Try it out
You can try the final version of the player here
Conclusion
Now I can use the event system to synchronize demos to the music. This will come in handy when I move forward with the Artsy refactoring. I will keep adding to the js-mod-player project as I move forward, and I will push the changes to GitHub. If you like my work, feel free to use it. The player is licenced under a Creative Commons Attribution-NonCommercial 4.0 International License, which means you are free to copy, redistribute, remix, transform and build upon my work, as long as you give appropriate credit, provide a link to the licence, and indicate if changes were made. You may not use my work for commercial purposes or apply legal terms or technological measures that legally restrict others from doing anything the licencse permits.
You can try this solution at atornblad.github.io/js-mod-player. The latest version of the code is always available in the GitHub repository.
Articles in this series: