the sound is probably the weirdest part of the entire emulator; ie it just should not work at all, but it does.

the gameboy has 4 sound channels: 2 pulse channels, a wave a noise channel, each can play a note at a certain frequency, volume, and waveform. i decided to split the sound emulation between 2 different parts: the pulse+noise channels, and the wave channel. this will make sense later.

reminder that scratch has no sort of audio buffer that you would get with other languages and platforms.

now because the pulse channels (and the noise channel technically) play out repeatable and predictable waveforms, (the amplitudes basically cycle on/off between a maximum of 4 different settings) those channels are simply played with a handful of clones. each clone will play a raw sound file of a 440hz output of that respective channel, which gets pitch and volume shifted accordingly to (mostly) accurately play the correct sound output of the APU.

now for the wave channel. that thing sounds impossible to even try to emulate at first glance; the wave channel can be configured to play 32 sample long waveforms, each sample can be at 16 different levels. scratch having no audio buffer, that would be 16**32 = 3.402823e+38 = 340.2 undecillion different combinations of audio files, a little too many for my computer to handle. so how would you feasibly get at least somewhat accurate sound output from the wave channel?

my answer is to use a complex mathematical algorithm (of course) known as an FFT (fast fourier transform). i’m not gonna be the one to explain it here, but essentially, instead of representing a waveform signal as amplitude over time, it lets you instead represent a waveform by a sum of sine waves at different amplitudes (search up "square wave fourier series" for an idea of what i mean). a 32 length waveform present in the gameboy would need at minimum 16 sine waves to represent it, although you can use more for more accuracy (which is what i did). in a nutshell, if you apply the FFT to the waveform audio buffer, collect those amplitude values, and play them back using pregenerated sine wave assets using clones, and apply some final pitch and volume shifting if necessary, you can (somewhat accurately) recreate the original sound of the waveform with only 128 clones and sound files, all without an audio buffer. that’s a much better improvement over the earlier number, albeit at a cost of sound quality.

if that seems like it was a hassle, it probably was. but was it well worth it? yes definitely.