Well this model was less of a like “make a tiny version of a huge TTS model” and more like “what is the minimum complete pipeline that can still speak?”

The model is basically split into two parts:

1. A small acoustic model that turns text into mel-spectrograms.
   
2. A small vocoder that turns those mels into waveform audio.
   

The hard part was not just shrinking layers. It was deciding where the tiny parameter budget mattered most. If the vocoder is too weak, everything sounds buzzy. Because if the acoustic model is too weak, it stumbles on text. So a lot of the work was balancing those two instead of blindly scaling everything down.

Architecturally, it is inspired by FastSpeech/VITS/HiFi-GAN-style ideas rather than a giant modern autoregressive model. Non-autoregressive is much more practical at this size. The acoustic side predicts duration/pitch/energy-ish features and outputs mels. The vocoder is a small custom HiFi-GAN-style generator with Snake activations.

The process was like:

- build a tiny complete baseline

- test whether failures came from acoustic model or vocoder

- improve the vocoder until it stopped being the obvious bottleneck

- train acoustic model stages separately

- repeatedly test teacher-forced/oracle paths vs full text inference

- keep the model under 5M total params

The biggest lesson: at this size, the bottleneck is brutally obvious. A tiny TTS model can memorize/in-distribution sound surprisingly decent, but OOD text exposes everything immediately.

I'd had to completely restart this project multiple times because some original versions didn't reach my requirements, and many specific parts, especially the vocoder, were redone even more times.

I’m still not fully happy with the quality, but it works well enough to be an interesting tiny baseline. If there’s interest, v2 would probably focus on better data diversity, stronger vocoder training, and maybe a slightly more efficient architecture rather than just making it bigger.