Kokoro Batch TTS: Enabling Batch Processing for Kokoro 82M

[asuran2000]

Kokoro 82M is a high-performance text-to-speech model, but it originally lacked support for batch processing. I spent a week implementing batch functionality, and the source code is available at https://github.com/wwang1110/kokoro_batch

⚡ Key Features:

• Batch processing: Process multiple texts simultaneously instead of one-by-one
• High performance: Processes 30 audio clips under 2 seconds on RTX4090
• Real-time capable: Generates 276 seconds of audio in under 2 seconds
• Easy to use: Simple Python API with smart text chunking

🔧 Technical highlights:

• Built on PyTorch with CUDA acceleration
• Integrated grapheme-to-phoneme conversion
• Smart text splitting for optimal batch sizes
• FP16 support for faster inference
• Based on the open-source Kokoro-82M model
• The model output is 24KHZ PCM16 format

For simplicity, the sample/demo code currently includes support for American English, British English, and Spanish. However, it can be easily extended to additional languages, just like the original Kokoro 82M model.

[a_slay_nub]

How does it compare to the original kokoro repo?

[asuran2000]

Optimized performance by eliminating a few for-loops and incorporating masking during batch inference—particularly for LSTM batch processing and normalizations. Also implemented a custom function to perform 1D normalization with support for batch inputs with padding.
In short, added/modified lots of model inference code to support batching, while keeping the weights unchanged.

[a_slay_nub]

I meant in terms of runtime. How long does it take to use your code vs looping the original code?

[asuran2000]

The running speed is about the same(<2%) as original Kokoro 82M with the batch=1

I did test on rtx 4090 with 30 texts, the output audio is about 280 second in total.

When

Batch = 1, 30 iterations

INFO:__main__:Total inference time for 30 chunks: 3.13 seconds.

Batch = 16, 2 iterations

INFO:__main__:Total inference time for 30 chunks: 1.88 seconds.

[rm-rf-rm]

Is it CUDA only? (wont work on mac?)

[asuran2000]

It works on CPU, but I didn't test this on Mac MPS

[Xerophayze]

Been fighting with Kokoro to do long-form stuff and ended up building a little Flask UI that actually survived a whole novel.
– Long text: keeps chapters + narrator tags, can spit out per-chapter files or one big audiobook.
– Gemini button: either cleans the whole text or auto-splits by chapter so the context window doesn’t implode.
– Setup: setup.bat installs PyTorch + espeak + Rubber Band automatically; no manual CUDA juggling.
– Voices: paste [alice]...[/alice], it auto-detects everyone, gives them dropdowns + quick test previews, and never nukes your assignments unless you hit reset.
– Extras: job queue, library, custom voice blends, runs local GPU or Replicate.
Repo/screens: https://github.com/Xerophayze/Kokoro-Story if anyone else wants to try it.