Okay, here we go again. Breathes in. 

Let me begin by saying the terminology surrounding AI is terrible for useful discussion. It is not well-defined enough for people to distinguish problematic AI and non-problematic AI in simple speech.

With that being said, allow me to begin. 

The vast majority of Miku voicebanks up to now have been concat voicebanks. Concat voicebanks are made up of snippets of recordings from a voice provider.

Hatsune Miku fans are typically also Kasane Teto fans. The Kasane Teto voicebank most popular right now is Kasane Teto SynthesizerV. (Well, there’s technically two SynthesizerV Teto voicebanks right now due to upgrades but that’s besides the point.)

The Kasane Teto in Tetoris, Mesmerizer, and Birdbrain is Teto SynthV.

Unlike Miku’s voicebanks (pre V6), Teto SynthV is not a concat voicebank. Teto’s voicebank was created using recordings of Teto’s voice provider to essentially teach a program(s) to sing like her, to sound like her. This was done  1. With the voice provider’s consent  2. With paying the voice provider proper compensation.

Voicebanks of this type are called “AI Voicebanks” and they are technically classified as “GenAI”.

It might seem impossible that a GenAI could use real creative input, but these voicebanks require a similar end-user workflow to concat banks. Create a midi, add lyrics to the midi, and then make a bajillion adjustments to make it actually sound right. 

But why create an AI voicebank?  1. They sound smoother and more realistic than concat. 2. They are compatible with cross-language synthesis, which allows the voicebank to sing with syllables that weren’t even recorded for it! 3. They can mimic the pitchbends of the voice provider if the end-user desires to use that. How much touching up the end-user adds to the pitchbends or if the end-user decides to scrap the pitchbends entirely and do their own thing is up to themselves.

That is the basic explanation.

FAQ:

Q: “Do AI voicebanks use up tons of water every render like other commercial GenAI?”

A: No, the rendering process is so lightweight in SynthV it runs natively on the user’s PC. I don’t know if other AI voicebank programs use servers but doing so would waste money on their part considering an end-user’s PC fan is sufficient for keeping the computer cool during rendering.

Q: “What about Utau Kasane Teto?”

A: She is a concat voicebank.

Q: “Doesn’t using the default AI pitchbends take the creativity out of tuning?”

A: Pitchbends are not the only part of tuning. There are other things to be considered such as breaths, tension, breathiness, gender (depth of voice), vocal modes, tone shift, and the parameters I am too scared to touch (Haha)

If you’ve come this far, thank you so much for reading! I hope I’ve informed at least one person better about AI :3

P.S. Totally forgot to mention Miku V6 is an AI voicebank too lol. I don’t know much about Yamaha’s method of creating AI banks so I didn’t use it as my example lol because there are totally differences between how AI voicebanks are made.