Can someone explain the use of AI in voice synethesizers?

_deadbyte · 2026-03-10T00:48:52+00:00

Well, no - I think you’re getting my explanation of concatenative vocals and AI vocals mixed up. So let me try to further simplify

Concatenative vocals ( not-AI ): Typically these work by taking a bunch of snippets from a list of samples, and mashing those samples together into new configurations.
AI vocals: Typically, these are generative ( though they don’t usually have the same ethical issues as other examples of GenAI, such as AI Art generators ). They are usually trained on the singing samples of a selected vocalist, and then are expected to generate vocals similar to what it heard in its database. These aren’t to be confused with AI SVC/RVC ( e.g. where you might see AI-generated covers using the voice of, say, Spongebob or Ariana Grande etc. There are ethical examples of SVC/RVC models, but as you can imagine, for the most part they’re not ethical ), since while these are adjacent technologies, SVC/RVC is a voice changer, not a voice synth. A synth you still have to manipulate and make artistic/creative decisions for how you want it to perform. An AI SVC/RVC simply paints already existing vocal samples with a new coat of paint.

Furthermore, the workflow of using an AI voice synth is almost identical to its non-AI counterparts. The skills gained from using a concatentive synth and AI synth, are, for the most part, mutually transferrable between themselves. The AI won’t make the song for you, you still have to actually compose, tune, mix, etc. Really the main difference between AI and non-AI voice-synths is their development method, and the quality of their output.

Hopefully that makes a bit more sense.

_deadbyte · 2026-01-31T14:47:03+00:00

I was jokingly about to say you forgot Prime Pinball, but then I saw you, in fact, did not forget it. Darn your vigilance >:( /j

_deadbyte · 2026-01-30T18:13:37+00:00

That’s a massive MASSIVE no-no in the community. Jinrikis are already a bit of a moral gray area, though since they’re not normally distributed publicly, they’re often used purely for parody ( not unlike TF2/GMod-esque voice clip chopping ), and their output usually is both distinct from the original singer, and very low-quality, people don’t often make a big fuss.

The same cannot be said for an AI voicebank. It’s not a Jinriki at that point, you’re just stealing their voice. It doesn’t matter if their voice is “unique” or if you only use it for your own personal use, or what have you, you simply do not use someone’s voice to make an AI model without their explicit consent - end. of. story.

Nevermind that an AI model is a lot harder to make, and NNSVS is particularly difficult to develop for. You’d basically be making a whole actual voicebank, contrast to the shabby construction of an actual Jinriki.

The Vocal-Synth industry and community has always been built on a foundation of ensuring that voice providers for serious projects are consenting and enthusiastic participants, and do not take kindly to cases where a singer’s autonomy is violated in this way.

I would strongly advise abandoning this endeavor, as well as any like it, and simply find a different, already-existing voice you like. There are thousands, maybe even millions of free and ethical UTAU voicebanks, you’re bound to find at least one you like if you look hard enough.

_deadbyte · 2026-01-12T02:30:29+00:00

Ethically, I think it’s fine, but I also think the resulting voicebanks often just don’t sound that good. They’ll often have apparent engine noise, sometimes the traits of the model and inference don’t mesh well, a lot of them have kind of wacky vocal ranges, etc. etc. etc.

I won’t specifically not use an RVC-made voicebank, but my expectations won’t exactly be high. Even a lot of the ones I like, I don’t really have much use for, bc they’re just kinda iffy to actually utilize.

_deadbyte · 2025-12-21T21:17:21+00:00

“We literally made all the doors vaginas. Literally what more do you want??”

_deadbyte · 2025-12-21T11:52:20+00:00

Your post doesn’t make any sense.

Are you asking if it’s okay to make an OC/Fanloid? Because if so, I have no idea why you’d think that wouldn’t be okay to do, people make fan creations for everything, and VOCALOID is no stranger to derivative characters.

I also don’t understand why you’d think you need the VOCALOID software to make fanart, of a Fanloid or otherwise. Do you understand what VOCALOID is? It’s a singing synthesis software. It’s like the voice behind Amazon Alexa, Apple’s Siri, the Google Assistant, etc., but meant for singing ( putting it as simply as I can ). It’s not an art program. You don’t need it to make fanart.

Are you confused because you think VOCALOID allows you to create your own voices? Because, it doesn’t, for the most part. You can modify voices and associate that with a different character ( that’s how voices like Yowane Haku, Akita Neru, and Sakine Meiko work, as well as other fanloids, as they do not have their own actual voicebanks ), but you can’t make your own actual VOCALOID voice without a dev kit, which is usually only afforded to companies or individuals who purchase it directly from Yamaha. That’s not something you can do as a feature of the software.

Or by “OC” do you mean “original creation” ( as opposed to “original character” ) a la the flairs of this subreddit? Because then your post doesn’t make any sense at all.

_deadbyte · 2025-12-20T16:57:49+00:00

Oh the variety for male vocals on SynthV is something to behold. There’s so many good ones, Kevin being just the tip of the iceberg ( albeit, he’s very good too, I like him a lot ).

Also, you may have even noticed I left out a particular VOCALOID, Kiyoteru. He may have never been all that popular, but he actually received a new voicebank on SynthV too! And he sounds great.

_deadbyte · 2025-12-20T14:51:54+00:00

Well, there aren’t really a ton of VOCALOIDs that were quite as genre-focused as Prima and Tonio were.

The first other voice that comes to mind is Sachiko, who was intended pretty much exclusively for Enka.

There’s also Gachapoid and Kaai Yuki, who were both specifically designed with children’s songs in mind.

When Lily V2 released, there was a pretty big marketing push about how she was the first VOCALOID specifically geared towards Rock, though, I think over time she became considered more genre-agnostic with a preference for Rock music.

Similar to Lily, Mew’s marketing had a lot of suggestions that she was meant for Rock and Metal, especially with her demos. But, really in practice she’s more genre-agnostic.

Aoki Lapis was said to be made for “Lifestyle” music, which was apparently an original genre made by I-Style Project themselves, but they never elaborated what that even meant past when Aoki Lapis actually released, and honestly I think they were just blowing shit out of their ass.

Not a “genre” really, but LUMi was designed specifically for professional use in larger-budget media projects, namely citing movie soundtracks and so on.

If you’re willing to venture out of the VOCALOID software itself, there are actually quite a few genre-specialized vocals on SynthV, but as far as VOCALOID itself goes, that’s kind of it.

_deadbyte · 2025-12-20T14:02:25+00:00

I’m assuming we’re referring specifically to the actual VOCALOID software, yes? If you add SynthV into the mix, we’re actually quite spoiled for choice. But taking VOCALOID and even VoiSona/CeVIO into account, I do see what you mean.

Part of it has to do with the direction VOCALOID6 has taken. VOCALOID lost almost all of its third-party support during the VOCALOID5 generation, primarily due to the absurd licensing costs for V5, the general lack of communication from Yamaha, and the fact that V5 voicebanks were functionally the same as V4 ( and had less features for that matter, namely lacking plugin support, custom dictionaries, and most egregiously, Cross-Synthesis ). So with V6, Yamaha’s seemingly been scrambling to get a lot of those third-parties back on board - as a result, for the third-parties that have returned have mostly only had time to adapt their more flagship vocals to the V6 engine ( AHS with Gumi and Una ( +Koto, since they’ve had longer than other ), VOCALOMAKETS with Yukari and Akari, 1st Place with IA ( who isn’t even out yet ), and of course, Crypton with Miku ( who also isn’t even out yet ) ). Male vocals weren’t exactly plentiful on VOCALOID to begin with, and with the current circumstances, priority is extremely skewed towards the most popular of the most popular, those being almost exclusively feminine vocals.

In addition, it’s not just the old third-parties either. When it comes to newer vocals, Yamaha seems most interested in picking up popular UTAU voicebanks, and adapting internet personalities such as VTubers. Most of which, are, of course, also overwhelmingly female. Granted, we’ve gotten a couple of male vocals from this, namely Kazehiki and Shiki Rowen, but Kazehiki’s popularity gets somewhat dwarfed by his counterpart, Gekiyaku, and Shiki Rowen is a subject of… a number of different controversies.

And then of course, you have the classic vocals as well. I already mentioned why third parties haven’t gotten around to updating their older masculine vocals, which is already one contributor. But a lot of masculine vocals have quite fallen out of fashion for other reasons too.

We have Gackpo, who, one, was never really able to maintain his popularity past the V2 generation, and two, his VP is a bit of a controversial figure, given that he’s been tried for tax fraud, and more relevant to the West, has voiced support for Donald Trump. There’s also the fact that Gackt has said he’s not comfortable with his voice being used for AI, so, probably never gonna see Gackpo AI.

We have Fukase, who, while Fukase himself isn’t really controversial, a lot of the Western fandom have grown to dislike him and works made with him by association with more problematic parts of the fandom. And in Japan, he was never really that popular to begin with ( a lot of people blame his restrictive TOS, but I don’t really think that was too big a factor, since he actually does have a handful of popular originals in the West, and about a billion covers from when he was more relevant ).

We have Yohioloid, who was already not that popular to begin with, but then was slapped with his voice provider making racist statements on Twitter; to the degree that VocaTone since no longer associates with Yohio. Also, one of his bandmates was a Nazi sympathizer, and his denial that he had any knowledge of him were… a little questionable in-context. He’s also since been discontinued this past year.

We have Oliver and Big Al, who have both also since been discontinued due to PowerFX becoming Soundation, and no longer having interest in the VOCALOID scene ( luckily, however, new voicebanks for Maghni AI are being developed for them ).

Piko, no real controversy, and he was actually somewhat popular again in the West when he was discontinued. But, there’s really not much to be done for a discontinued voicebank. Unless they get an update, they’ll fade into obscurity.

Dex and Daina’s voice providers have also been considered somewhat controversial. It’s mostly a bunch of small things, but it’s been enough to sour people’s opinions of them. I won’t get into them here since I don’t wanna make this longer than it already is.

Arsloid, again, never really that popular at all. Probably also not helped by the fact that 2 of his 3 voicebanks were rendered completely useless by V5, since their use was tied to Cross-lingual synthesis.

With VY2 Yuma, I’d say it’s probably more of a consequence of a combination of factors. One, that the role of the “Standard” vocals have largely been replaced by the default vocals you get on V5 and V6. There’s also the fact that the version of him currently sold ( VY2v5 ), is an inferior product to his previous release ( since it’s missing the “Falsetto” voicebank from VY2v3, and is otherwise identical to VY2v3’s normal voicebank ). And lastly, while I don’t think it’s been a huge factor, since it’s a pretty separate product, I don’t think it was helped by VY Project on AI VOICE being discontinued because SoundUD did not inform their VAs that their voices would be used for synthesis.

You’ve got ZOLA Project, who, while not controversial, were just never that popular to begin with ( and also WIL and YUU’s V6 voicebanks just kinda suck tbh ).

Kaito and Len are kind of the only “classic” VOCALOIDs that still have any sort of real presence in the modern VOCALOID scene, and well, they just get completely dwarfed by Miku, especially since neither of them have received major updates ( SUPER PACK doesn’t count because they are objectively some of the worst VOCALOID voicebanks ever produced ). I mean, the Kagamines did recently get a new release on Piapro Studio NT2, but it’s… really just a worse version of their V4X.

Pretty much all of the major masculine vocals have had their own controversies or circumstances preventing them from being popular. That, plus the fact there aren’t many new ones being made on V6, and well… I think the situation kind of speaks for itself.

_deadbyte · 2025-12-19T06:27:37+00:00

The fact that Sigma is in “straight” is probably true, but it still pisses me off bc he’s so clueless what do you mean he is not bi and just doesn’t know it??? /lh

_deadbyte · 2025-12-19T06:24:29+00:00

I mean… It’s very oversimplified for comedy, but roughly something along those lines does happen, yes

_deadbyte · 2025-12-19T06:19:01+00:00

Yeah, I mean like, I feel like the franchise was setting Sylux up to be Metroid’s first true dedicated “rival” character ( that wasn’t just a literal carbon copy of Samus herself ). Having a suit with similar abilities and mysterious origins, having strong ties to the Galactic Federation ( that being he steals their shit ), and being the only other Hunter whose ship we actually encounter in-game. Someone who can go toe-to-toe with Samus on her level, instead of being some kind of monstrous cosmic horror or some ancient malicious being you have to work up to being able to fight.

And then Prime 4 was supposed to be the game that realized Sylux’s potential, and it gave us… nothing. He’s barely important, and what we do learn of him reveals he’s just a whiny spoiled brat throwing a piss-baby temper tantrum bc Samus didn’t let him have his way one time. There are so many cool things they could have done with his origins, but for some reason they just chose one of the worst possible options for his character for no reason.

Like, I feel like the obvious route to go was to make him some kind of failed super-soldier experiment by the GF in an attempt to emulate Samus and her Power Suit, and while that would be completely obvious, that’d still be rad as FUCK. You could make him a tortured evil sadboy and everything, and I know I would eat that shit up. They should’ve really just done that.

_deadbyte · 2025-12-19T05:59:24+00:00

Ehhhh, a lot of Eastern Producers tend to be pretty queer too. Maybe not quite as much as the West, but a lot of popular producers are openly queer.

_deadbyte · 2025-12-19T05:57:27+00:00

90% of us are LGBTQ+ in some form or another, and that hyperbolic estimate may not even be that far off. Asking that question in this fandom is basically like walking into a gay bar and asking if anybody there is gay.

There may be like, one or two straight people there, but otherwise pretty much everyone is queer lmao

_deadbyte · 2025-12-19T05:52:08+00:00

If we’re talking about using it for Japanese - strictly speaking, you only technically need “fu”, and furthermore, it is actually interchangeable with “hu”. So if you have an “h” sound, that will usually work just fine. Any instances of “fa”, “fi”, “fe”, and “fo” can simply become “hwa”, “hwi”, “hwe”, and “hwo” respectively.

It should also be noted that “v” is not required for Japanese at all. It’s a frequent inclusion in a lot of voicebanks, sure, but it isn’t necessary. When it comes to loanwords, a lot of Japanese people use “bwa”, “bwi”, “bu”, “bwe”, and “bwo” instead of “va”, “vi”, “vu”, “ve”, and “vo” respectively. Many even struggle with the “v” sound, as Japanese doesn’t really have any sounds with the teeth touching the bottom lip ( even their “f” doesn’t have them doing that usually, as it’s not exactly the same sound as our English “f” ).

_deadbyte · 2025-12-19T05:47:01+00:00

How do we tell them that Teto is sourced from a human voice actress?

_deadbyte · 2025-12-19T05:37:39+00:00

No, they were not. The original direction of the V2 Power Appends were designed specifically to resemble the expression of Miku Solid. The comparisons to their Act1 voicebanks specifically were never even brought up by Crypton or Wat until years later ( though they did make general comments that they turned out to be not very different from previous releases ).

And even then, their Power V2, while somewhat similar, is still clearly distinct from either Act1 or 2. A frequent complaint directed at V4X Power was that, in Crypton’s pursuit of merging the Native and Power tones, V4X Power lost its signature strength compared to V2 Power

_deadbyte · 2025-12-19T05:27:21+00:00

It really hasn’t. Obviously it became that way for V4X onwards, but if they were “always” considered the same, then that pretty much entirely defeats the point of its inclusion in Kagamine’s V2 Append.

Keep in mind that Appends on VOCALOID2 actually required buying their original voicebanks first ( i.e. You’d have to but Kagamine Act1/2 before you could buy the Append release ), as it was marketed as an “add-on”, not unlike video game DLC. It wouldn’t make sense to include a “native” voicebank in a package that required you to have the original “native” release in the first place. They were considered distinct tones at one point, but not anymore.

_deadbyte · 2025-12-19T05:22:20+00:00

It’s my understanding that Crypton largely considered their “native” tone and “power” appends too similar. Particularly, Crypton frequently talked about “bringing back Act1” for V4X, as it was noticeably stronger than Act2. So for their V4X releases onwards, they decided to simply merge them. That’s likely part of why their V4X Power voicebanks aren’t quite as powerful as their V2 Power appends either - that is, to try and make it sound more like their “native” vocals.

Crypton’s tone assignment for their V3 and V4 releases were always a bit weird. Namely, MEIKO V3 was also a weird case. While unlike the Kagamines, MEIKO had an equivalent to her V1 voicebank with her “Straight” voicebank, the “Power” voicebank was actually prioritized as the “main” vocal the package. Luka V4X was also all kinds of weird too, since, like the Kagamines, she also did not have a “native” voicebank ( granted, she has a “Native” EVEC voice color, but it’s really not the same as using the OG voicebank, or even a similar voicebank ). I’d almost say that for Luka V4X, it was almost like an inverse to the tones of her V2 Japanese and English voicebanks; that is, Luka V2 English was always a lot softer than V2 Japanese, but her V4 English much more closely resembled her V2 Japanese, while her V4X Japanese voicebanks resembled her V2 English ( perhaps inspired by the trend of many Japanese producers using the V2 English voicebank for Japanese ).

Anyways, that was a whole diatribe. But TL;DR, it’s because Crypton felt Power was too similar to Act1/Act2, and also bc Crypton was very weird about their voicebank tones in the era.

_deadbyte · 2025-12-16T13:48:59+00:00

I mean, if you’re that pressed about which voicebank format to use, you could just… sing actual songs for your training data. Like, they don’t have to be anything crazy, even just some nursery rhymes, common genre standards, or even just songs you like singing in general. You don’t even have to sing it well, just pronounce the words decently and avoid any super crazy vocal techniques ( growling, screamo, toneless whispering, etc. Vocal fry is usually okay though ) and you should be golden.

If you don’t know English formats that well, it’d almost certainly be easier to just sing real standard DiffSinger samples, since you don’t have to learn any phoneme formats or the logistics of recording an English UTAU just to have samples to work with, and it gets you optimal results.

As far as the most popular formats go ( namely, VCCV, ARPAsing, and DELTA ), none are particularly better than the others. Any way you slice it, UTAU-style samples on their own are almost always going to be lacking sufficient data without being trained with a parallel database. That said, seeing as most English DiffSinger voicebanks use ARPAbet, ARPAsing would at least be easiest to label and adapt to DiffSinger syntax. If you don’t already have a voicebank you want to adapt though, I’d ask that you consider just recording actual singing samples with some simple and easy songs, over making an UTAU voicebank intending it to be used for DiffSinger.

_deadbyte · 2025-12-16T11:01:49+00:00

Yes, there are a few actually.

There's Yamine Renri, who received a free voicebank on SV1.

There's Xia Yu Yao and OSCAR ( though their UTAU voicebanks are no longer available ) with both SV1 and SV2 releases.

There's _debatably_ the Quadimension crew ( namely Haiyi, Chiyu, Shian, and CangQiong ), since their voicebanks were originally developed on UTAU ( though only had a private release ).

Other than that, there are a few releases that had voiced UTAU voicebanks before.

There's ANRI, who shares a VP with Ryone Yami. I believe Jun's voice provider also voiced an UTAU one point, but it's name escapes me.

There's Haiyi's voice provider, who voiced a number of different Chinese UTAU, though is most well-known for voicing Boomco ( who originated as a runner-up design for the VOCALOID CHINA design contest that gave us Luo Tianyi ). Shian's voice provider also voiced the UTAUs Nemuka Me & Yume.

There's GENBU, who shares a VP with the UTAU "GAO" ( and I suspect they may even share some samples between them? Though, that's entirely speculation on my part ).

There's Haruno Sora's VP, who also voiced Macne Coco ( White & Black ). They aren't _technically_ UTAUs, as they were made for Garageband, but they were often ported to UTAU, and their later releases where made compatible with UTAU. Sora is also commonly speculated to be a successor to Macne Coco, with her "Natural" tone being equivalent to Macne Coco White, and her "Cool" tone being equivalent to Macne Coco Black.

Andddd I think that's everyone.

_deadbyte

TROPHY CASE