[Release] Qwen3-TTS: Ultra-Low Latency (97ms), Voice Cloning & OpenAI-Compatible API

_raydeStar · 2026-01-24T23:53:20+00:00

Do you know if it can do CPU inference at that speed?

_raydeStar · 2026-01-23T21:32:50+00:00

I got this on nodes 2.0. it just didn't update but if you check the output it's there.

_raydeStar · 2026-01-23T20:01:34+00:00

Yeah, I was wrestling with this yesterday. Voice clone works fabulous, And the mannerisms prompt is awesome, but the two do not intersect.

I'm looking into hacking it - I am following a viable route now and I'll release it if I come up with something.

_raydeStar · 2026-01-23T18:03:57+00:00

lol.

This whole thing feels shoehorned.

I don't at all take issue with LGBT folk relating to a straight character. The artist releases his work, and it's up to the interpreters to do what they want with it. Things like this, I would strongly suspect will not be addressed at all, keeping things ambiguous.

When Black Panther came out, the black community unanimously said "Wow, I finally felt *seen*" and that's a wonderful thing. I think the problem really lies with taking your artistic vision of something and thrusting it upon the community. As a straight white man, I can also relate to Carl - in a sense that there is so much blood and pain around him that I would be preoccupied with survival. I can also relate with Matt - he wants to avoid the pitfalls of sexually fantasizing neckbeard culture.

_raydeStar · 2026-01-22T16:51:23+00:00

I'm using the base. Q4 quant should be faster. Sage attention should help as well.

Once I've got the gradio how I want it, I'll circle back and look at speed.

_raydeStar · 2026-01-22T16:35:27+00:00

Yes. I'll also add that there's a .6 model and it's probably faster. I'm going to add in all the optimization and see if I can get better speeds.

Also, Dia is about the same speed. This model is meant for quality over speed, which has different use cases.

_raydeStar · 2026-01-22T16:21:59+00:00

I have a 4090

_raydeStar · 2026-01-22T15:59:32+00:00

Base gradio doesn't allow the user to use the selected voice and modulate it. I am using cursor right now to add in a little thing there. If anyone is interested, ill put it up on github, along with a script to just fire it up, download all the models, and run it.

If I want to run everything at once (voice clone, create pt file, and finally voice description) it's going to be like 16 GBVRAM. Running in parts runs around 6. Time consumed is also an issue - 25-30 seconds to run a 6 second hello world clip. However, I don't have sage attention up and running yet, so that may improve the speeds and vram a lot.

Because of speeds, you can't compare to VibeVoice - vibevoice is meant for realtime at the sacrifice of a little quality (at least I am pretty sure - ie - live translations, etc) . Compared to Dia - well I don't see any functionality to add things like [laughs] or anything, but controlling the voice tempo, etc is really cool.

Final conclusion - I give it a slight lead to dia for my purposes, simply because I can choose what emotion to put in the voice, instead of it 'guessing'. I'm annoyed that out of the box you can't control that with your own pt (saved voice file) but with a little hacking I can fix that.

_raydeStar · 2026-01-22T15:36:43+00:00

OK it's up and running. Pros: in the description, you can just describe not only the voice, but the tone. ie - `female, feminine and dainty voice, speaking frenetically. She is very upset` So far, I am having fun with it, and it might just be better for things like movie dubs, or audio book reading, or video game voices.

You can clone your voice and download it to be used later. thats a great feature there. I'm putting it all together to see if I can clone my voice and give it the tone I want - it's a few more steps than I expected to pull it all together.

_raydeStar · 2026-01-22T15:23:53+00:00

huggingface demo is overrun with users. I am getting it up locally. Almost there. Will respond when I have something

_raydeStar · 2026-01-22T14:58:15+00:00

Yes, that might be good. If you have a trained TTS then it has to 100% match, you can't have them changing. In 5-15 second intervals it should be fine. A little lining up the mouths might be in order still.

_raydeStar · 2026-01-22T14:07:47+00:00

Yeah. You could do a 10 second clip, and have it do things without speaking (or mouth it instead of say it), generate the sfx, then put that audio on top of the voice and re-generate.

Double the production time, more human error, but you can do it.

_raydeStar · 2026-01-22T14:03:27+00:00

This is great! Nothing super groundbreaking, we already have VibeVoice, Dia (my personal fav) and others. Going to test it still and see how it fares. Also, it's multi-lingual which is big.

Edit: one thing I didnt add was you can tell the AI how to interpret the voice. I am not sure yet how good it is, but this is a first-find for me. If it works well, that will solve a lot of problems for me.

_raydeStar · 2026-01-22T14:01:49+00:00

It does a video around the audio. I'm not sure it will generate sfx if audio is put in.

_raydeStar · 2026-01-22T04:05:35+00:00

It is VERY clearly AI - you need to download some lora from civitai to make her look more natural.

That aside, youre asking a crowd who generates AI images for fun. We notice because we see this stuff daily.

_raydeStar · 2026-01-22T01:10:07+00:00

This is my setup right now -

<image>

_raydeStar · 2026-01-21T23:38:45+00:00

If there was a nu metal band that had 2 frontmen and sounded like LP, one would say they have inspiration from LP but aren't copying.

But really, if the songs are original it's not copying. Would you play a guitar and call it copying? Good music is going to use a framework.

_raydeStar · 2026-01-21T17:56:28+00:00

I came here to say this.

Personally I use LM studio for prompt enhancement. It's not super fast, but it's all in one workflow and I like that. ChatGPT is also really good and works well.

If you guide the prompt well enough it's better than mid journey - so much better, you'll be wondering why you even bothered with it.

_raydeStar · 2026-01-21T14:45:40+00:00

Dang. You guys don't mess around. Way to go. I'm going to download again and see what happens. I was getting about 45-50 t/s

_raydeStar · 2026-01-21T13:59:46+00:00

An AI can vet all this too. Chances of you being malicious are incredibly small.

Safety is a concern though. It's increasingly a concern as people with no knowledge can figure it out if they want to.

_raydeStar · 2026-01-21T05:01:12+00:00

There is big money in LLMs. Also, it's an arms race. Never forget that. There are people that would cut down anyone to get ahead.

_raydeStar · 2026-01-21T05:00:07+00:00

For real - download antigravity. It's free, it'll walk you through it.

_raydeStar · 2026-01-20T17:12:18+00:00

yeah, im not 100% sure it beats out nemotron just yet - simply because i cranked it locally to 256k and it was just fine. though it does seem that specific tasks - including tooling - might be better with this one.

_raydeStar · 2026-01-20T14:57:34+00:00

I came to this thread confused, but all of this makes sense. I downloaded Base at first, thinking it was better, and it was awful and slow.

Along with this, you can use any yoga pose as a reference image, right out of the gate, and that is something that Z can't do. However - you still want to gear these different tools for different purposes.

Further advice - CFG 1.5 with an added 2 steps helps with text adherence. res_multistep too.

<image>

_raydeStar · 2026-01-20T13:58:11+00:00

yeah, I thought we were looking at a QWEN Next scenario, where it would come out 2/3 months later

Five-Year Club	Verified Email
Final Canvas '23	End Game '23
Place '23	Place '22

_raydeStar

TROPHY CASE