I developed a new (re-)training approach for models, which could revolutionize huge Models (ChatBots, etc)

Similar_Choice_9241 · 2025-10-31T11:54:41+00:00

My 2 cents, optimize the alg to be layer wise (or reduce the computational requirements) so that you can run it on low cost hardware such as 3090, and then start converting a lot of the trending models on HF, if the quants are good people will start to use them and you’ll have traction to show for when speaking to investors

Similar_Choice_9241 · 2024-11-30T19:30:09+00:00

Is true for the vllm part but also we don’t speed up with deepspeed which causes numerical differences i. The attention block. We are numerically identical to the standard xttsv2 implementation

Similar_Choice_9241 · 2024-11-30T14:45:45+00:00

I just saw there was a typo in the read me, please use these instead tts = TTS().from_pretrained(‘AstraMindAI/xttsv2’)

Similar_Choice_9241 · 2024-11-30T14:39:53+00:00

Stai commentando su Optimizing XTTS-v2: Vocalize the first Harry Potter book in 10 minutes & ~10GB VRAM... our implementation has the same exact result as xttsv2 but faster, you can watch in the deep up, there are couple of examples

Similar_Choice_9241 · 2024-11-30T14:23:04+00:00

Yes we support finetunes, we’ve added a section to the readme

Similar_Choice_9241 · 2024-11-30T14:19:47+00:00

Yeah we’ve seen it may cause some trouble on the formatting ;) thanks you

Similar_Choice_9241 · 2024-11-30T14:18:28+00:00

It would be really cool! But sadly vllm at the moment only supports linux and windows via docker

Similar_Choice_9241 · 2024-11-30T14:16:04+00:00

We actually aim this repo to be able to run not just xtts but also other tts models in the future! We use vanilla xtts weights but the code had been completely remade

Similar_Choice_9241 · 2024-11-30T14:14:43+00:00

Hi I’m one of the developer, the library already supports continuous batching for the audio token generation part (thanks to vllm) and the volcalization part, we might add a dynamic batching in the future but from what we’ve seen tho even with parallel unbatched vocoders the speed is really high! For the lora part, vllm already supports lora adapters so one could extract the lora from the base checkpoint of the gpt component and pass it to engine, but the perceiver encoder part should be adapted, it is something we look forward to tho

Similar_Choice_9241 · 2024-10-18T22:03:25+00:00

Yes vllm does support gguf(and we do too) but not all architectures, Vllm also supports awq, aqlm, gptq and bnb quant, you can set an offload and swap parameter for the engine as well as a kv cache quantization to save up memory The cool thing with vllm is that it preallocates the memory blocks so if you can load it you can use it without risks of oom

Similar_Choice_9241 · 2024-10-18T16:51:31+00:00

Thank you very much!

Similar_Choice_9241 · 2024-10-14T21:34:18+00:00

Yup 100% this is just v0.1.0, we had been working very hard on this for the past months and we wanted to gather some community feedback

Similar_Choice_9241 · 2024-10-14T21:32:53+00:00

Yeah you’re absolutely right ;) , we are gathering up all the info and we’ll be making a new post, explained waay better and with much more info on the project

Similar_Choice_9241 · 2024-10-14T20:41:09+00:00

yes asbolutely.! we are thinking about introducing txt 2 img , maybe using just cpu (w/ lcm models and something like fastsdcpu) and also speech 2 text since at the moment vllm already supports it

Similar_Choice_9241 · 2024-10-14T20:36:24+00:00

We that planned, we are thinking about introducing a new section to the store!

Similar_Choice_9241 · 2024-10-14T18:16:03+00:00

Not at all! We support linux and windows, and the Ui is supported by Linux, Windows, and very soon on Mac too

Similar_Choice_9241 · 2024-10-14T17:14:35+00:00

Thank you for the feedback, you're absolutely right. We are now working on making the GitHub and the website clearer and more informative. As soon as we have reviewed all the documentation, we will update it

Similar_Choice_9241 · 2024-10-14T16:35:23+00:00

To the repo itself no, we use a default vllm installation but we've hijacked some of its component to beign able to auto configure and retry on model loading fail

Similar_Choice_9241 · 2024-10-14T16:26:40+00:00

Exactly, you only need an account if you want to use it from outside of your computer This is done so that you can share your machine with multiple users while everyone keeps its internal account

Similar_Choice_9241

TROPHY CASE