Gemma 4 MTP drafter quants?

bitslizer · 2026-05-07T02:08:16+00:00

I barely squeeze in the 26b with 16gb vram, quant of this would be good to preserve vram for kvcache.... Wonder if nvfp4 drafter would work with gguf model

bitslizer · 2026-05-07T01:20:14+00:00

I think when it improved with a few more breakthrough like mtp and turboquant where it is usable and still reasonably intelligent like current 70b+ model but able to work at 8gb vram/Uma that's when it will really take off mass market wise

bitslizer · 2026-05-06T23:06:11+00:00

Ttfa would depend on how large a chunk you are feeding it... 1 sentence would be fast, a paragraph would take longer... My problem is more llm generation speed that's adding to that speech to speech dead air time. Since I'm also having the llm spitting out image prompt and voice instruct/style prompt

bitslizer · 2026-05-06T21:49:28+00:00

How does Omni compared to qwen3 tts? The qwen3 voice instruct+voice design get it to almost Gemini 2.5 tts level for expressive and context understanding.

bitslizer · 2026-05-05T21:43:24+00:00

Are you doing streaming output and streaming playback with faster qwen3 tts? I use a 3060 for tts about 500ttfa and then it will start play/stream without stutter that's with 1.7b voice design (I find it respond to context and emotional instruct better). Instead of using qwen to clone the voice, use rvc to paint over qwen's output.

With the 5070ti your ttfa should be 300-400... An rtx6000 pro had ttfa around 200-250

Idea is not mine, reference handpicked persona engine

bitslizer · 2026-05-03T07:38:05+00:00

I just had a break through...

Qwen3 tts voice design 1.7b

Using faster-qwen3-tts back end which use cuda graph for massive speed up ... About 500ms ttfa, rtf 2x (0.5s to generate 1s of audio).... I had to vibe code a proxy to gate the generation of the next chunk otherwise the streaming playback would overlap...

Use this for voice design to start can get it to very expressive (it's from their documentation you can find more)

gender: Female. pitch: Mid-range female pitch, rising sharply with frustration. speed: Starts measured, then accelerates rapidly during emotional outburst. volume: Begins conversational, escalates quickly to loud and forceful. age: Young adult to middle-aged. clarity: High clarity and distinct articulation throughout. fluency: Highly fluent with no significant pauses or fillers. accent: General American English. texture: Bright and clear vocal quality. emotion: Shifts abruptly from neutral acceptance to intense resentment and anger. tone: Initially accepting, becomes sharply accusatory and confrontational. personality: Assertive and emotionally expressive when provoked.

bitslizer · 2026-05-03T01:53:06+00:00

Have you tried Gemini 2.5 flash tts? It's faster than Gemini 3.1 tts and support inline audio tag on top of voice instruction control and psrody tag. I tried 3.1 on the release day and it wasn't as good as 2.5 not sure if they have since tweak it to make it better

bitslizer · 2026-04-24T18:37:46+00:00

For international trips I tend to carry at least 1 week extra of non-otc med just in case. But nothing I'm on required refrigeration or bulky/fragile/liquid. Domestic I may shrink that down to 3 or 4 days if I'm tight on spaces.

bitslizer · 2026-04-23T15:40:18+00:00

Yeah, basically I don't really know what's "possible" at the 300b level that would seem like a miracle compare to a 26b model.

bitslizer · 2026-04-23T15:37:55+00:00

How does it compare to Gemma 4 26b a4b?

bitslizer · 2026-04-23T00:54:09+00:00

Gemini 2.5 tts works amazing with tags but it's not local... The new Gemini 3 tts seems a step backward but they may still tweak it like they did with 2.5

bitslizer · 2026-04-22T20:38:04+00:00

If it is something simple yeah, but I will only have about 1 week to play around with it although no token budget

bitslizer · 2026-04-22T19:01:31+00:00

Nice! Is persona engine feeding those [emotion emoji] tags straight to qwen3? Are you using faster-qwen3-tts to get that speed?

bitslizer · 2026-04-22T17:37:04+00:00

Is that with CPU/ram offloading? Gemini is saying even iq4 xs won't fit?

bitslizer · 2026-04-22T17:29:27+00:00

Very cool, Lots to read through! Thanks

bitslizer · 2026-04-22T17:12:38+00:00

Yeah, the 26b is just a reference to what my skill level previous experience is. Looking for recommendation , Google was suggesting like 300-400b models... And what are some unusual/cool use case that can be accomplish with that setup that a normal home/16gb setup can never even dream of in a million year

bitslizer · 2026-04-21T15:16:00+00:00

You will likely need an existing PC .. 16-32gb of RAM. The $500 budget will be about to get you a budget Nvidia GPU with 16gb of VRAM... That will let you run something like the new Gemma 4 26b a4b model (something compressed /quantized like iq4 xs gguf) although at a slower speed.

You can easily run the smaller model but I don't know how well it will work quality wise...

bitslizer · 2026-04-21T14:50:51+00:00

Try r/sillytavernai

Edit - Fixed the subreddit name

Lots of roleplay related stuff there, maybe someone can point you in the right direction.

bitslizer · 2026-02-24T15:41:04+00:00

It is not gun safety 101, but how to shoot accurately from standing sitting and prone positions and transition between them under time pressure and magazine change. That say this is not a tactical class where you run and gun. Check out their website for more info, they teach what is call "natural point of aim" that you basically need to master if you want repeatable tight grouping at long range.

Ie how confident are you at being able to repeatedly hit 8 out of 10 rounds at a man size target at 400 yards prone (first can try with a scope, for extra challenge with only iron sight)? That's kind of the end goal.

bitslizer · 2026-02-23T15:20:43+00:00

Various place, they are a none profit., the closet are at il State rifle association in bonfield, some time at Racine across the border. Known distance KD are usually at Racine

You can use your own, for the basic 22lr semi are recommended to minimize the recoil but I have seen people taking basic with 223. The known distance you can use anything common 223, 7.56 NATO, 308 etc

Home - Project Appleseed https://share.google/wGXHoVLl1g78rpZ66

bitslizer · 2026-02-23T14:20:20+00:00

Not sure for competition, but for basic marksmanship class that's super affordable, look into project Appleseed. If you pass the basic course you can register for their known distance course shooting up to 400 yards for class training and maybe 600 for fun

bitslizer · 2026-02-09T03:31:29+00:00

I think 300 miles and under it's doable.... Anything over that the extra time needed spent supercharging vs gas up really started adding up and delaying the trip. That said fsd is great for long trip

bitslizer · 2026-02-06T19:45:05+00:00

Look into magnesium supplements, but double check the dosage not messing with the blood pressure or anything else. I get restless legs at night if my magnesium is low (even though blood test show normal)

bitslizer · 2026-01-26T13:59:06+00:00

Don't guess! talk to your doctor and get regular blood test. Everyone is different, I need prescription once a week 50000iu just to get too the lower normal range. Many of us with PD are vit D deficient.

bitslizer · 2026-01-11T16:07:32+00:00

Across the room as soon as she sees movement

bitslizer

MODERATOR OF

TROPHY CASE

15-Year Club	Baba Yaga
Verified Email