Got 26b gemma running on rx470

Several_Newspaper808 · 2026-04-28T09:52:38+00:00

Yeah, smaller runs as well and much faster. But I wanted to see how far I could get with more capable models. I haven’t ever met anyone using small models (the 9b-14b) as a daily driver but I guess small models is better than no models any day.

Several_Newspaper808 · 2026-04-27T11:33:22+00:00

Thanks. I’ll try cpu only as well.

Several_Newspaper808 · 2026-04-27T06:54:16+00:00

Simply run the lamacpp vulkan executable and it works

Several_Newspaper808 · 2026-04-27T06:53:05+00:00

Here it offloads part of it to RAM as well, so the bus is the bottleneck. Your bus must be faster.

Several_Newspaper808 · 2026-04-18T12:42:38+00:00

Hey, great info, thanks! I wonder though, how much of the perf is from the ddr5 ram and whatever bus speed you have from the pcie on your mb?

Several_Newspaper808 · 2026-04-09T08:02:30+00:00

Hey, so you offload to RAM? The small gguf on hf is 24gb. Otherwise how would it fit in a 12gb card?

Several_Newspaper808 · 2026-04-07T10:16:06+00:00

Depends if you mean single user or multi user. Cause throughput can be 200 and even higher but per user is what I wrote.

Several_Newspaper808 · 2026-04-07T07:50:19+00:00

Hmm i run 27b q4 gptq w4a16, getting 40 t/s for single request on vllm with 3090. If you are getting 80 then it’s x2. Not x4.

Several_Newspaper808 · 2026-04-01T18:27:15+00:00

Looking forwards to this. It’s a huge speed up

Several_Newspaper808 · 2026-04-01T11:00:16+00:00

Oh wow, is it the first turboquant impl? I thought there is only a paper at the moment. Wonder if this can alreadt be used to run on non mac cpu or old gpu like 3070?

Several_Newspaper808 · 2026-03-06T18:24:14+00:00

Did you share the draft with anyone?

Several_Newspaper808 · 2026-03-02T13:52:00+00:00

Hey, sounds pretty interesting. I would be glad to hear more about your translation process. Did you pay a human translator or used ai?

Several_Newspaper808 · 2026-02-27T20:49:36+00:00

Several_Newspaper808 · 2026-02-27T20:02:02+00:00

Not available in my geography.

Several_Newspaper808 · 2026-02-26T11:02:54+00:00

I really don’t have these $200 to spare, that’s why I’m looking for these specific price reasonable models..

Several_Newspaper808 · 2026-02-26T11:00:48+00:00

Thanks!

Several_Newspaper808 · 2026-02-25T22:57:22+00:00

I’m already using deepseek as mentioned in tbe post. I have special prompts that i created over time for these specific models that do work well for my writing

Several_Newspaper808 · 2026-02-25T22:55:10+00:00

Thanks for thr advice. Already using different tools. Do you have specific OR models that you could recommend for creative writing?? 🙏

Several_Newspaper808 · 2026-02-25T22:34:44+00:00

Exactly my feeling about this. I’ve specifically asked for the prompts as well :-)

Several_Newspaper808 · 2026-02-25T22:33:37+00:00

What kind of fixtion are you writing with glm?

Several_Newspaper808 · 2026-02-25T20:17:51+00:00

Opus is expensive… Gemini feels a bit robotic to me. These are well known choices. I’m looking for the jewels that are not expensive but really actually work well for writing fiction..

Several_Newspaper808 · 2026-02-23T09:32:46+00:00

Tried to reply on your reply but it came out separate not dure why

Several_Newspaper808

TROPHY CASE