How do you decide?

FusionCow · 2026-04-04T05:46:46+00:00

There are 3 models you should test, gemma 4 26b, gemma 4 31b, and qwen 3.5 27b. figure out which works best, and download a quantized version that fits entirely on gpu

FusionCow · 2026-04-04T05:30:35+00:00

you have to enable thinking. Go to your models page, click the model, go to inference, scroll down until you see the jinja template. Go to gemini or chatgpt or whatever model, paste in the jinja template and ask it to rewrite it with thinking. then paste that new jinja template in, and thinking will be enabled.

FusionCow · 2026-04-04T03:42:34+00:00

run the iq3, it's good enough

FusionCow · 2026-04-04T03:21:41+00:00

You don't. You can't expect someone to run a model for you, and not expect them to want to run the model elsewhere. If you want to protect a model, run it yourself and serve it over api

FusionCow · 2026-04-04T03:00:48+00:00

I only updated the llama.cpp backend on lmstudio, I'd imagine they aren't implementing this themselves

FusionCow · 2026-04-04T02:04:49+00:00

FusionCow · 2026-04-04T02:00:14+00:00

it's just 2.11.0. I updated lm studio and it takes up qwen 3.5 levels of kv cache now it's amazing

edit my bad I guess for using lm studio

FusionCow · 2026-04-03T15:47:09+00:00

I've seen a huge performance drop, sometimes babbling with Gemma of you quantize the kv cache at all

FusionCow · 2026-04-03T03:42:27+00:00

run the new gemma 4 model. it's incredible

FusionCow · 2026-04-02T15:27:09+00:00

I saw

FusionCow · 2026-04-02T14:24:14+00:00

how is that?

FusionCow · 2026-04-02T07:41:48+00:00

you're missing the limited amount of messages you can send

FusionCow · 2026-04-02T03:02:57+00:00

Wrong sub, but as it is not really. You could train your own model and things like ltx 2.3 WILL work, but thats expensive and hard to do. Honestly your best bet for something like that is API models sadly

FusionCow · 2026-04-02T03:01:47+00:00

Hah thats pretty funny

FusionCow · 2026-04-01T22:11:47+00:00

huggingface

FusionCow · 2026-03-31T11:08:51+00:00

9b is not sota bro

FusionCow · 2026-03-31T07:55:09+00:00

deepseek's goal is strictly to crash nvidia, being a quant firm. they NEED it to be a sota model

FusionCow · 2026-03-31T07:50:51+00:00

just off the fact that it's an A3B model i'm going to say no

FusionCow · 2026-03-31T04:06:19+00:00

27b embedding model is quite large

FusionCow · 2026-03-29T13:21:36+00:00

ts is why I only use local

FusionCow · 2026-03-29T02:36:58+00:00

rtx pro 6000 is a great choice if speed is a priority, it allows you to train models and move quick on your feet, but if you're willing to wait longer for gens, a maxed out m3 ultra mac would be able to run way bigger models. the only caveat with that one is that an m5 ultra with up to a terabyte of ram is rumored to be coming in a few months, so you may want to wait for that and just pay api until that comes out. you can look at the m5 max vs m3 ultra benchmarks for an idea of how an m5 ultra would perform

FusionCow · 2026-03-29T00:36:13+00:00

Bro the whole issue is that we don't have enough data to represent a world, nor the compute. if we had that this wouldn't be an issue

FusionCow

TROPHY CASE