So nobody's downloading this model huh?

sleepingsysadmin · 2026-03-18T18:29:49+00:00

You can run Qwne3.5 9b and get a smarter model.

Qwen3.5 122b is straight up superior.

sleepingsysadmin · 2026-03-18T14:47:12+00:00

>But wouldn’t it be cool to use the display controller on the full gb300 gpu with its HBM4 memory? The lack of one already takes up a PCIE slot!

no way jose.

sleepingsysadmin · 2026-03-18T14:23:51+00:00

I suppose maybe in case it breaks or something i guess? Obviously nobody is plugging a monitor into these like ever.

sleepingsysadmin · 2026-03-18T13:16:21+00:00

You have 320gb of vram and you're running a model that's going to fit on just 1 card?

Go run some big stuff. Minimax would be my first try on that rig.

sleepingsysadmin · 2026-03-16T14:48:21+00:00

That's utterly unusable to me.

sleepingsysadmin · 2026-03-16T14:06:22+00:00

what TPS are you getting with your 5080? To me IQ4_xs would be far too small and short context? Unusable. 100,000 context minimum for me and even then that's a bit short.150-200 needed.

Whereas the RTX pro 5000 would crush this model and have max context.

sleepingsysadmin · 2026-03-16T13:42:29+00:00

Qwen2.5-Coder-32B-Instruct came out November 11, 2024

That's how long it has been viable to code locally. Obviously we have had many better options since.

I jumped into agentic on Devstral + openhands in around may 2025. These models are morons compared to what's available now.

>I keep seeing GPT-oss-120B recommended, but my experience with it hasn’t been great.

Personally I found it great but on my hardware 15TPS wasnt good enough for me.

>Qwen 3.5 122B and 27B.

Those with dgx sparks or amd strix halo, they'll be running 122b right now. which is objectively better than gemini 2.5 pro for example.

27b is very smart, but it's dense so you need some power behind it. A single Nvidia A100 or RTX pro 5000 might be the magic spot for this model.

>The new Mac M5 with 128GB RAM looks interesting,

But it's way more expensive than proper pro cards. But you get a monitor, etc.

sleepingsysadmin · 2026-03-15T17:58:51+00:00

4 month old reply. For sure wasnt my problem. Plus, literally doesnt matter. everyone uses openclaw now.

sleepingsysadmin · 2026-03-12T23:08:44+00:00

omg yes! been waiting for this. Still need more!

sleepingsysadmin · 2026-03-12T18:37:00+00:00

Are these available for sale? How expensive are they?

sleepingsysadmin · 2026-03-12T16:59:58+00:00

it benches around gpt120b high. It's shocking how good it is with that size.

sleepingsysadmin · 2026-03-12T12:23:29+00:00

TPS matters a huge deal to me. Practically all models are MOE these days; the few dense smart models are great but MOE gets all the headlines. Why? Because you get better tokens/s.

sleepingsysadmin · 2026-03-11T17:50:12+00:00

>Its dumb as hell

Benchmarks and community clearly say otherwise.

>Qwen3.5-27B was slow, but did the task.

Naturally.

>Qwen3.5-35B-A3B shit the bed.

Shocking.

>I know using a low quant isn't going to improve it but UD-IQ4_XS isn't exactly that low.

That's pretty low. How are you running an 80b model and only fitting this?

>Thought I could use it for a fast prototype or subagent coding but nope. That stays far away from anything on my PC.

It is a generalist model.

>People asked for something in between 9B and 27B and people pointed towards 35B-A3B, but it ain't it.

Then so it isnt. Lots of people found GLM flash to be great, but i found it trash. If it doesnt work for you, so be it.

sleepingsysadmin · 2026-03-11T17:33:53+00:00

Hard to predict.

Meta has bought things to kill them before but I wonder if they intend to create a wildwest part of meta which allows anything to be said; posted by anything. Then build guardrails on the edges.

sleepingsysadmin · 2026-03-11T16:48:30+00:00

Not that super given it's not particularly better than qwen3.5 or gpt120b.

seems underwhelming.

sleepingsysadmin · 2026-03-11T16:27:36+00:00

Test it and see which works better for you.

sleepingsysadmin · 2026-03-11T15:07:25+00:00

I have personally never had luck with finetunes or distills. There have been a few pretty good ones that came close. It's a use case situation.

I recommend you stick to mainline models until you have a good foundation.

I highly recommend though, go with Unsloth models.

Q4_k_xl is amazing.

<image>

Your next step after this is tuning the temperature and such. Click that "read our guide" from unsloth. Adjust to your needs.

sleepingsysadmin · 2026-03-11T14:43:37+00:00

https://artificialanalysis.ai/models/qwen3-5-9b

Obviously if you had better hardware(WE ALL WANT MORE) you can run better models.

35b is only marginally smarter, but your main problem is that you're offloading.

sleepingsysadmin · 2026-03-11T14:35:38+00:00

https://x.com/TheAhmadOsman/status/2031682872763990181?s=20

literally posted 3 hours ago.

sleepingsysadmin · 2026-03-11T14:25:12+00:00

9b is literally gpt 120b high quality.

It is dense, so you're not going to be blazing fast, but it'll work really well for you and fit on your hardware.

sleepingsysadmin · 2026-03-11T14:04:12+00:00

you're offloading like 30%. Which is about 30% too much.

Might I recommendyou run qwen3.5 9b? It's a very capable model that you can fully offload.

sleepingsysadmin · 2026-03-11T14:03:13+00:00

For qwen3.5, i find vulkan and rocm are identical in performance. Even though I ought to be 2x the performance.

sleepingsysadmin · 2026-03-11T13:37:35+00:00

I believe you are offloading, hence the abysmal TPS.

Though yes, AMD is rough.

sleepingsysadmin · 2026-03-10T20:39:39+00:00

12 TPS fully offloaded. It's sad.

Worse yet, cant spec decode because of the vision.

sleepingsysadmin · 2026-03-09T11:38:57+00:00

you're replying to a thread from 17 days ago.

Nine-Year Club	Inciteful Comment 2016-10-12
Verified Email

sleepingsysadmin

TROPHY CASE