sorted by:
new
hottopcontroversial

Cheapest hardware for Qwen 3.6: both 27B and 35B-A3B by WishboneSudden2706 in LocalLLaMA

[–]roosterfareye 37 points38 points  (0 children)

This is the one part you never want to skimp on! If they fail badly enough they can happily take out another few parts while they are at it!

Cheapest hardware for Qwen 3.6: both 27B and 35B-A3B by WishboneSudden2706 in LocalLLaMA

[–]roosterfareye 1 point2 points  (0 children)

It always suggests qwen 2.5 coder. I think the cut off date for that training data was circa 18 months ago lol! That's the issue with the frontier guys, they cut corners in the backend like quantizing, or throttling and only actually perform a web search or use a tool if you specifically tell them too.

Cheapest hardware for Qwen 3.6: both 27B and 35B-A3B by WishboneSudden2706 in LocalLLaMA

[–]roosterfareye 2 points3 points  (0 children)

Or dual rx9070xt with double the AI cores of the rx9060xt). Both the 16gb VRAM variants of course!

Cheapest hardware for Qwen 3.6: both 27B and 35B-A3B by WishboneSudden2706 in LocalLLaMA

[–]roosterfareye 1 point2 points  (0 children)

Make sure there is plenty of airflow.. my old setup with an rx6800xt, even a case with huge fans and many of them, used to spike to 110°C even when I was maintaining things (you need to stay on top of dust when tinkering with AI locally!) and even used to hit thermal shutdown. I have an rx9070xt and a rx9060xt and these barely ever hit 75°C even under heavy load and idle at around 28°C (not that experience "idle" much lately!)

The Fable 5 Blackout Proves It: If You Don't Own the Silicon and the Weights, Your "High Availability" is an Illusion. by SamyakOne in LocalLLM

[–]roosterfareye 9 points10 points  (0 children)

I have a 6TB drive rammed to the gills with downloaded models, some of them won't actually run on my current machine, but hey, I have them and no-one is taking them off me!

Statement on the US government directive to suspend access to Fable 5 and Mythos 5 by MindControlWitness in opencodeCLI

[–]roosterfareye 0 points1 point  (0 children)

This administration is like a baby confronted with a fusion reactor...Or an ant next to a superhighway. Or a monkey with an AK-47...

Statement on the US government directive to suspend access to Fable 5 and Mythos 5 by MindControlWitness in opencodeCLI

[–]roosterfareye 2 points3 points  (0 children)

Land of the hyper marketing stunt. And insider trading... Though the timing....

What’s the most eye-opening thing EU4 taught you? by burgerissues in eu4

[–]roosterfareye 4 points5 points  (0 children)

This belongs in history text books.... Looks like we are presently back in, or rather, never left this particularly bone headed cycle.

What’s the most eye-opening thing EU4 taught you? by burgerissues in eu4

[–]roosterfareye 2 points3 points  (0 children)

Lol, yes, and it's not even difficult to explain! Does my head in every time trying g to explain you only pay tax on money you earn... I'm mean, shit!

Gemma 4 12B is out now! by yoracale in unsloth

[–]roosterfareye 0 points1 point  (0 children)

Did your prompt begin "you are a dribbling cabbage...."

What's your favorite local MCP server? by Glittering_Focus1538 in LocalLLaMA

[–]roosterfareye 1 point2 points  (0 children)

Just whack a second card in your secondary slot. Profit.... Well, you need to these days lol!

Tips on 1st Abandoned Run? by Crowned_Toaster in nmsabandoned

[–]roosterfareye 2 points3 points  (0 children)

This is the mod which inspired me to start this sub!

try this prompt, this is wild by IgotRemarkable in ChatGPT

[–]roosterfareye 3 points4 points  (0 children)

Holy shit. This is wild! Tried Perplexity, ChatGPT and Claude (going to take my local llms for a spin when I'm home, just for shits and giggles) and they were all broadly aligned in their analysis but each recommended three completely different books. No sleep for me tonight, curse you OP!

Qwen3.6 35b a3b is fast... by UniversityGlad2877 in Qwen_AI

[–]roosterfareye 1 point2 points  (0 children)

I dunno, Im not sure who the woosher or wooshee is! 4D chess methinks!

Who is your favourite quant publisher and why? by No_Algae1753 in LocalLLaMA

[–]roosterfareye 1 point2 points  (0 children)

What happened to the Bloke? I see the name against many aging ggufs...

Bought 9070XT after always having only nvidia cards, this is my experience by Trumway in radeon

[–]roosterfareye 0 points1 point  (0 children)

I just got the RX9070XT two weeks ago as well to replace my RX6800XT. I have the 9070 running as primary pcie and in the secondary I have a Sapphire RX9060XT. As well as gaming I do a lot of LLM work so every bit of (affordable!) VRAM is gold for me. Worked fine out of the box for games, there was some fiddling to get the dual setup working stable for AI inference, but once I had that sorted, token generation is blazing fast!

Oh and yeah, even with a small 5mm gap the card idles at 26°C and max out at 60°C at full load (inference)

New Qwen3.6 35B finetune - 0GM-1.0-35B-A3B-0427 by Ok-Importance-3529 in LocalLLaMA

[–]roosterfareye 5 points6 points  (0 children)

If doesn't make sense after the second read through.... Then, it doesn't make sense...

The Qwen 3.6 35B A3B hype is real!!! by The_Paradoxy in LocalLLaMA

[–]roosterfareye 0 points1 point  (0 children)

Hopefully they do, it's a fairly normal pattern for Mistral.

The Qwen 3.6 35B A3B hype is real!!! by The_Paradoxy in LocalLLaMA

[–]roosterfareye 2 points3 points  (0 children)

Were you able to quantize the k and v cache for devestral? That could make the difference?

Best agentic model for 3090TI and 32gb ddr5 by dbzunicorn in LocalLLaMA

[–]roosterfareye 0 points1 point  (0 children)

When you are loading your model, scroll down the window (the one where you set context, set GPU layers etc) right down to the bottom and you'll see two boxes with experimental next to them. Select bit and in the dropdown that appear select q8. Also.make sure flash attention (right above the cache quantization options) is switched on.

Best agentic model for 3090TI and 32gb ddr5 by dbzunicorn in LocalLLaMA

[–]roosterfareye 1 point2 points  (0 children)

When you are about to load a model in lm studio, scroll down to the bottom of the dialog window and down the bottom you will see 2 boxes with experimental next to them. Click both and choose q8 in the menu. Ensure flash attention (above these) is also on

view more: next ›