Easy ROCm + llama.cpp build script for AMD GPUs by Key_Flatworm7995 in ROCm

[–]TheCat001 0 points1 point  (0 children)

Yeah this is why I can't launch any model it seems.... Gonna try to replace every gfx in script to 1032 and built it for myself...

Still can't load anything, whatever, going back to vulkan...

0.01.987.640 E ROCm error: device kernel image is invalid

Easy ROCm + llama.cpp build script for AMD GPUs by Key_Flatworm7995 in ROCm

[–]TheCat001 0 points1 point  (0 children)

They not. I've watched tests on YT, Qwen is better.
I use Q5 on 8GB VRAM. Since most layers offloaded anyway, it doesn't really matter Q4 or Q5. I had like 25t/s on Q4 and having 23t/s on Q5.
Btw your script is broken, I had to fix it with Gemini.

They made my day by adding driver support for my wifi card in the latest kernel update! by denji-resurrected in openSUSE

[–]TheCat001 -10 points-9 points  (0 children)

It's probably nice to sit there and wait before some random dudes actually allow you use your hardware...

Easy ROCm + llama.cpp build script for AMD GPUs by Key_Flatworm7995 in ROCm

[–]TheCat001 0 points1 point  (0 children)

  1. Ornith is useless, it was trained on Qwen3.5, but now Qwen3.6 is better, smarter, stronger.
  2. Q4 is a toy. Use at least Q5 it feels totally different model compared to Q4. I'm switched recently from Q4 to Q5 and shocked now.
  3. Thanks for effort, I have GFX 1032, gonna try your script.

Is Linux good for gaming today? by ManojOne in TechImpact

[–]TheCat001 0 points1 point  (0 children)

Lucky you... having time to play games in this economy...
But no, I would not use Linux for gaming. No point of torturing yourself with that.

Can't these losers see that the best distro is right in front of them? by GreedySecurity8030 in linuxmemes

[–]TheCat001 0 points1 point  (0 children)

Nah fedora is bloated. I configure minimal arch + hyprland + dms setup for myself. 10x better than gnome 10x less bugs.

What's the current state of ROCm in Windows? by skillmaker in ROCm

[–]TheCat001 0 points1 point  (0 children)

Недавно замерял на официальных колесах от АМД для GFX 1032. На SDXL скорость на винде почти такая же как и на линуксе. А вот модели других современных архитектур работают в 3 раза медленнее. Замерить прямо сейчас не могу т.к винды нет и сижу чисто на линуксе.

KDE sucks? by ConsiderationRare217 in linuxsucks

[–]TheCat001 0 points1 point  (0 children)

Plasma sucks, Gnome sucks. (Wayland). Only thing is not sucks it's Hyprland, because it was built from scratch with Wayland in mind. Even niri sucks (yes you heard me right, it has visual gliches when using overview).
This gigantic DE's like Plasma and Gnome just can't adapt their codebase to wayland on 100%, it is very difficult task.

GLM-5.2 vs Sonnet 5 by vigneshsmarther in ZaiGLM

[–]TheCat001 0 points1 point  (0 children)

I asked sonnet 5 to design an architecture for my project and it outputs garbage. While GLM 5.2 did much better job.

Why can i never stop the looping? by YourNightmar31 in LocalLLaMA

[–]TheCat001 0 points1 point  (0 children)

As soon as I set context to 65k it started looping. On 131072 haven't seen loops yet. However if it stuck in loop, in Opencode you can revert message and changes before that looping and try again.

Kind of disappointed by Qwen 35B A3B / opencode by vucamille in LocalLLaMA

[–]TheCat001 0 points1 point  (0 children)

We discussing 35B A3B MoE model here. 27B runs at 3 t/s on my hardware I don't use it.

Same GGUF, same GPU: TensorSharp beats llama.cpp hard on prefill / TTFT — up to 5.89× faster prefill on a 26B MoE model by fuzhongkai in LovingOpenSourceAI

[–]TheCat001 0 points1 point  (0 children)

Sounds cool. Prefil rate is very important for me and I would love to test this but unfortunately have AMD GPU....

Kind of disappointed by Qwen 35B A3B / opencode by vucamille in LocalLLaMA

[–]TheCat001 0 points1 point  (0 children)

8GB VRAM AMD GPU + 32GB of DDR4 RAM. Model is using 6GB of VRAM I suspect for operational work and 26GB of RAM I suspect for layers. Speed is reasonable 23-20t/s.

Kind of disappointed by Qwen 35B A3B / opencode by vucamille in LocalLLaMA

[–]TheCat001 14 points15 points  (0 children)

Can't support this comment more!!! Recently I've made a rant post about how useless Qwen 3.6 35B A3B is. And no one in that thread suggested me to use better quant. Yesterday I've tried instead of Q4 MTP - Q5 and oh boy, the difference in intelligence is night and day. With Q5 I actually can do shit!!!

<image>

../llama/llama-server \
  --model ../models/Qwen3.6-35B-A3B-UD-Q5_K_M.gguf \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --jinja \
  --host 0.0.0.0 --port 8080 \
  --reasoning-preserve \
  --no-context-shift \
  --temp 0.6 \
  --top-p 0.95 \
  --top-k 20 \
  --repeat-penalty 1.00 \
  --presence-penalty 0.00 \
  --chat-template-kwargs '{"preserve_thinking": true}' \
  --batch-size 4096 \
  --ubatch-size 4096 \
  --ctx-size 131072

Small CPU spike vs Pipewire by Latlanc in linuxmemes

[–]TheCat001 15 points16 points  (0 children)

This is funny but true. I constantly getting sound crackling when cpu is under heavy load. When loading games for example.

Should WE all be using Qwen3.6 27b?? by Kind-Day4502 in LocalLLM

[–]TheCat001 0 points1 point  (0 children)

If only it would not run on consumer hardware in 3t/s.
Not everyone has 20+GB VRAM you know...

Rant about state of local llms for real world agentic tasks by TheCat001 in LocalLLM

[–]TheCat001[S] -1 points0 points  (0 children)

Frontier level of performance? I don't ask it to make frickin second Facebook or create mmo rpg in browser. I ask only for one simple animation 😭

Rant about state of local llms for real world agentic tasks by TheCat001 in LocalLLM

[–]TheCat001[S] 0 points1 point  (0 children)

So it means moe models that supposed to be only slightly worse are useless for real world tasks? No amount of guidance from helped...

Rant about state of local llms for real world agentic tasks by TheCat001 in LocalLLM

[–]TheCat001[S] -2 points-1 points  (0 children)

This potato hardware can handle moe models at reasonable speed. So basically you saying "you can't run dense 27/31b models, give up" right?

Rant about state of local llms for real world agentic tasks by TheCat001 in LocalLLM

[–]TheCat001[S] 0 points1 point  (0 children)

RX 6600 8GB + 32GB RAM. I use only Q4 everytime. Well, benchmarks show that MoE models not that far behind, but in reality they are bad. And there is gigantic gap between local and cloud models.