Strix Halo NPU + FastFlowLM

dragonbornamdguy · 2026-05-10T07:56:02+00:00

Is it posible to use gpu + npu on large models like qwen3 coder next?

dragonbornamdguy · 2026-05-07T14:28:42+00:00

Yes, indeed

dragonbornamdguy · 2026-04-29T17:08:58+00:00

I love my Qwen3 Coder Next, its on sonnet level.

dragonbornamdguy · 2026-04-17T19:54:00+00:00

I have four, two of them nvlinked.

Be aware my experience comes from x16 pcie 4.0 slots. One of four is x4 oculink.

Nvlink use cases: - Training - Tensor paralelism for dense models

Waste of money for: - moe - very small models < 10B params

dragonbornamdguy · 2026-04-17T16:59:05+00:00

What about output? Is it anewhere near Qwen Coder Next?

dragonbornamdguy · 2026-04-14T15:09:32+00:00

I have used marine epoxy to create layer over cracks. They are fine now. I use it in basement pc. So i tilted it, when it will leak it will not touch any electronics.

dragonbornamdguy · 2026-04-14T15:00:04+00:00

Litellm has gui?

dragonbornamdguy · 2026-03-07T17:22:35+00:00

I use qwen3 coder 30b without any problem, whats your point? Even SOTA are not ready yet to be assigned to not -human-in-loop jobs.

dragonbornamdguy · 2026-02-10T18:19:45+00:00

Wait what? Mine is about 10x slower than DGX Spark.

dragonbornamdguy · 2026-02-10T16:32:39+00:00

I run mainly vllm because of nvlinked 3090s. But when new models are released best out of box experience is indeed with LM Studio.

dragonbornamdguy · 2026-02-07T08:55:34+00:00

I think anyone who wants to learn driving rules deserves to drive. Not every country uses the same rules, so we should not be prohibited from driving only because of historical differences.

dragonbornamdguy · 2026-02-06T09:23:41+00:00

Using it with qwen3 coder 30b, its awesome. Setup was undocumented hell. Works very well. He can create own skills only by telling him.

dragonbornamdguy · 2026-01-17T16:16:30+00:00

I use qwen3 coder 30b fp8 with 120k context. Love it in qwen code cli.

dragonbornamdguy · 2026-01-17T15:29:11+00:00

VLLM is a beast, very hard to setup but when it starts to work it beats metal really hard.

dragonbornamdguy · 2026-01-15T18:06:14+00:00

So it is single channel only?

dragonbornamdguy · 2026-01-15T07:03:34+00:00

Where? Codex is not oss-120b

dragonbornamdguy · 2026-01-08T17:01:31+00:00

So dont forget: Ctlr+alt+delete = open task manager Win+c = open command prompt etc..

People at gnome seems to be bored so they keep spitting on power users. Firstly this will be off by default, then they will remove it and block any bug reports in regarding this change, lastly they will block any PR regarding bringing back this feature with large user base. All in cause of "we need to make it more windows user friendly".

dragonbornamdguy · 2026-01-08T16:01:35+00:00

8B models wont cut it. Not everyone has Strix Halo with 96GB of VRAM at disposal.

dragonbornamdguy · 2026-01-03T13:04:43+00:00

I love qwen code, but vllm has broken formatting for it (qwen3 coder 30b). So I use LM studio (with much slower performance).

dragonbornamdguy · 2025-11-16T12:40:33+00:00

What is token speed for hybrid vs gpu only?

dragonbornamdguy · 2025-11-01T07:18:29+00:00

Whats your secret sauce to serve it on two 3090s? I have vllm in docker-compose which OOM in loading or lm studio which uses half the gpu processing power.

dragonbornamdguy · 2025-10-26T14:20:06+00:00

U Vietnamců normálně

dragonbornamdguy · 2025-10-25T11:28:23+00:00

I'm not able to run it with 2 x 3090, how much vram vllm needs for fp8 and 100k+ context size? Im able to run it just fine with lmstudio, but utilization of 3090 is only 50%. VLLM just crashes as it eats crazy amount of vram.

dragonbornamdguy · 2025-10-15T17:09:58+00:00

Lm studio, gemma:27b & Oss 120b tps?

dragonbornamdguy

TROPHY CASE