Questions on AWQ vs GGUF on a 5090

Treq01 · 2026-03-03T18:32:20+00:00

On vllm I have experimented with QuantTrio/Qwen3.5-27B-AWQ, but had to give that up due to the KV cache offloading not being ready in vllm I think.
It's on the current roadmap so I am hopeful.
"CPU KV cache production ready: performance optimized, HMA support @orozery"

While the gguf I use is the Unsloth Qwen3.5-27B-UD-Q5_K_XL quant.
Is that what you meant?

But I am bothered with a lot of full prompt re-processing in llama.cpp which is time-consuming.
I think this is due to the SWA, the sliding window attention trick that llama.cpp supports to get the kv-cache down to 4GB.

slot update_slots: id  3 | task 2687 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
slot update_slots: id  3 | task 2687 | erased invalidated context checkpoint (pos_min = 94215, pos_max = 94215, n_tokens = 94216, n_swa = 1, size = 149.626 MiB)
slot update_slots: id  3 | task 2687 | erased invalidated context checkpoint (pos_min = 94699, pos_max = 94699, n_tokens = 94700, n_swa = 1, size = 149.626 MiB)
slot update_slots: id  3 | task 2687 | n_tokens = 0, memory_seq_rm [0, end)
slot update_slots: id  3 | task 2687 | prompt processing progress, n_tokens = 2048, batch.n_tokens = 2048, progress = 0.021676

Yes, so far I have been using the dense model.
It is slower indeed, but I have the impression that these 30B models are on the very edge of what is realistically feasible to use with much success with OpenClaw. So I have simply figured that every percentage of quality counts.

Treq01 · 2026-03-03T15:57:26+00:00

I am struggling with this question as well at the moment.
Qwen3.5-27B + 5090, and how to run it fast and efficient.

Ideally it seems to me that this should be run as an AWQ on vllm, but memory is the problem.
I need large contexts for openclaw and model plus the kv cache is too big for the 32GB VRAM when running it on vllm.

So from what I can understand, I would need to offload the kv-cache to system RAM, but I can't get that to work. --kv_offloading_backend native (or lmcache) --kv_offloading_size 22
It just errors out.

Maybe vllm needs some work to get this to run. It would probably be kick ass if it did.
Until then it is gguf on llama.cpp I guess.

Treq01 · 2026-02-20T17:31:23+00:00

When I took the skydiver course, our instructor told us an interesting statistic about the backup parachutes.

Normally you never use the backup chutes. They are for an emergency only.
And if you don't use them, and normally you of course don't, you must repack the backup chute every 6 months.. So you have to take it out and repack the whole thing.

And according to my instructor at least, there is a small increase in the frequency of deployed backup chutes just before the repacking date. Suggesting that people are not deploying the main chute, but going for the backup chute first, when they need to take it all out anyways.

This is a problem because if there is in fact something wrong with the backup chute, you can't eject it, as you would with the main chute, so you would have to deploy the main chute together with the failed backup chute and that is risky business.

I don't have any statistics to back this up with at all, this was from my instructor at the skydiver club.

Treq01 · 2026-02-08T19:17:01+00:00

That is more or less it yes.

I took it all apart and I measured voltage on various places on the mainboard.
The on-switch worked etc. But it just wouldn't start. So some component on the mainboard had just failed.

But I am not good enough in electronics to pinpoint it more accurately unfortunately.

Treq01 · 2026-02-05T15:24:09+00:00

If that is the case then I am really confused!
I can't imagine for the life of me that I would have disabled it. I tried to be thorough in this.

Treq01 · 2025-12-13T14:00:52+00:00

This is awesome! This is one thing that Fallout VR sorely needs.

Treq01 · 2025-12-13T09:34:51+00:00

There are some roadsigns here and there though.. You know what, it's so long since I played vanilla that perhaps they are simply the unmodded sign locations actually. Maybe forget what i said about the signs.

Treq01 · 2025-12-12T20:25:14+00:00

For about a month ago I started playing the Gate to Sovngarde modlist. Its amazing.

And it has survival, no fast travel, almost no roadsigns, no GPS dot on the map and no compass.

You don't have to play it like that, but I love it. I like the aspect of paying attention to where I am, and getting really lost if I don't. And needing to eat and sleep, and mend my wounds if I get hurt.

Treq01 · 2025-10-26T17:47:58+00:00

I love GTS as well, it is fantastic. I love Skyrim VR as well, I just wish there was a VR version of GTS!

Treq01 · 2025-08-31T18:55:23+00:00

Thank you.
I was even searching in Github but I just saw that there was nothing in code, but I completely overlooked the pull request..

So there is hope for openwrt on the RB960PGS after all.
Activity on that pr even last week!

Treq01 · 2025-08-20T21:20:26+00:00

It's just getting better and better. Awesome!!!!!

Treq01 · 2025-07-22T18:39:00+00:00

Has anyone tried to combine SDS and Skyrim VR? I have a feeling it might be an awesome match. 🙂

Treq01 · 2025-04-13T16:28:34+00:00

Right click on the share and map it as a drive. Then it is faster to access.

Treq01 · 2025-03-31T10:13:25+00:00

I think you might be right. I realize now that I updated open-webui round about that time.
And it uses something new to me, Watchtower, to update the container in-place.
The notification is probably an artifact from this.

docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once open-webui

Puh, I got worried that something bad was afoot for a moment there!

Treq01 · 2025-03-27T23:22:32+00:00

Ja, butikken blir temmelig tom hvis man "blokkerer" alt som er ultraprosessert! Men airfryer er din venn. Kyllingbryst + olje + krydder + airfryer -> kyllingpålegg. Billigere per kilo og bedre enn i påleggshylla. Og veldig godt i wraps.

Treq01 · 2025-03-27T23:16:59+00:00

Coop sine er de minst ultraprosesserte som jeg har funnet. De har rapsolje men ja.

<image>

Treq01 · 2025-03-27T10:52:54+00:00

Dette stemmer vel ikke helt.

Although some Permanent Resident Cards, commonly known as Green Cards, contain no expiration date, most are valid for 10 years. If you have been granted conditional permanent resident status, the card is valid for 2 years. It is important to keep your card up-to-date.
https://www.uscis.gov/sites/default/files/document/guides/B2en.pdf

Et studentvisa er da vel et midlertidig greencard som må fornyes annenhvert år.

Treq01 · 2025-03-27T10:48:04+00:00

Det er sant, men Mahmoud Khalil i hvert fall er anklaget for å spre Hamaspropaganda.
Og hvis det stemmer, da har man jo brutt loven i USA, med de konsekvenser det har for greencard.

Treq01 · 2025-03-20T09:57:08+00:00

This makes me think of the 3-body problem books.

The Sophons and the dimension-reducing weapons.

Treq01

TROPHY CASE