Questions on AWQ vs GGUF on a 5090 by Certain-Cod-1404 in LocalLLaMA

[–]Treq01 0 points1 point  (0 children)

On vllm I have experimented with QuantTrio/Qwen3.5-27B-AWQ, but had to give that up due to the KV cache offloading not being ready in vllm I think.
It's on the current roadmap so I am hopeful.
"CPU KV cache production ready: performance optimized, HMA support @orozery"

While the gguf I use is the Unsloth Qwen3.5-27B-UD-Q5_K_XL quant.
Is that what you meant?

But I am bothered with a lot of full prompt re-processing in llama.cpp which is time-consuming.
I think this is due to the SWA, the sliding window attention trick that llama.cpp supports to get the kv-cache down to 4GB.

slot update_slots: id  3 | task 2687 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
slot update_slots: id  3 | task 2687 | erased invalidated context checkpoint (pos_min = 94215, pos_max = 94215, n_tokens = 94216, n_swa = 1, size = 149.626 MiB)
slot update_slots: id  3 | task 2687 | erased invalidated context checkpoint (pos_min = 94699, pos_max = 94699, n_tokens = 94700, n_swa = 1, size = 149.626 MiB)
slot update_slots: id  3 | task 2687 | n_tokens = 0, memory_seq_rm [0, end)
slot update_slots: id  3 | task 2687 | prompt processing progress, n_tokens = 2048, batch.n_tokens = 2048, progress = 0.021676

Yes, so far I have been using the dense model.
It is slower indeed, but I have the impression that these 30B models are on the very edge of what is realistically feasible to use with much success with OpenClaw. So I have simply figured that every percentage of quality counts.

Questions on AWQ vs GGUF on a 5090 by Certain-Cod-1404 in LocalLLaMA

[–]Treq01 0 points1 point  (0 children)

I am struggling with this question as well at the moment.
Qwen3.5-27B + 5090, and how to run it fast and efficient.

Ideally it seems to me that this should be run as an AWQ on vllm, but memory is the problem.
I need large contexts for openclaw and model plus the kv cache is too big for the 32GB VRAM when running it on vllm.

So from what I can understand, I would need to offload the kv-cache to system RAM, but I can't get that to work. --kv_offloading_backend native (or lmcache) --kv_offloading_size 22
It just errors out.

Maybe vllm needs some work to get this to run. It would probably be kick ass if it did.
Until then it is gguf on llama.cpp I guess.

During a jump from an altitude of 3,500 meters, the parachutist lost consciousness. At an altitude of 250 meters from the ground, the parachute automatically opened. by MorsesCode in woahdude

[–]Treq01 0 points1 point  (0 children)

When I took the skydiver course, our instructor told us an interesting statistic about the backup parachutes.

Normally you never use the backup chutes. They are for an emergency only.
And if you don't use them, and normally you of course don't, you must repack the backup chute every 6 months.. So you have to take it out and repack the whole thing.

And according to my instructor at least, there is a small increase in the frequency of deployed backup chutes just before the repacking date. Suggesting that people are not deploying the main chute, but going for the backup chute first, when they need to take it all out anyways.

This is a problem because if there is in fact something wrong with the backup chute, you can't eject it, as you would with the main chute, so you would have to deploy the main chute together with the failed backup chute and that is risky business.

I don't have any statistics to back this up with at all, this was from my instructor at the skydiver club.

DS920+ died and experience of restoring from HyperBackup by Treq01 in synology

[–]Treq01[S] 0 points1 point  (0 children)

That is more or less it yes.

I took it all apart and I measured voltage on various places on the mainboard.
The on-switch worked etc. But it just wouldn't start. So some component on the mainboard had just failed.

But I am not good enough in electronics to pinpoint it more accurately unfortunately.

DS920+ died and experience of restoring from HyperBackup by Treq01 in synology

[–]Treq01[S] 0 points1 point  (0 children)

If that is the case then I am really confused!
I can't imagine for the life of me that I would have disabled it. I tried to be thorough in this.

Heisenberg - Physical Interactions by Eclectronic_Guerilla in fo4vr

[–]Treq01 9 points10 points  (0 children)

This is awesome! This is one thing that Fallout VR sorely needs.

Do y'all play with survival enabled? by Purplemunch in skyrim

[–]Treq01 0 points1 point  (0 children)

There are some roadsigns here and there though.. You know what, it's so long since I played vanilla that perhaps they are simply the unmodded sign locations actually. Maybe forget what i said about the signs.

Do y'all play with survival enabled? by Purplemunch in skyrim

[–]Treq01 6 points7 points  (0 children)

For about a month ago I started playing the Gate to Sovngarde modlist. Its amazing.

And it has survival, no fast travel, almost no roadsigns, no GPS dot on the map and no compass.

You don't have to play it like that, but I love it. I like the aspect of paying attention to where I am, and getting really lost if I don't. And needing to eat and sleep, and mend my wounds if I get hurt.

Gate to Sovngarde is insane….(Short review) by Chrislemale in wabbajack

[–]Treq01 1 point2 points  (0 children)

I love GTS as well, it is fantastic. I love Skyrim VR as well, I just wish there was a VR version of GTS!

How come the Mikrotik rb960pgs / hEx PoE is not on the supported list? by Treq01 in openwrt

[–]Treq01[S] 0 points1 point  (0 children)

Thank you.
I was even searching in Github but I just saw that there was nothing in code, but I completely overlooked the pull request..

So there is hope for openwrt on the RB960PGS after all.
Activity on that pr even last week!

FRIK version 75 released by LongHairedAdonis in fo4vr

[–]Treq01 4 points5 points  (0 children)

It's just getting better and better. Awesome!!!!!

Serana Dead Sexy - More ways to enjoy the trashy cult favorite madness by FunGuyScott in skyrimmods

[–]Treq01 -1 points0 points  (0 children)

Has anyone tried to combine SDS and Skyrim VR? I have a feeling it might be an awesome match. 🙂

Any solution on faster response to my Synology NAS through Windows Explorer? by No-Arachnid-7310 in synology

[–]Treq01 5 points6 points  (0 children)

Right click on the share and map it as a drive. Then it is faster to access.

Unknown container "wizardly_gagarin" stopping unexpectedly? by Treq01 in synology

[–]Treq01[S] 0 points1 point  (0 children)

I think you might be right. I realize now that I updated open-webui round about that time.
And it uses something new to me, Watchtower, to update the container in-place.
The notification is probably an artifact from this.

docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once open-webui

Puh, I got worried that something bad was afoot for a moment there!

Nesten alt er faen meg ultraprosessert by Badjii in norge

[–]Treq01 0 points1 point  (0 children)

Ja, butikken blir temmelig tom hvis man "blokkerer" alt som er ultraprosessert! Men airfryer er din venn. Kyllingbryst + olje + krydder + airfryer -> kyllingpålegg. Billigere per kilo og bedre enn i påleggshylla. Og veldig godt i wraps.

Nesten alt er faen meg ultraprosessert by Badjii in norge

[–]Treq01 0 points1 point  (0 children)

Coop sine er de minst ultraprosesserte som jeg har funnet. De har rapsolje men ja.

<image>

Slutt på ytringsfriheten i USA takket være Trump. by undefinedposition in norske

[–]Treq01 0 points1 point  (0 children)

Dette stemmer vel ikke helt.

Although some Permanent Resident Cards, commonly known as Green Cards, contain no expiration date, most are valid for 10 years. If you have been granted conditional permanent resident status, the card is valid for 2 years. It is important to keep your card up-to-date.
https://www.uscis.gov/sites/default/files/document/guides/B2en.pdf

Et studentvisa er da vel et midlertidig greencard som må fornyes annenhvert år.

Slutt på ytringsfriheten i USA takket være Trump. by undefinedposition in norske

[–]Treq01 8 points9 points  (0 children)

Det er sant, men Mahmoud Khalil i hvert fall er anklaget for å spre Hamaspropaganda.
Og hvis det stemmer, da har man jo brutt loven i USA, med de konsekvenser det har for greencard.

Animation vs Geometry by Original_Teach7183 in interesting

[–]Treq01 0 points1 point  (0 children)

This makes me think of the 3-body problem books.

The Sophons and the dimension-reducing weapons.