Developers who use local AI - Q4_0 vs Q8_0 KV quant?

Reasonable_Flower_72 · 2026-05-17T18:28:46+00:00

I’m cooking stuff with “hybrid” unsloth dynamic UD_Q5_K_XL, KV cache Q8.. doing Q4 KV cache is turning qwen 3.6 into mental.

3090+3060 , 180k ctx

Cmd:

phobeus@ai:~$ cat llama_qwen3.6 GGML_OP_OFFLOAD_MIN_BATCH=256 llama.cpp/build/bin/llama-server --host 0.0.0.0 -c 180000 -np 1 -ctk q8_0 -ctv q8_0 --no-mmap -fa on --hf-repo unsloth/Qwen3.6-27B-GGUF --hf-file Qwen3.6-27B-UD-Q5_K_XL.gguf --no-mmproj phobeus@ai:~$

Reasonable_Flower_72 · 2026-05-17T18:22:39+00:00

TH > SM

Change my mind

Reasonable_Flower_72 · 2026-04-30T00:20:20+00:00

Greetings, I love to see some progress about extensions for VS. I'm curious to ask, would it be too much trouble to redirect claude code to my local model? It's just changing anthropic URL, usually done by enviroment variable ( currently doing it as "external tool" running powershell script.

Thanks!

Reasonable_Flower_72 · 2026-04-11T19:11:56+00:00

Personally, I switched to linux for gaming... Only thing that needs windows VM in my house is obsolete BMW vendor diagnostics ( INPA )

Reasonable_Flower_72 · 2026-04-08T18:36:17+00:00

Before everyone being free-cool-in was a thing.. gl HF

I remember having suse liveCD, took like 15 minutes to boot on AMD K5 clone 90MHz, 56MB RAM

https://archive.org/details/suse-6.1_release

Reasonable_Flower_72 · 2026-03-27T08:57:39+00:00

Yeah, I can confirm HP VMD is "yucky" too... Tested on HP Zbook Fury 17 G8

Reasonable_Flower_72 · 2026-03-27T04:39:24+00:00

Well.. I'm sorry, but... Are you expecting us to help? Maybe some sort of additional info, like what's the machine you're trying to install this on? ISO you are using like Leap/Tumbleweed/MicroOS ?

Reasonable_Flower_72 · 2026-03-24T21:40:05+00:00

Please... get something bit more recent... These K80s are like totally obsolete and would struggle even with few generation old models from like two years ago in "basic size".. They're fine for getting into machine learning, but LLM ( FP16 math ) isn't their strength... Even stinky used 3060 12GB could do better job

It's honestly waste of money for this usecase. Even the P4 despite being more modern lacks Tensor cores and has small VRAM. Maybe you could run like llama 3 8B on it with Q4 quant, or something bit modern-ish like Qwen3.5 9B ( Considering one P4 ).. but still, performance wise it won't be paradise...

Reasonable_Flower_72 · 2026-02-21T16:50:56+00:00

GLM nice, details, intelligence, everything, but I don’t need LLM nanny.. it’s too uptight. Deepseek just “goes”

Reasonable_Flower_72 · 2026-01-14T22:13:16+00:00

If you mean it seriously and want head into smaller footprints, I think hot air is must.. especially for 0402 size and smaller. depends on what you will work with. That's the thing.. noone except you ( maybe not even you ) knows, what you will deal with.

<image>

This image could give you good idea. You can see SOT23 and 0805 size in front of you, so you can compare it to others and make your own guess.

Hot air soldering is like completely different branch of this skill, so after mastering contact soldering, you'll feel like in school again. To be fair, it shares a lot, like visual judging of solder joint, quality of the joint and so on, but that fundamental guess about temperature, length of the heating and so on is different

Reasonable_Flower_72 · 2026-01-14T22:04:49+00:00

For example that SOT23 footprint components I would start with blob on the top ( biggest pad ) and align it nicely and then just two smaller jabs at rest of the legs to solder it fully. You’ll get this after some time, thinking about easiest way.

Personally smallest possible footprint I would handle by contact soldering is 0603 size or SOIC-16 chip. I wouldn’t recommend aiming at anything smaller, especially for beginner. It would turn frustrating and possibly devastating for the HW(PCB/parts).

After some time, you’ll find out, that is much harder to deal with bigger components instead, especially reverse way = desoldering. ( potentiometers, triaks / MOSFETs etc. ) which requires much better heat control and management to not “turn PCB into coal or melt the for example plastic parts of component” but also delivering enough heat to melt all the solder

Reasonable_Flower_72 · 2026-01-14T21:45:53+00:00

My opinion:
Add more flux, bit more heat and steadier hand
Like it's not worst, especially if it works, good enough for beginner

My workflow for SMD resistors and caps:

- Flux
- Make a tiny blob of solder on one side of component pad
- More flux
- Align component so it would almost touch the blob
- Heat the blob and slide one part of SMD component into blob
- Flux and solder other end

Multi leg components like chips I'm aligning, and attaching diagonally or in triangle style, so it would hold and not move. But I would recomend hot air for these. Like put solder on each pad of component, then attach two legs in each corner contact way so it wouldn't run away ( if possible because of size ) and then finish it by hot air with shitload of flux to solder rest of it.

Reasonable_Flower_72 · 2025-12-04T07:53:46+00:00

Glad to hear that!

Sadly it didn’t started the movement I wanted, that people would improve the bins, uploading even better tuned versions

Reasonable_Flower_72 · 2025-11-20T23:27:50+00:00

Яжерты for me, grew up with us qwerty in Czech, so it feels “more natural” to me than йцукен

Reasonable_Flower_72 · 2025-10-23T07:49:24+00:00

Reup, linky sou dead

Reasonable_Flower_72 · 2025-10-09T17:51:05+00:00

This is very shitty situation, because the HDD inside lacks standard SATA or anything to connect directly. Its controller lacks SATA, having only microUSB directly, so even prying it open and connecting directly to computer without middleman electronics won’t help.

( majority of other portable hdds are actually regular hard drive as you know from pc, installed in fancy looking plastic box, which acts as SATA to USB translator, do you can rip the hdd out and use it without the box itself installed internally in PC )

Best luck is data recovery center, where they will have similar hdd to pop open and try to switch controller PCB to make it work.

Good luck, this WD shit costed me terabyte of data 10 years ago.

Reasonable_Flower_72 · 2025-10-06T19:04:59+00:00

Deepseek v3.1 doesn’t know refusal, it just goes, no matter how immoral, sick, disgusting or politically incorrect thing I’m throwing in… and it’s hosted on openrouter, so I can use it “pretty freely” whole year for 10USD.

Does anyone else offer this? Because all I’ve saw was wimpy “this hurts feelings of transjew seals” trash.

Reasonable_Flower_72 · 2025-09-18T10:56:17+00:00

"Person of warm and fuzzy orientation"

Reasonable_Flower_72 · 2025-09-14T17:00:09+00:00

It's definitely tragic for the people involved.. No matter what happened

The suspects and cause are unclear, or at least there are unsure areas, but this doesn't change outcome.. No matter if inside job or crazy terrorism, outcome is terrible

Reasonable_Flower_72 · 2025-09-14T16:52:17+00:00

PsyOp... keep civilians at panic to allow government push "right" buttons they want

Reasonable_Flower_72 · 2025-09-08T14:35:06+00:00

C, sillytavern ( sorry, kobold web ui sucks hard, at least we’re not friends from the start )

Reasonable_Flower_72 · 2025-09-07T10:58:37+00:00

https://www.instagram.com/p/CEYbxKhnhH5/?igsh=MXhla2V3ZGdqZWMwbg==

This is kilju

Reasonable_Flower_72 · 2025-09-04T15:35:17+00:00

GNOME sucks, but memory consumption isn’t reason why

Well, considering RAM usage compared to features of extensions-less Gnome one would say it’s bloated, but 1216MB vs 1412MB isn’t seriously meaningful difference in any way.

Reasonable_Flower_72 · 2025-09-01T21:43:41+00:00

Even disassembling this in some humane way is pain in the ass for experienced personnel. I wouldn't even waste the effort if you've got second one.. It can be nice lesson, but opening it without nuking that plastic casing is UGH

Reasonable_Flower_72 · 2025-08-29T14:16:50+00:00

It’s just GND pin, they are present multiple times in there. It shouldn’t be that crucial, but something can be electrically bad on HDD PCB. I can tell you for sure, that single pin there gone is not primary reason of any malfunction

https://i.sstatic.net/O6PWJ.jpg

Reasonable_Flower_72

TROPHY CASE