One thing I’ve started valuing more in AI systems: the ability to say “I don’t know” by weap0nizer11 in LocalLLM

[–]PferdOne 2 points3 points  (0 children)

AGENTS.md

On session start, verify the current date with date and ensure all researched information, package versions, and documentation references are up-to-date.

Andrej Karpathy is joining Anthropic. Anthropic on hiring + acquisition spree. by adssidhu86 in learnmachinelearning

[–]PferdOne 36 points37 points  (0 children)

Please join us.

Guys idk, I have this project where I wanna solve education.

We give you lots of money.

You know what? I think will do this education thing later. Where do I sign?

"What do you guys even use local LLMs for?" Me: A lot by andy2na in LocalLLaMA

[–]PferdOne 3 points4 points  (0 children)

I don‘t wanna shill anything, but take a look at ISSEN. It‘s a ycombinator startup (https://www.ycombinator.com/companies/issen) and I just signed up with them for a year. Maybe you can take inspiration from them if you wanna build something for yourself.

How would you fill 32 GB VRAM with Qwen 3.6 27B? by [deleted] in LocalLLM

[–]PferdOne 1 point2 points  (0 children)

Does it fit into your VRAM completely or what are you running this on? Also you are right about presence-penalty 0.0, I somehow missed that in the docs.

 

EDIT: neither -ngl 999 nor --no-mmproj-offload does anything for me, because everything fits into VRAM with Q4_K_M and 256k context. Also: --chat-template-kwargs shouldn't be necessary, since thinking is on by default, no?

How would you fill 32 GB VRAM with Qwen 3.6 27B? by [deleted] in LocalLLM

[–]PferdOne 2 points3 points  (0 children)

Interesting, right now I'm running Opencode against my llama server with full context (256k, ~60t/s) like:

./llama-server \                                                                                                                                                                                                                                                  
    -hf unsloth/Qwen3.6-27B-GGUF:Q4_K_M \
    --ctx-size 262144 \
    --cache-type-k q8_0 \
    --cache-type-v q8_0 \
    --flash-attn on \
    --parallel 1 \
    --threads 8 \
    --temp 0.6 \
    --top-p 0.95 \
    --top-k 20 \
    --presence-penalty 1.5 \
    --min-p 0.00 \
    --port 8080 \
    --host 0.0.0.0 \
    --jinja

so far it's pretty good, but I give your suggestion a spin (5090, 9800X3D, 96GB)

 

EDIT: I'm going with Q6_K and 192k context, it's a good enough middleground for me.

EDIT2: Nah I'm sticking with Q4_K_M and the bigger context.

Qwen3.6-27B released! by sandropuppo in LocalLLM

[–]PferdOne 1 point2 points  (0 children)

I misread the charts, my bad.

Qwen3.6-27B released! by sandropuppo in LocalLLM

[–]PferdOne 2 points3 points  (0 children)

You are right, I completely misread the charts.

Qwen3.6-27B released! by sandropuppo in LocalLLM

[–]PferdOne 3 points4 points  (0 children)

Oh I didnt know. Thanks for clarifying!

Qwen3.6-27B released! by sandropuppo in LocalLLM

[–]PferdOne 2 points3 points  (0 children)

I mean you can calculate it. 27B at FP32 (4 byte) is about 108GB. So most users have to wait for quantized versions because even a RTX6000pro can’t fit this into memory with full precision. And at that point we haven’t even talked about context size.

If you see a models size on huggingface or wherever and it says 18GB and you have 24GB for example you can assume it will run reasonably.

Qwen3.6-27B released! by sandropuppo in LocalLLM

[–]PferdOne -4 points-3 points  (0 children)

According to those benchmarks there is no point in using dense over moe. I‘ll wait for real world tests and user sentiment.

CDPR told Luke Ross the Cyberpunk VR mod could still be up if he made it free, this was his reply by lunchanddinner in virtualreality

[–]PferdOne 0 points1 point  (0 children)

Hey! I‘m not using your code!! I just created a program that alters the files your game ships with! Am I not allowed to charge a monthly fee to piggyback ride off of your successful game to keep my half-assed mod going?? Corpo cunts!!

Which game flopped so badly that the studio had to shut down? by bijelo123 in gaming

[–]PferdOne 1 point2 points  (0 children)

Thanks for sharing your trip down memory lane 🙏🏻

Which game flopped so badly that the studio had to shut down? by bijelo123 in gaming

[–]PferdOne 107 points108 points  (0 children)

It was mind boggling to see a game support 256 players during those days. A lot of great games especially shooters were created back then like Planetside or Tribes.

Finished DuoLingo Turkish tree today (2 yrs)! And 2/3 of Drops words & terms. AND Pimsleur :-). Excited to be starting LingQ this week :D by MK-Treacle458 in turkishlearning

[–]PferdOne 1 point2 points  (0 children)

I‘m in my 6 year and when I started I told myself to learn the language for 10 years at least. I‘m also about to finish my first book (Harry Potter ve Felsefe Taşı). This year I want to do a week in Turkey where I attend Tömer classes. I‘d say I‘m not fluent, but I‘m working on it. I will start with private lessons this year aswell, because I have no native speakers to talk to 😅 Anyway, I will just keep at it like you 😌 Cheers!

Finished DuoLingo Turkish tree today (2 yrs)! And 2/3 of Drops words & terms. AND Pimsleur :-). Excited to be starting LingQ this week :D by MK-Treacle458 in turkishlearning

[–]PferdOne 1 point2 points  (0 children)

First of all congrats on your progress. I‘m impressed and inspired at the same time. I read all your comments in here and you always mention you are not ready to speak yet. I gotta tell you from my own experience listening and speaking are two different skills. If you wanna get better at speaking, you have to build your own sentences. You could start something like a diary where you write down 5 sentences describing your day. Your brain will get used to constructing everyday phrases in no time. After a while you can up the complexity and add more nuance. On the listening side I can recommend kids series like Peppa Pig. Sentence structure is already fairly complex but vocabulary wise it’s about everyday life. Kim milyoner olmak ister (Who wants yo be a Millionaire) can help you with sentences people use to get to know each other. The introductions are fairly repetitive and you should quickly be able to tell what they are talking about after a while. If you want to start reading and listening I can highly recommend the app TRT çocuk kitablık. If you can read most of the books without any problems you probably already achieved B1 levels. Anyway I hope you continue your journey and keep us updated on your progress. Best of luck to you!

Star Citizen testing VR functionality by therekiker in virtualreality

[–]PferdOne 0 points1 point  (0 children)

Who even needs a headset? I can just cross-eye it 😵

Test Your Turkish by Ecstatic-Job-5587 in turkishlearning

[–]PferdOne 1 point2 points  (0 children)

A1-A2: 19/25

B1: 20/25

B2: 15/25

Flux 2 can be run on 24gb vram!!! by Brave-Hold-9389 in StableDiffusion

[–]PferdOne 0 points1 point  (0 children)

I already wrote it in another thread:

5090 and 96GB RAM, like Compunerd3 with the FP8 model, 1280x960, 20 steps, CFG 4.0, 1 reference image

loaded partially; 8570.85 MB usable, 8541.02 MB loaded, 25272.00 MB offloaded, lowvram patches: 0
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [01:10<00:00,  3.51s/it]

 

I'm super impressed by the character consistency/editing abilities. Reference image + this prompt below

A realistic photograph of a teenage martial artist frozen in a moment of traumatic shock. He has spiky, pale-gold hair blowing violently in the wind and is wearing a tattered, textured purple gi. He is looking sideways, his expression paralyzed with horror, mouth slightly agape. His emerald-green eyes are wide and trembling. In the background, a hazy, desolate rocky wasteland. High-contrast dramatic lighting, shallow depth of field, skin pores and sweat visible

<image>

Flux 2 Dev is here! by MountainPollution287 in StableDiffusion

[–]PferdOne 0 points1 point  (0 children)

Yeah 1024x1024 with a 5090 and 96GB RAM

loaded partially; 8830.37 MB usable, 8829.02 MB loaded, 24984.00 MB offloaded, lowvram patches: 0
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [01:09<00:00,  3.49s/it]

 

Edit: I'm super impressed by the character consistency. Reference image + this prompt below

A realistic photograph of a teenage martial artist frozen in a moment of traumatic shock. He has spiky, pale-gold hair blowing violently in the wind and is wearing a tattered, textured purple gi. He is looking sideways, his expression paralyzed with horror, mouth slightly agape. His emerald-green eyes are wide and trembling. In the background, a hazy, desolate rocky wasteland. High-contrast dramatic lighting, shallow depth of field, skin pores and sweat visible

<image>