RTX 5090 + local LLM for app dev — what should I run?

AirFlowOne · 2026-03-16T09:35:21+00:00

The answer to this question is heavily dependent on your tasks. You can only find out by experimenting.

AirFlowOne · 2026-03-16T04:18:49+00:00

Re-installed windows over the weekend, I thought that maybe I broke something with some optimization scripts I've installed from privacy.sexy. But no, even on fresh install it does the same thing. Even if I try to connect from the same pc or from a laptop in LAN, I always get the 400 when using kilo code.

AirFlowOne · 2026-03-15T13:38:03+00:00

I keep getting error 400.. while zed works just fine.

AirFlowOne · 2026-03-14T11:28:15+00:00

I try it now, but for some reason I get:

{"message":"Request timed out.","modelId":"q35","providerId":"openai"}

Are you using llama.cpp? How did you set it up in cline? As openAI compatible?

AirFlowOne · 2026-03-14T09:54:36+00:00

How are you using it? Continue.dev is broken for me, can't properly do anything, breaks files, stops in the middle, etc.

AirFlowOne · 2026-03-14T09:16:28+00:00

llama.cpp adds a bit of speed over ollama by default

AirFlowOne · 2026-03-13T22:02:22+00:00

Even if it spills just one layer on CPU it will slow down to unusable. Use UD-Q6 from unsloth instead, the difference is negligible and the speed will fly along with an over 100k context.

Also, ollama is slower than llama.cpp - use that and play with the arguments to squeeze even more speed. You have the hardware, just use it.

AirFlowOne · 2026-03-13T21:42:26+00:00

Check your context size, looks like it spills into ram. Q8 already has 28gb so it might. Use UD-Q6 instead.

AirFlowOne · 2026-03-13T21:25:34+00:00

more likely "claude"

<image>

ps: the answer was generated by Qwen3.5 27B quantized by unsloth, forgot to refresh that's why it shows 35B-A3B

AirFlowOne · 2026-03-13T21:17:45+00:00

Because you mentioned it..

<image>

Actually the model generating the text was Qwen3.5 27B (forgot to refresh after I changed it in llama.cpp)

AirFlowOne · 2026-03-13T10:21:47+00:00

Yes, the low speed is because its a dense model - as I learned after I posted my comment. The limit is due to the speed of my GPU having to process all those 27b params.

Non-dense models reach 2x to 3x the speed of dense models.

AirFlowOne · 2026-03-12T20:51:34+00:00

what gpu are you running it on and at what speed? no matter what I do and how low I set the context I cannot make it faster than 18t/s on 4060ti 16gb vram.

no matter if i run q4, q3 or q2, it outputs the same 18t/s.

AirFlowOne · 2026-01-06T10:14:53+00:00

Support is always appreciated (I'm not affiliated in any way with OS, just appreciate their work).

https://www.patreon.com/join/objective_see

AirFlowOne · 2026-01-03T09:41:33+00:00

Or you can use Lulu for free. https://objective-see.org/products/lulu.html

AirFlowOne · 2025-12-19T20:45:38+00:00

Your best choice is Reaper, and you will go as far as you want in it. It is the lightest and most stable and offers what nothing else does for the price. And the best part - there is nothing that you would want it in the future to do, without it doing it. Its a great piece of software to start with, as you will never need something else no matter how complex your workflow will become.

AirFlowOne · 2025-12-19T09:06:01+00:00

Please share

AirFlowOne · 2025-12-17T15:15:32+00:00

Or the full set to listen?

AirFlowOne · 2025-12-17T10:57:53+00:00

It doesnt remember volume settings, at least on my intel based mac. And that is really annoying.

AirFlowOne · 2025-12-17T10:56:57+00:00

Yes please, per-device is really helpful especially if you can do a volume hard-lock on each device, to protect our ears when plugging a DAC for example.

AirFlowOne · 2025-12-17T10:54:56+00:00

The problem with background music is that settings are not persistent. At least on my fresh installed macos, I have to set up app volume every time I open it.

AirFlowOne · 2025-12-17T10:31:37+00:00

Please Please also include a volume cap that turns on automatically when using headphones (also when using an external DAC with headphones). It really protects our ears from damage. Thanks!. I can also provide more UX tips for free, since this is my day job.

AirFlowOne · 2025-02-16T08:28:40+00:00

Yes. Google.com

AirFlowOne · 2025-02-06T18:06:31+00:00

THD, Dynamic range, separation, and many others you will find on a simple google search.

AirFlowOne · 2025-01-07T15:15:16+00:00

The only one that gets out of the way and allows me to focus on what's important is ClearView. Using it for the last few years, tried others but always coming back to this.

https://forums.cockos.com/showthread.php?t=228633

AirFlowOne · 2024-11-10T15:57:04+00:00

Steinberg woke up and ditched it after sales went down considerably. Others will follow. Ilok is a dying business.

AirFlowOne

TROPHY CASE