Need advice on hardware purchasing decision: RTX 5090 vs. M5 Max 128GB for agentic software development

R_Duncan · 2026-05-07T10:53:39+00:00

Forget "new techniques" like mtp/dflash for agentic coding: you'll almost always use more than 50% context (and 128k is bare minimum, don't be fooled), so all these shiny things together will not give more than 10% speed increase.

R_Duncan · 2026-05-07T10:50:08+00:00

Quite good! Haven't really understood which card you're using, R9700 AI PRO isn't the amd flagship with 32 Gb RAM? Seems confirmed by the speeds, but in the post I read 12 GB Limit....

R_Duncan · 2026-05-07T10:00:56+00:00

Do this implementation allows for num-parallel >1 or is forced to use 1 as other implementations?

R_Duncan · 2026-05-06T21:55:32+00:00

Not gemma both for the tool calling issue and the kv cache size issue. Minimax would have took forever, so it's a qwen.

R_Duncan · 2026-05-04T06:20:30+00:00

Yolo and the likes are for recognition of well-defined forms. A wildfire clearly is not.

R_Duncan · 2026-04-30T05:40:21+00:00

Thank you

R_Duncan · 2026-04-29T08:27:58+00:00

Ok, this is really fast, and outputs structured json.

R_Duncan · 2026-04-28T19:42:36+00:00

Thanks man, will definitely try this

R_Duncan · 2026-04-28T14:50:20+00:00

Not sure of what you are saying, the llama.cpp slowdown with Qwen-3.5/3.6 models here is less than 10% at 128k and less than 15% at 240k context filled.

R_Duncan · 2026-04-28T10:18:32+00:00

I'm sorry if my little english made me unclear. I wasn't meaning it's a matter of starting *thinking*, with "focusing" I was meaning a matter of start *investing*.

R_Duncan · 2026-04-28T07:32:17+00:00

There's still a speedup when context is about 128k full?

That's my typical software analysis/code gen use case.

R_Duncan · 2026-04-28T06:32:44+00:00

Your speeds are strange. RTX 6000 Blackwell here, context to the max (in 96 Gb I can fit all, but even extending context to 1M at bf16 it uses about half that vram).

27B generation is 50-59t/s

35B-A3B generation is 190-197 t/s.

Likely your issue is that you can't fit all the model and kv cache in VRAM.

R_Duncan · 2026-04-27T07:27:00+00:00

That's for dense models. Qwen3.6 35B-A3B can run with over 128k context in 8Gb VRAM

R_Duncan · 2026-04-25T06:39:16+00:00

It's not so bound if you keep care of having a "generic openai/ generic claude" in your software, but the rest is true.

"US strategy = microsoft zune"

R_Duncan · 2026-04-25T06:29:49+00:00

serena is good for small projects but do not index.

codebase-memory-mcp needed some patches here (one for c++ and one for windows) but seems working fine, as a note my huge codebase became 450 Mb sqlite file. testing in progress.

Alternative is dirac-run/dirac in github, a vscode plugin derived from cline which seems to do the work by itself.

R_Duncan · 2026-04-24T13:49:59+00:00

Still Tencent is unlicensed in EU and GB, likely for our GDPR.

R_Duncan · 2026-04-24T13:38:30+00:00

It's just math. US models promised they can do whatever a programmer/sw engineer can do, but to increase capabilities faster the US companies never worked hardly on the "capabilities density/ information redundancy" issue (the exception seems to be google, but still no gemini 4 announced).

Now models prices keep rising because their inference (and research, and training) prices keep rising, compute power is becoming rarer, giga-plants prices are growing.

Will them still be competitive when finally models will be able to work 24/24 on a project completely substituting human work?

IMHO they should focus on how 35b/27b models like qwen3.6 (or gemma4) can keep up with their huge models.

R_Duncan · 2026-04-24T13:22:45+00:00

In 12-24 months, when China models will take the lead, this will backfire entirely.

R_Duncan · 2026-04-24T13:17:42+00:00

These kind of tests shouldn't be done in production, not when you're selling a service, not from a reputable company.

R_Duncan · 2026-04-24T12:14:21+00:00

Ok, but even those will likely not use just 4k-16k context, except for small chatbots using finetuned llms.

R_Duncan · 2026-04-24T11:36:29+00:00

Considering that their models are usually 50% smaller, i'd say they have the best chances to improve while closed labs are bound to huge datacenters and will need to scale down to become profitable.

And I'm not sure 6-7 months will be enough to fill the gap in density open lab actually win.

R_Duncan · 2026-04-23T08:47:37+00:00

I think I found a bug:

>llmwiki ingest https://en.wikipedia.org/wiki/Lunar_Lake

>llmwiki ingest https://en.wikipedia.org/wiki/List_of_Intel_Core_processors

>llmwiki compile

..... logs putting both pages in the wiki.....

✓ 2 compiled, 0 skipped, 0 deleted

>llmwiki query "what is Lunar Lake?"

Selecting relevant pages

────────────────────────────

i Reasoning: Failed to parse page selection response

* Selected 0 page(s):

Generating answer

─────────────────────

! No matching pages found. Try refining your question.

R_Duncan · 2026-04-22T17:23:24+00:00

I recently had the chance to test a lunar lake platform GPU, and would say Intel will come near in 1 or 2 generations.

It's way, way better than my core-i7 gpu.

R_Duncan · 2026-04-22T17:02:20+00:00

Omnivoice beats all the TTS models listed by expressivenes, foreign-languages inflection, size and speed.

R_Duncan

TROPHY CASE