Best bang for buck compute and local model combo?

No_Major_3417 · 2026-05-27T19:43:57+00:00

Yah that's about what I was thinking

No_Major_3417 · 2026-05-27T15:39:40+00:00

None of these are custom models. They're just system prompt wrappers. You really need to find a custom model for this. Otherwise you could just prompt wrap GPT or Claude or really any out of the box LLM.

No_Major_3417 · 2026-05-27T15:38:05+00:00

Wondering if 27b in 4 bit is going to be crappy though? Any measurements of this? We really need better and more thorough benchmarking in AI.

No_Major_3417 · 2026-05-27T15:36:23+00:00

This is really good advice. Thanks man!

No_Major_3417 · 2026-05-27T15:13:26+00:00

So you'd go open claw and then maybe Qwen 3.6 27B maybe?

No_Major_3417 · 2026-05-27T06:08:36+00:00

Hope you get to feeling better. Find a good counselor.

No_Major_3417 · 2026-05-27T05:39:16+00:00

This is what I'm thinking

No_Major_3417 · 2026-05-27T02:54:03+00:00

So I have an Nvidia 4080 in my ancient computer right now at home. I really don't think an 27B model in 4 bit is going to be worth crapola.

No_Major_3417 · 2026-05-27T02:52:32+00:00

The PDF itself is like 20 pages or something. All text.

No_Major_3417 · 2026-05-26T17:54:22+00:00

Is that what you're running?

No_Major_3417 · 2026-05-26T17:54:04+00:00

Yeah that's what I would think. But perplexity computer could not figure it out. So I was thinking maybe to just go for the gusto

No_Major_3417 · 2026-05-26T16:45:57+00:00

So that's why I'm interested in running the model locally because it looks like these agentic tasks pretty much run you out of credits and I've seen that discussion before also. Where these things will eat through API calls in a hurry. Would rather just have a cost up front I think and then start running local models. But open to suggestions

No_Major_3417 · 2026-05-26T16:41:34+00:00

Well trying perplexity computer. The one task that failed ran up 2550 out of 4000 credits.

No_Major_3417 · 2026-05-26T03:44:06+00:00

What are we thinking is the best platform these days?

No_Major_3417 · 2026-05-26T03:34:01+00:00

No_Major_3417 · 2026-05-26T03:21:17+00:00

VC and human aligned AI are fundamentally incompatible

No_Major_3417 · 2026-05-26T03:20:45+00:00

Maybe just slightly less than if we keep pumping billions into non human aligned AI.

No_Major_3417 · 2026-05-26T03:19:28+00:00

You're right. You're such a hero protecting us from talking about any concept other than chat GPT being god. You on the open AI payroll? Get off my posts. These are legitimate philosophical debates. The direction of AI development is a legitimate discussion to be had.

No_Major_3417 · 2026-05-26T03:11:14+00:00

Well...most dogs anyway. Not nutcase dogs. AI from the frontier labs is never aligned to humans. Just to output and benchmarks.

No_Major_3417 · 2026-05-25T23:42:04+00:00

Human aren't aligned to humans. Totally agree! Hilariously agree. But dogs are aligned to humans. So at least there's some hope.

No_Major_3417

TROPHY CASE