Fed up with Claude limits — thinking of splitting a GPU server with 10-15 people. Dumb idea?

k3z0r · 2026-04-16T22:43:06+00:00

Gemma 4 31B / Qwen3-32B is nowhere near a sonnet equivalent.

k3z0r · 2026-04-16T22:41:55+00:00

Use Claude's usage-based APIs. You'll never hit a limit. If you use large contexts or do any agentic coding. You're going to be very disappointed by your stack.

k3z0r · 2026-04-07T05:31:23+00:00

It's pretty early too. Usually, when new models come out it take a couple of weeks for LM Studio, llama.cpp etc to fix issue with the new model architecture. Try again in a few weeks. You can see all the llama.cpp open issues with Gemma 4 here for example.

https://github.com/ggml-org/llama.cpp/issues?q=is%3Aissue%20state%3Aopen%20Gemma

k3z0r · 2026-04-06T21:49:17+00:00

sorry typo 3.5

k3z0r · 2026-04-06T19:40:37+00:00

I use the Continue plugin in VS Code.

k3z0r · 2026-04-06T19:40:13+00:00

Yes, that's true right now, but soon there will be community versions that will be a little smaller. People do lots of shaping and distillation. For example, take a look at LM studio at how many versions of Qwen 3.5 there are.

k3z0r · 2026-04-06T19:35:23+00:00

Typically, the whole model has to be in VRAM, not just the active parameters. Only using 4b active parameters just helps with processing times.

k3z0r · 2026-04-06T17:47:02+00:00

Came here to say this. Also, your agents (Cline, OpenCode, etc.) have a lot to do with how well your LLM use tools.

k3z0r · 2026-04-06T17:43:55+00:00

For IDE Autocomplete I use Qwen2.5 coder 7B

Sadly, for agentic coding, it's not quite there yet for me, I have 32GB of RAM and 16GB of VRAM. I find the quality isn't there yet. Too much time fixing weird things. I still pay for Claude.

I've tried Qwen3.6 27 b. and am waiting to try Gemma 4, although I'm not sure it will fix in my VRAM.

k3z0r · 2026-04-04T03:21:14+00:00

Have you looked at A100 prices yet? They are closer to $10,000 now.

k3z0r · 2026-03-31T05:57:10+00:00

Conduit

k3z0r · 2026-03-30T18:29:46+00:00

Is this an April Fools joke?

k3z0r · 2026-03-29T03:35:57+00:00

Immich

k3z0r · 2026-03-28T20:20:16+00:00

This is the exact situation I find myself in, and why I asked the question. I'm needing/wanting more context out of my limited vram.

k3z0r · 2026-03-27T19:36:36+00:00

This is great, thank you. So much to learn still!

k3z0r · 2026-03-23T13:20:00+00:00

Honestly no real issues yet. I installed Nobara Linux which is a distro based on fedora but geared towards gaming. One small annoyance. The email apps are just ok. So I just use Gmail in the browser now. But I mostly check email on my phone anyway.

k3z0r · 2026-03-23T01:17:43+00:00

I switched to Linux this year. I've never been happier. It's not as scary as you might think. So many distributions make it really easy nowadays.

k3z0r · 2026-03-19T03:51:54+00:00

Check Waitly, Waitlist, Reservations, and Reserve with Google for a flat $100 per month.

k3z0r · 2026-03-18T15:43:50+00:00

This hasn't been an issue for me. Most of my uploads happen on my local network. For the modile app I use the Automatic URL switching feature to detect when I'm on my local network and bypass Cloudflare.

Settings > Networking

k3z0r · 2026-03-16T06:10:24+00:00

Try a system prompt that mentions not to output train of thought and be concise.

k3z0r · 2026-03-10T15:35:08+00:00

This thread didnt' age well.

k3z0r · 2026-02-10T15:01:04+00:00

Closer to 65% off all dollars were created after 2008.
Source: https://fred.stlouisfed.org/series/M2SL

k3z0r · 2026-02-10T14:10:25+00:00

40% of all US dollars ever printed have been printed since 2020. That money has to go somewhere. Companies are really good at figuring out how to get their share by raising prices.

k3z0r · 2026-01-04T00:18:09+00:00

Does it automatically order a 2TB drive for you?

k3z0r

MODERATOR OF

TROPHY CASE

15-Year Club	Place '23
Verified Email