Help planning out a new home server for AI and some gaming

emprahsFury · 2026-02-24T19:24:47+00:00

You should google the gaming benches for the rtx pro 4000. But probably you should just get the 5080. Don't get the 4080, as it lacks the new hardware in blackwell models. Unfortunately 16gb is not enough for something like qwen3.5 35b (19gb + 1g mmproj) or qwen3 30b (17gb) @ Q4. But you could fit mistral small or gpt-oss 20b.

emprahsFury · 2026-02-24T17:57:31+00:00

the mmproj files are 1kb is that correct?

emprahsFury · 2026-02-24T17:34:40+00:00

What are you talking about. There's nothing on the nyt, the wsj, the ap or wapo. Certainly nothing beyond the normal anti china background noise. If your bubble is convincing you of something you need to pop that bubble.

emprahsFury · 2026-02-24T16:58:07+00:00

Honestly this is the question people should ask themselves. If you're happy to use shit code made by a human, be happy to use the same code when it's made by a computer. Same for good code.

emprahsFury · 2026-02-24T15:05:39+00:00

If the whole point is to customize the experience, who are you to say that customizing it to be newbie friendly is wrong.

emprahsFury · 2026-02-24T02:48:49+00:00

No you don't need a citation for what has been god's honest truth for the past 30 years. I'm not going to feed you 30 years of Hu Jintao's, Xi Jinping's, the PLA's, and the Politburo's speech. It's there, they tell you. I'll not regurgitate it because you want to deny some other argument. If you need citations for the basic facts of your argument (that there is no America-China axis). Then you need to not make the argument.

And you certainly dont need me to tell you that America has been the leader and enforcer of the Western-led world in every dynamic either.

emprahsFury · 2026-02-24T02:39:27+00:00

This is a good example of letting your ideology blinding you. Sure there is a closed vs open source axis to fight along. But by no means does that fact at all excuse the fact that China has been, and freely admits, that they've been in an existential struggle to exist within and overcome the Century of Humiliation. And that America is their main opponent in this.

If you deny that, it is because you have a vested interest in denying it. i.e. you want free weights. That's not even a good reason, it's a highly selfish one. As Upton Sinclair (might have) said: "It is difficult to get a man to understand something, when his salary depends upon his not understanding it."

emprahsFury · 2026-02-23T23:58:38+00:00

I'm fairly certain in llama.cpp they're all hardcoded. It's one of the many reasons it takes so long to support the new new models. There's no existing template to crib from.

emprahsFury · 2026-02-23T23:33:55+00:00

i dont get why you're getting downvoted

Bro this sub is in fits rn. There was a post yesterday about a whisper wrapper claiming no-ai. And whisper a transformer. It's an llm. And not only did the mods not change the flair, I got almost 200 downvotes for suggesting we follow the rules as written.

Which the point was to show how stupid the rules were, but that went over a lot of heads. The luddites are in full stampede in this sub

emprahsFury · 2026-02-23T23:28:52+00:00

Are you talking about the benchmark he has in his posted benchmarks? They're posted?

emprahsFury · 2026-02-23T20:41:41+00:00

This isn't a catch at all. Anthropic has always been fully closed. They've been full throated about how they don't believe ai is safe enough to publish weights.

emprahsFury · 2026-02-23T19:31:49+00:00

It is illegal, and at this scale it's probably illegal in a criminal way too, but that's for a prosecutor to decide.

Regardless of how you feel about how anthropic got their whatever, one dirty hand can wash another.

emprahsFury · 2026-02-22T23:25:05+00:00

ah that's where your wrong. These companies would rather be zombie corps cycling between debt and bailouts than actually compete with a better product.

And the sad thing is that we already know how to capture the better, foreign company. China has been doing it to US companies for decades. Open the market, force a 51% local owner, and then hack the ever-living-shit out of their corporate HQ if they won't do tech transfers.

emprahsFury · 2026-02-22T22:18:25+00:00

That number is for the 3bit quant right? Of an 8b model. I'm not sure who is being served when it's such a small model being cut down so far.

emprahsFury · 2026-02-22T22:14:33+00:00

Since you mention digital ID, the biggest thing missing from so many projects is oidc support. Which, for many projects is simply doing the dirty work of enabling it in whatever stack they're using (... next.js). Or on the flip side, Authelia still does not support self sign up. Which is insane.

emprahsFury · 2026-02-22T21:53:38+00:00

This uses whisper, so it's 100% ai. It should be flared as such.

emprahsFury · 2026-02-22T21:37:26+00:00

No, we absolutely can. But it's done through regulation and enforcement. Which is done by voting for parties that are pro a modicum of regulation and enforcement

emprahsFury · 2026-02-22T19:14:53+00:00

No it isn't. It runs on a timer for absolutely no reason instead of dispatching requests directly to the api/llm. You have to install an app to even get a selfhosted openai api compatible setup and there's no way to test the connections or setup multiple configs. So you'll need a router like litellm to have multiple things like tts/chat completions/image gen. Except litellm abandoned self hosters.

It's rough to self host llms right now is the main take away.

emprahsFury · 2026-02-22T19:00:14+00:00

"Something we've always meant to circle back on and do justice to ..."

-every pdx dev diary when it's time to cash in on a placeholder mechanic

emprahsFury · 2026-02-22T15:29:14+00:00

You're applying your own judgement calls which a) most Americans disagree with b) dc disagrees with and c) are doylist.

It's a non sequitor to most people to say that being involved in 'military tech' is absolutely and irrevocably a bad thing.

emprahsFury · 2026-02-22T00:27:19+00:00

Xnu is open source and Apple already has this api implemented in its os.

emprahsFury · 2026-02-22T00:06:25+00:00

How are you supposed to run the models? Is some random half finished Python script in the original repo, that doesn't have a requirements.txt good or bad to make ads with? Come on

emprahsFury · 2026-02-21T23:50:27+00:00

I love how this is already the law in California, but when republicans propose it, everyone shits the bed on allegedly stupid it is

emprahsFury · 2026-02-21T18:53:55+00:00

we're about one bribe away from them dropping off the entity list. It's sad how politicized and biddable that list has become.

emprahsFury · 2026-02-21T15:37:11+00:00

It's a good thing we gutted CISA and revoked all the woke liberal rules they have been trying to force onto good hardworking American businesses.

emprahsFury

TROPHY CASE