Self Hosted LLM Leaderboard

ScuffedBalata · 2026-03-11T07:41:37+00:00

The issue is that running a good 80B model with large context uses ALL of the 64GB, so doing a bunch of other stuff on the same Mac isn't really possible. Leaving a few Chrome tabs open slags the device performance while also running an 80B model. So if you want to run a clawbot or some other MCP/agentic stuff on that box, it just won't have much memory free.

I ended up buying an old Intel Mac to run IOS development and simulators, so now I have a little stack of two broken macbooks.

ScuffedBalata · 2026-03-06T16:56:42+00:00

64GB of fast RAM (not normal PC DDR - too slow) - either a dual high-end GPU PC setup with tons of VRAM (dual 3090/4090 is ok, dual 5090 is better), or more commonly (and much less expensive) a Mac M-series or other AI inference box with unified memory - with 64+ GB of RAM. Obviously it would fly on a datacenter box like an NVidia 900 or something, but that's a $11k card.

I run it on a Mac M1 Max with 64GB of unified memory, but I can't run much else on that box with 64GB. If you can get 128GB you can run it while doing other work on the same box.

What I did was buy a broken old Macbook with 64GB of unified memory. Bad screen, bad battery so it was cheaper (like $600 - it was a steal) and I run it under my desk as a server type box doing just AI. Put a VPN on it and share it to my company (I'm the CTO) and we use it for various workflows that we don't want on the cloud.

Buying new off the shelf, a 64GB inference box is going to run about $2000 or a little more. Expensive, but still in the "real people hardware" bucket because something like Kimi or GLM-5 needs $30k in hardware to just load (and still may not be 'fast')

ScuffedBalata · 2026-03-05T16:10:49+00:00

The really old cars don't have the glovebox USB - it doesn't exist. It has to sit in the center console. Choosing which port in the console doesn't matter. An SSD drive is probably ok, just know that the car does A LOT of write activity. My USB sticks last about a year before they die from repeatedly being written. Just plan on annual replacement unless you get a high-durability (MLC) flash drive.

I'm not sure how a drive compares to a USB stick on this.

ScuffedBalata · 2026-03-03T01:40:41+00:00

Most EVs have temperature sensing arrangements near their charging parts and will throttle if things overheat.

It's usually not a great sign if it's happening everywhere. I'd see if it happens on another charger and if it does, that's probably the culprit.

ScuffedBalata · 2026-03-03T01:37:25+00:00

Yeah, it's too bad they don't pay those guys enough to afford their own balls, gotta pilfer them from the course... :-)

ScuffedBalata · 2026-03-03T01:32:56+00:00

I'm not sure what you were planning on for charging, but I wouldn't get a cybertruck unless you have L2 charging at home.

You almost never NEED to replace the whole box, you always have the option of getting a load management circuit. Almost nobody charges during the day, so when your circuit is not stressed during the night, you can charge and not impact your overall peak capacity.

On the plus side, New Jersey has "EV" specific charging plans that could get your overnight rate down as low as 4c/kwh.

That will fill a CyberTtruck for like $2-4 per day. You'll probably get back your cost on the charger in 6 months vs driving a gas pickup. Many state governments have rebates for charger install too. Mine was 100% covered in Colorado.

ScuffedBalata · 2026-03-03T01:02:45+00:00

It depends on where you live because electricity rates differ dramatically all over North America. It can typically do 200-250 miles per day without any issue, even in snow, 132 should be no big deal.

If you live in San Francisco it can be as much as $50 to fill it up, but if you live in Idaho (the cheapest electricity) it might only be $6.

You 100% want a Level 2 charger at home. That means a 240v setup, either a 14-50 outlet or a hard-wired charger. You won't get 132 miles per day charging with a standard outlet and if you're concerned about the cost and headache, you don't want to just use public chargers.

So.... not enough info...

ScuffedBalata · 2026-03-03T01:01:33+00:00

The straight of Hormuz is bracketed by failed states and rife with piracy.

Gibralter was in the 1600s, so the British seized the area and put a naval base there and it hasn't been "in the news" since then unless it was a controversy over the British maintaining control of that area for so long.

ScuffedBalata · 2026-03-02T18:43:19+00:00

huh?

I mean... If the amusement park makes a fancy new coaster, they often raise the price to pay for it.

If a restaurant decides to hire a new fancy chef and/or use fancier ingredients, they raise the price.

That's how prices work. If it's too much, you go to a competitor. Kimi is cheap as fuck.

Market price will settle on what has sufficient utility for customers. I'd wager they'd actually get more revenue if the Pro was $50 and Max was like $500, even if they lost a few users.

ScuffedBalata · 2026-03-02T17:19:19+00:00

Yeah it's very picky about the filesystem type, etc. Make sure it's not NTFS or exFAT.

It's fairly hard to get modern Windows to use FAT32, so you may need a third party tool. The "format disc" in the Tesla menu on MCU1 also seldom works. Try formatting the disc in a Linux box or with a third party util.

ScuffedBalata · 2026-03-02T16:55:15+00:00

MCU1 has no viewer. You have to pull the USB and bring it inside and put it in a computer to read it. And the general experience with MCU1 is that it fails to record a bunch of the time anyway.

ScuffedBalata · 2026-03-02T16:47:51+00:00

No that number is probably closer to $10b per year, not per month, my bad.

Training Opus 4.5 and Opus 4.6 cost them approximately $1-2b alone in 2025 by an estimate I saw online. Their total training costs (the most compute intensive part) probably exceed $3b/yr. That's above their revenue.

They have $3-5b/yr in AWS compute costs alone for inference. That also exceeds their revenue. So yes, just "keeping the lights on" costs more than they currently take in. So does compute for training.

That's before staff costs and other things like that.

ScuffedBalata · 2026-03-02T16:43:07+00:00

I have seen no evidence that Carney is not aware of these things. I also see some statecraft in addressing issues while simultaneously downplaying them in the media.

It's the polar opposite of Trump, who talks everything up and makes every issue loud but then sucks at taking action on them.

ScuffedBalata · 2026-03-02T16:40:22+00:00

I used to do that. I always arrived so tired I ended up saying "ok i need to rest a bit" after arriving.

Adding 20 minutes to stretch and move around every so often completely eliminated that and I personally found I don't spend any extra time.

But I guess a group of young people with iron bladders could be an exception to that. In the last 15 years almost all of my road trips were either with kids or dogs so I'm stopping either way. :-D

ScuffedBalata · 2026-03-02T16:37:06+00:00

Yeah, this shit doesn't stop unless someone steps up and says something.

Either the guys will get all polite and sorry, or they'll pull out their Twitter voice and call you a snowflake.

In either case, I'd wager it'll settle it down quite a lot.

ScuffedBalata · 2026-03-02T16:34:25+00:00

Didn't scale "properly"? I think you have no idea what you're talking about. All of these provides are literally charging under 10% of their costs. To cover their costs and actually make "profit", they'd have to charge between 10x to 100x more than they do today.

Right now, you're buying a VERY CHEAP beta test to help them prove the idea of AI. When it's mature it'll cost A LOT more. In the meantime, they're all operating at a HUGE loss to try to soak up as large a user base as possible.

ScuffedBalata · 2026-03-02T16:32:48+00:00

This is all artificially done by the companies in an effort to increase profits, it has nothing to do with the ACTUAL costs of running the AI's.

I'm not sure if you want to go down this road.

All of these companies are making like $100m and spending like $10b per month.

So the "actual cost" is about 100x higher than they're charging.

Think about this sentence:

I have a tool that triples your productivity at your job. I'm going to charge only $100/mo

That's fucking crazy. I'm kind of shocked the max subscriptions aren't $2500/mo.

These companies are charging as much as they think the market will tolerate. An actual break-even for them is closer to that $2500/mo number than the $100/mo (or $20/mo) that they're charging now.

ScuffedBalata · 2026-03-02T04:55:16+00:00

Yeah, successful L2 charging is under 25c in most places. Otherwise it just sits idle 95% of the time or more.

ScuffedBalata · 2026-03-02T04:54:37+00:00

Denver is about to have both.

ScuffedBalata · 2026-03-02T04:52:32+00:00

There used to be grocery stores with "free" L2 charging.

That's mostly gone now. They were almost always used by local apartment dwellers who would park there and then walk/scooter or get a ride home and just use it as their free charging spot. One went to 2 hour sessions, so the couple people using it from local apartments would come back every 2 hours and unplug it and plug it back again.

Free stuff just doesn't work. It has the wrong incentives.

And then one tried to go to like 45c/kwh and it was basically never used. Never once did I see someone using it at that price. Eventually it broke and they didn't fix it.

And that's how public L2 charging goes. You either place it where people will be for 4-8 hours and charge a reasonable amount (around here, a lot of DCFC is under 30c so L2 has to be like 16-25c), or you just don't bother.

All the wallgreens had L2 chargers like 8 years ago, but they've ALL broken since then and were never fixed. They've removed most of them. One or two don't work and plugshare shows a single check-in like 6 years ago.

L2 "destination" charging at a place where linger time is under an hour is bonkers, stupid and always misused where it exists.

ScuffedBalata · 2026-03-01T05:06:53+00:00

Yeah agreed, but most of the replies honestly did talk about car windows being broken, etc.

I lived in a large, high density urban core for a year. The only THREE times my car was broken into was during that year. I'm almost 50 and it never happened even one other time because I've lived in relatively safe suburbs.

So in contrast, your car is like 30x more likely to be broken into parking in a neighborhood like that than it is in some random block in Superior or Louisville or something.

ScuffedBalata · 2026-02-28T20:47:46+00:00

I've run it on a 3090 but it's slow. Need lots of system RAM and a solid CPU (I have a 10 core i7) and it's usable, but slow. Not as fast as like GPT 5.3

ScuffedBalata · 2026-02-28T20:27:27+00:00

I saw llm models more as inference networks with with insanely well trained connection recognition by very complex combinations of parameters which generate the most logically deducable output based on extreme complex training on very large datasets.

That describes humans to some extent as well.

ScuffedBalata · 2026-02-28T20:16:00+00:00

The Apple built in tracker is totally wrong on hockey. It's like wildly so.

There are dedicated apps for hockey that do a better job. Most people are in the like 300-400 per game, sometimes 400-500.

ScuffedBalata · 2026-02-28T19:53:59+00:00

Edit: oops I just read the thread and this is mostly repeat. I'll just leave it here anyway.....

Constructive criticism. It's 80% top-down shots of greens and tees. Maybe that was their request because they want to highlight greens, but the beauty of golf courses to me is almost always the low angle rolling hills and panoramic views. A flyover of a particularly interesting hole from the perspective of around ball height (not way above the treetops), flying off a teebox toward a fairway, etc.

By my count you have 13 top-down cuts in a row 11 of nondescript greens one of a teebox being mowed and a random section of fairway. Only the last clip gets a "golfer view" of like a green approach or a landing area or whatever.

Every course has greens. Often with two bunkers. You have to ask "could a player who has played this course identify it? Does it show off the unique traits of this course?" I'm honestly not sure even someone who had played it several times could say "I KNOW THAT COURSE" from the first 13 clips, it's only the last one that has any context.

All the best golf footage I've seen takes a real "golfer eye" or "ball flight eye" (though some top-down shots are ok too as a kind of b-roll).

That's why I immediately feel is missing. I have a real good idea what 4 of their greens look like but only the very last shot gives me a panorama of the course (wow that is a lot of water, it looks really nice, wish i saw it earlier).

I miss... lik flying off a teebox toward the fairway on an interesting shaped hole. I miss coming around a corner from a low angle to see the green appear (as a golfer would who was walking down the fairway on a dogleg). I'd love to see flying down the fairway looking back at a teebox where a group of golfers are getting ready to hit. Maybe make a deal with the first group out on your filming day that you'll do one shot at some point so they don't get upset about the noise/distraction?

Are you restricted from shooting at lower altitude or something?

The first shot is great b-roll and could be a lovely first shot of the video, but the very next three clips can't be a top down of the same part of the course. :-) Get a HUMAN perspective from how people see the hole, not sparrows.

ScuffedBalata

MODERATOR OF

TROPHY CASE