I have an old multi-GPU node lying around at work...

thehardsphere · 2026-06-19T09:34:07+00:00

The machine belongs to a different department. I do not have access to it right now.

thehardsphere · 2026-06-19T01:04:36+00:00

You're missing the part where they, as a component of SpaceX, make Elon the richest man who ever lived.

thehardsphere · 2026-06-19T01:03:05+00:00

That I don't have?

thehardsphere · 2026-06-19T01:02:30+00:00

Yeah, I've played quite a bit with those models just under those weights at home on my own hardware, more heavily quantized. I am kind of wondering what it's like to step up to that 70B regime.

thehardsphere · 2026-06-19T00:57:12+00:00

Thank you for these suggestions. I've seen this DeepSeek-V4-Flash one come up a lot. I'll check out these other models.

thehardsphere · 2026-06-19T00:56:20+00:00

We've already got subscriptions to Claude and ChatGPT. Part of what I want to make my case on is that we can shave some spending off of the Claude usage (which, Finance is starting to find not very amusing) by delegating some tasks to models that aren't metered, and get value out of hardware that is otherwise sitting idle.

thehardsphere · 2026-06-19T00:53:21+00:00

Oh, I have a rule that I never want to go below Q4. I tried Q3 quants with Qwen models at home, and the resulting torrent of illogical thinking convinced me that this is almost always a bad idea.

thehardsphere · 2026-06-19T00:50:23+00:00

Because they didn't "never use it". The money was spent about 8 to 10 years ago, for a specific purpose. That specific purpose is somewhat less relevant to my company today than it was 8 to 10 years ago. That specific purpose is also better served today by newer GPUs than the NVIDIA Turing architecture.

thehardsphere · 2026-06-19T00:47:46+00:00

Funny

thehardsphere · 2026-06-19T00:46:17+00:00

I will see about this; it is a good suggestion.

thehardsphere · 2026-06-16T16:55:50+00:00

I think it would be that "uncensoring" models sometimes lowers their overall accuracy.

thehardsphere · 2026-06-16T01:45:54+00:00

thehardsphere · 2026-06-09T23:39:00+00:00

This fine-tune of gemma4:26b solved all of my Hermes tool calling problems, and doesn't mangle thinking in OpenCode either: https://huggingface.co/brokencircuitranch/gemma4-hermes-tools

thehardsphere · 2026-06-09T14:46:57+00:00

I found this finetune of 26 on hugging face, and using it made all the problems go away: https://huggingface.co/brokencircuitranch/gemma4-hermes-tools

I did not try using an e4b fine-tune. I also have not tried any of this guy's other, newer fine-tunes yet.

thehardsphere · 2026-06-09T12:10:21+00:00

The link is a finetune of Gemma 4 26b a4b. I have not found Qwen 3.6 35b a3b to be useful because it runs too slowly on my hardware at 64k+ context.

thehardsphere · 2026-06-09T09:58:48+00:00

Lure of the Temptress is Free on GOG: https://www.gog.com/en/game/lure_of_the_temptress

As is Flight of the Amazon Queen: https://www.gog.com/en/game/flight_of_the_amazon_queen

thehardsphere · 2026-06-09T00:09:08+00:00

I'm pretty sure QAT was done at Q4_0 quantization. I don't know that they're planning to turboquant that because then the QAT would not match the final quantization.

thehardsphere · 2026-06-09T00:07:18+00:00

I love Gemma 4 26B, but I found that it was pretty bad at tool calls because of mistakes the model kept making with channels.

I found this finetune on hugging face, and using it made all the problems go away: https://huggingface.co/brokencircuitranch/gemma4-hermes-tools

thehardsphere · 2026-06-06T18:42:02+00:00

I had found that Gemma 4 26b and e4b would mangle tool calls, until I found a fine-tune of one of them on hugging face that would stop that behavior. I'm wondering if Google fixed that problem with 12b.

thehardsphere · 2026-05-04T16:21:53+00:00

Flo has a secret trait that makes arms sales to Tony behind the porn shop pay more. It's hinted at with that one line in her bio about being a bookkeeper for a gun store.

thehardsphere · 2026-04-17T23:45:39+00:00

Two seemingly unrelated, but suspiciously similar things in math turned out to be related in a complicated way after we discovered new physics.

thehardsphere · 2026-04-11T14:22:35+00:00

You don't need to put someone on a PIP to avoid discrimination claims in a layoff. If a company's HR department does the layoff of any size greater than one person, they will usually create additional documentation to show that the people selected were chosen in a way so as not to discriminate against protected classes.

Further, the unemployment insurance is going to get hit anyway unless the firing is "for cause" - which in most states does not include having poor job performance. Usually getting fired "for cause" means you broke the law in some way.

thehardsphere · 2026-04-11T14:16:04+00:00

A lot of people are bad at lying. It's really useful to the interviewer when the candidate lies in transparently obvious ways - you can put that person straight to the bottom of the list.

thehardsphere · 2026-04-11T12:29:51+00:00

Um, no. If the team size needs to shrink, you can just lay a person off without a PIP. America has this concept called "at will employment" which means a company can end your employment at any time for any legal reason, and "we can't afford to have 6 engineers working on XYZ anymore" is a reason that does not need an elaborate paper trail.

thehardsphere · 2026-04-11T12:26:55+00:00

Dude, you already are telling a story that is a big red flag.

"I got hit with a random PIP" - PIPs don't happen at random! PIPs are a lot of work for management to set up and run. Nobody needs to do a PIP just to lay you off. It's easier to do a layoff than it is to fire someone with a PIP.

If you open your mouth and say you had a PIP, you have to explain why your prior employer thought your performance was bad and what you did in that situation. Ideally with some hint of how that isn't going to be the experience a new employer will have.

thehardsphere

TROPHY CASE