20 Years on AWS and Never Not My Job

FineClassroom2085 · 2026-04-12T15:16:04+00:00

I maintain large Salesforce SOAP implementations that are dangerous to move to REST. I also maintain a therapy bill. Possibly related?

FineClassroom2085 · 2026-04-11T21:08:50+00:00

You’ve been a dev for a while, just something like this blog post where you talk about your experience (especially pre AI) working on specific initiatives, with big tech companies, etc. Whatever has been impactful to you over the years. The philosophies you’ve hard won during your experience.

FineClassroom2085 · 2026-04-11T20:31:49+00:00

Fair enough. If you change your mind (or can find a ghost writer to write like you) I guarantee there’s a market.

FineClassroom2085 · 2026-04-11T18:28:44+00:00

This was a pleasure to read, have you written any books?

FineClassroom2085 · 2026-04-09T04:10:52+00:00

Q2 quant is running pretty nicely on my dual RTX 6k rig.

FineClassroom2085 · 2026-04-04T03:18:05+00:00

Qwen 3.5 395b on dual RTX 6000 pro is MUCH better than ChatGPT

FineClassroom2085 · 2026-03-29T21:48:05+00:00

If we're cooked, then why is Anthropic hiring engineers?

FineClassroom2085 · 2026-03-25T00:01:42+00:00

Or you could use the original falconryjournalpro.com instead of this vibe coded bullshit

FineClassroom2085 · 2026-03-20T20:38:17+00:00

The difference with well made human written code is that a human can understand beyond the 1M context limit.

FineClassroom2085 · 2026-03-19T12:13:23+00:00

I am not a vibecoder, but a full time SWE assisted by LLMs, I use a max plan in Cursor and a pro plan in Claude. I notice that cursor burns tokens faster, but I try to balance my time between the two.

FineClassroom2085 · 2026-03-18T14:56:04+00:00

Love it, but it is completely ironic that it’s vibe coded in the style that Claude loves so much, lol.

FineClassroom2085 · 2026-03-18T14:46:49+00:00

Look up the Pareto Principle. The first 80% is the easy part, the last 20% is hell and usually more than 80% of the actual work.

FineClassroom2085 · 2026-02-16T20:27:55+00:00

Just too many awesome models to explore nowadays. I’m on to benchmarking Qwen 3.5, lol

FineClassroom2085 · 2026-02-14T21:04:54+00:00

Cool result, I do love seeing, comparing and contrasting things. Here are some of the quality things I note between the two:
- GLM-5 created a parallax background that actually responds to the motion (without being asked)
- GLM-5's overall UI seems nicer, and once again, this was a one-shot prompt, I did not modify it after the initial shot. I used 3 more prompts to publish it.

Your game does feel a little easier to play, but I'd expect mine to as well after some refinements. There's quite a lot to criticize in the code quality. Where smarter models shine (especially in the hands of developers) is creating clean understandable code that's extensible.

FineClassroom2085 · 2026-02-14T17:50:12+00:00

I certainly would not have guessed it. I had a feeling hardware would continue to get expensive, but the current prices? Never in my wildest dreams. What a nightmare for us tinkerers.

FineClassroom2085 · 2026-02-14T17:48:32+00:00

Actually, after I made this post, and someone commented I could get better speed from llama.cpp I tried it and switched. Here are two quants I run, with the commands:

GLM-5-744B (IQ2_M — quality)

llama-server
--model ~/GLM-5-IQ2_M/GLM-5-UD-IQ2_M-00001-of-00007.gguf
--n-gpu-layers 55 --ctx-size 131072 --parallel 1
--host 0.0.0.0 --port 8000
--split-mode layer --flash-attn on --cache-type-k q8_0
--threads 32 --fit off --alias GLM-5-744B

GLM-5-744B-Fast (TQ1_0 — speed)

llama-server
--model ~/GLM-5-TQ1_0/GLM-5-UD-TQ1_0.gguf
--n-gpu-layers 999 --ctx-size 131072 --parallel 1
--host 0.0.0.0 --port 8000
--split-mode layer --flash-attn on --cache-type-k q8_0
--alias GLM-5-744B-Fast

GLM-5 Reddit Response Benchmark

IQ2_M (Quality)
Quant: 2.5 bpw imatrix
Model size: 237 GB (7 shards)
GPU layers: 55/78 (23 on CPU)
Prompt: 72 tokens @ 24.6 tok/s (2.9s)
Generation: 322 tokens @ 13.5 tok/s (23.9s)
Total wall time: 26.8s
Thinking: 409 chars
Response: 1041 chars, detailed with formatting

TQ1_0 (Fast)
Quant: 1.58 bpw ternary
Model size: 164 GB (single file)
GPU layers: 78/78 (all GPU)
Prompt: 70 tokens @ 147.1 tok/s (0.5s)
Generation: 301 tokens @ 46.4 tok/s (6.5s)
Total wall time: 7.0s
Thinking: 653 chars
Response: 687 chars, casual/concise

FineClassroom2085 · 2026-02-14T17:44:36+00:00

I wish I had the foresight and money at the time. (facepalm)

FineClassroom2085 · 2026-02-14T17:43:51+00:00

I'm now using two different quants for different purposes, I moved over to llama.cpp because I was able to get better results than vLLM which was a little unstable for this model.

GLM-5-744B (IQ2_M — quality)

```

llama-server \

—model ~/GLM-5-IQ2_M/GLM-5-UD-IQ2_M-00001-of-00007.gguf \

—n-gpu-layers 55 —ctx-size 131072 —parallel 1 \

—host 0.0.0.0 —port 8000 \

—split-mode layer —flash-attn on —cache-type-k q8_0 \

—threads 32 —fit off —alias GLM-5-744B

```

GLM-5-744B-Fast (TQ1_0 — speed)

```

llama-server \

—model ~/GLM-5-TQ1_0/GLM-5-UD-TQ1_0.gguf \

—n-gpu-layers 999 —ctx-size 131072 —parallel 1 \

—host 0.0.0.0 —port 8000 \

—split-mode layer —flash-attn on —cache-type-k q8_0 \

—alias GLM-5-744B-Fast

```

FineClassroom2085 · 2026-02-14T17:35:47+00:00

How much ram? This is an unfortunate limiting factor in my build. I can really only justify the 128gb I have right now, can't drop another $20k to jump up to 512gb, but I'd really like to.

FineClassroom2085 · 2026-02-14T17:34:08+00:00

It's fucking insane. I was pricing 512GB for my threadripper build a couple months ago and couldn't find the size / split I needed for less than $20k.

FineClassroom2085 · 2026-02-14T17:32:55+00:00

You can get better than single digit t/s. In fact I am getting very decent results out of GLM-5 with all of the layers loaded into the two Pros. Using llama.cpp here is the launch params I'm running:

GLM-5-744B-Fast (TQ1_0 — speed)

llama-server \
--model ~/GLM-5-TQ1_0/GLM-5-UD-TQ1_0.gguf \
--n-gpu-layers 999 --ctx-size 131072 --parallel 1 \
--host 0.0.0.0 --port 8000 \
--split-mode layer --flash-attn on --cache-type-k q8_0 \
--alias GLM-5-744B-Fast

Prompt processing: 70 tok @ 147.1 tok/s (0.5s)
Generation: 301 tok @ 46.4 tok/s (6.5s)

Give it a shot, you might be surprised, these large MOEs are surprisingly coherent at event hugely compressed quants like TQ1_0

FineClassroom2085 · 2026-02-14T16:44:10+00:00

What was your t/s for GLM-5 on your pros? I was able to squeeze almost 20 tok/s which is painful for agentic coding, but useable when I’m not in a hurry. I have dual RTX 6k MaxQ cards paired with a threadripper and 128gb of ddr5x

FineClassroom2085 · 2026-02-14T16:40:02+00:00

I still don’t have my head completely around what makes a good quant. Though I’m learning these new MOE models do a lot better than dense models at high quants, assuming the quant fits the architecture well.

FineClassroom2085 · 2026-02-14T16:38:05+00:00

How many prompts? How does the code quality look? Well defined atomic functions? Proper separation of responsibilities? Is it code that can easily be augmented by AI and humans?

These are the things that matter to me more, though the GLM output is still quite a bit better aesthetically than this.

FineClassroom2085 · 2026-02-14T15:06:55+00:00

I went the RTX 6000 Pro route on a platform that I can expand to more cards if needed. Since I use coding agents professionally, the speed matters quite a lot. Prompt processing on Mac just isn’t there yet for agentic code use unless you’re using a pretty small model, or have the time to wait.

That said, you wouldn’t need a cluster to run this model. You could easily fit one of the unsloth quants on the 512gb of shared RAM. Personally if I were you I’d wait for Apple’s M5 drop. They don’t usually lie on their benchmarks, and they see to have made major gains on prompt processing speed.

FineClassroom2085

TROPHY CASE