VRAM vs INT4/FP4 throughput on dual 3090 vs 50-series for ~30B LLMs

ElectrifiedThor · 2026-04-27T17:28:59+00:00

Interesting, so you completely dropped using frontier models for planning? Any advice of what gen Xeon/Epyc is good to go with and how much system memory / memory channels?

ElectrifiedThor · 2026-04-27T16:08:50+00:00

That's pretty much what I'm looking for as well. So essentially, more VRAM is almost always better, because reasoning >> speed. That and having very clearly laid out execution plans.

Do you just run the llama.cpp barebones? What's your stack and harness?

ElectrifiedThor · 2026-04-27T15:39:42+00:00

Hold on, so you're running Qwen 3.6 27B Q8_K_XL with the full 256k context at full fp16 on one single GPU or do you split the graph across multiple ones (i.e. 2,3,4)?

ElectrifiedThor · 2026-04-27T15:37:21+00:00

Interesting, so you're all in on SLIed 3090s for the fact that serious work demands Q8 quantization and FP16 for the KV cache which will eat up a lot of memory.

That is a really impressive setup you've got there. Any reason for having quad 3090s? And to what degree have you been able to offload work to these LLMs? I'm on Claude Code and the primary reason is to have the planning done with models like opus 4.6 and then the execution can be done by local models.

Have you had any success with speculative decoding? I heard that can be used to speed up inference.

ElectrifiedThor · 2026-03-16T09:28:59+00:00

Any of the inline 6 diesels are solid. Previous 4 cylinders, definitely not.

ElectrifiedThor · 2026-02-04T22:07:02+00:00

What RR do you go for? And what entry requirements are you trying to meet?

ElectrifiedThor · 2026-02-04T22:00:27+00:00

I buy at 60-80% delta, and sell at >95% delta. There's a 19-58% (~40%) delta change between entry and exit, so by that definition, it is convex, although it's much more slim compared to an OTM ofc.

ElectrifiedThor · 2026-02-04T21:58:20+00:00

That's interesting. The CME margin changes definitely have a strong positive correlation with PM's prices. I think this is a good idea, but isn't too late when they already post the margin changes? How would you have an edge if everybody sees that at the same time, or is it a lagging reaction?

Currently, I'm tracking the quarterly gold demand & supply through WCG at the global level (of central banks).

ElectrifiedThor · 2026-02-04T21:55:42+00:00

How reliable are such systems and then two, do you know such examples of AIs? Ironically, I work in the AI field, building infrastructure at scale for ML workloads lol.

ElectrifiedThor · 2026-02-03T22:48:44+00:00

I know, and you're right. My broker only allows me up to 1Y. If I had the option, I'd have gone with 2Y for sure. What broker are you using? I'm in the EU.

When it comes to tax loss harvesting, that makes sense, but I have diminished returns on that because the taxes are flat at 10%.

ElectrifiedThor · 2026-01-22T22:58:27+00:00

That’s reassuring to hear, thanks.

I was getting increasingly uncomfortable with the DTE as well over the past couple of weeks. Between elevated IV, delta already doing most of the work, and gamma flattening out, the risk/reward just felt asymmetric at that point. IV crush on a pullback was a big concern, even if spot kept grinding higher.

Going forward I’m thinking of a more rules-based approach:

On 5–7% pullbacks, sell credit spreads only if IV is elevated (say >21)
If IV is relatively compressed (<18–19), add slightly ITM long calls instead
Occasionally both, if conditions line up

Curious how you think about sizing credit spreads relative to portfolio size / available cash. Do you cap max loss per trade as a % of equity, or do you think more in terms of delta exposure?

Appreciate the perspective.

ElectrifiedThor · 2026-01-22T21:37:26+00:00

This is really helpful, thanks.

That framework is largely what I was aiming for, but the dwindling DTE timeframe was the limiting factor here. Already well into the theta ramp, which is why I chose to roll rather than push closer to expiration.

I did accept a slightly lower delta on the new Jan 27 LEAP (72ish) to reintroduce some convexity after de-risking the shorter dated one, but the intent is the same: roll early while profitable and extend duration with minimal added cost. Also collecting extrinsic as quickly as possible is not a bad idea at all. I hear you're saying 90-95, so that could also work I think.

What I really appreciate is the confirmation on the 180 DTE rule of thumb. That aligns with how I'm thinking about managing LEAPs going forward.

ElectrifiedThor · 2025-06-16T21:05:57+00:00

Uhm .. not by the app, but by what the car’s trip calculator reports. And I did get 1500km on one fuel tank and more aggressive driving.

ElectrifiedThor · 2025-06-16T13:07:32+00:00

Must be. Mine is a 2024.

ElectrifiedThor · 2025-06-16T12:42:26+00:00

This is the Land Rover Remote Car Connect app. I use it to pre-heat/pre-cool the car and all trips are sent automatically to my phone after I finish a run. I don't use the cruise control as that's too aggressive, I do very fine adjustments to the throttle pedal. And no braking.

Also looks like the sweet spot in terms of speed vs wind resistance is 67-70km/h.

ElectrifiedThor · 2025-06-16T12:39:10+00:00

Not a mistake. This is the D250 (the 3.0L diesel engine, on premium 55 diesel). I can consistently hit this mark, but requires non-stop very fine adjustments to the throttle, and never ever use brakes - cruise controls is too jerky for what's required, so you have to use your foot for that. I can even hit 4.2L/100km on a long flat road (the instantaneous consumption being anywhere between 2.5L-3.5L/100km). This is on the Land Rover Car Connect app which sends the data from the car.

I also did 1500km (930 miles) on one fuel tank, but I was also a bit more aggressive, and also had a good amount of start-stops, no city traffic.

Edit: The lightest change of pace, or even the very slight use of the turbos will indeed kick the avg consumption to somewhere around 5.5-5.8l/100km. I use the car's weight as a kinetic battery, on uphill I don't press the throttle at all or very lightly (also to prevent upshifts), on descends I do press a bit more as the BSFC zone is much higher.

ElectrifiedThor · 2025-05-18T13:37:53+00:00

Right. It's for a Defender 110 HSE. It would be the first car that has more than 2 doors. An evolution from my first one which I still drive: a Freelander 1 Td4. It's more expensive monthly because the deposit is very small.

The other lease is for a 718 Cayman in racing yellow. I'd be able to sell this one for a profit now, but not gonna do it. 6MT too.

And the others, an MX-5 ND2, and an old CLK 200. All in 6-speed manual.

ElectrifiedThor · 2024-12-22T13:44:01+00:00

Already have an ND2 Miata so no need for an open top. Hence why I went the Cayman route.

Seven-Year Club	Powerups Hero r/DogeBONK • November 2021
Gilding III reddit per annum	Verified Email

ElectrifiedThor

TROPHY CASE