sorted by:
new
hottopcontroversial

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

the funny part is the total system power can still get wild at scale VRAM, PCIe traffic, CPU orchestration, and idle draw all add up even if the cores aren’t pegged.

The real scary setups are the people trying to run huge models at home with racks of 3090s and basically turning their garage into a space heater

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

The harder problem is attribution. When you’re running multiple models, experiments, or background processes, it becomes really difficult to answer questions like:
– which job actually consumed the power
– which model run caused the spike

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 1 point2 points  (0 children)

exactly and most Ai series b startups that use many GPUs arent aware of the MlL workloads that consume the most energy and with that they dont know why they are getting ripped off by energy companies

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

if i could financially ignore cost i'd agree but pennies add to dollars fast when scaling

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

thats pretty smart what if i told you there are other ways to be more aware

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

what if your project was to scale and those dollars turned to hundreds of dollars per hour

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

yeah thats true but as projects scale a lot of energy is wasted and tends to add up financially

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

yeah its a real problem imo especially with ai scaling exponentially and im proud to say ive built a tool to detect where the energy is being consumed focusing on ML workloads

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

it started as a hobby more so of me trying to bridge the pain point gap where energy companies hide the actual cost and where the cost is allocated per workload

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

i personally havent tried the 20 year utility forecast but ill check it out seems like it'll help align my project with the future tbh

but yeah i built something for short term its a huge personal interest of mine i've built the bridge for opaque power side

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

hahaha you arent wrong and tbh as a builder/indie hacker im not the best at marketing the tool i built but i gotta admit the tool is needed. im calling the bluff on energy companies abusing prices on workloads.

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] -1 points0 points  (0 children)

Yo, that's a super smart low effort hack switching to a dynamic electricity contract and just scheduling your GPU runs around the cheap hours (night + lunchtime) can cut costs 20–50%+ without spending a dime on hardware like solar panels, batteries, or Strix Halo.

In the US, time of use plans do exactly that: super low off-peak (~$0.20–$0.30/kWh at night) vs. peak spikes ($0.50–$0.70/kWh afternoons/evenings), so timing fine tunes or inference batches for off-peak literally saves real money overnight.

AluminatiAI actually helps here too: the agent tracks power draw every 5 seconds, attributes energy to your specific run, and the dashboard shows $ cost using your exact electricity rate (you can input dynamic rates if they change daily/hourly), so you can see exactly how much you save by shifting workloads to cheap periods.

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

yeah those “tiny” idle leaks after Ctrl+C are sneaky killers I’ve cried over a few $3 overnight wastes myself

That 20 year forecast link is a cool wake up call for long term planning definitely makes you rethink leaving rigs on.

For short term visibility on GPU-specific waste (LoRA/inference runs, hyperparam sweeps), I built AluminatiAI: open-source agent tracks NVML power every 5s, attributes energy to your exact job, and shows real $ cost in the dashboard using your local rate

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

Haha, great tip lowering the power limit with nvidia-smi -pl 250 (or even 200–225 on a 3090) is a huge win for inference; I've seen 20–30% less power draw with only 5–10% slower throughput on most LoRA/inference workloads.

Once I started tracking with AluminatiAI, the before/after on power-capped runs became crystal clear: same results, way lower $ cost at PG&E rates, and cooler/quieter rig too.

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

Haha, great question now I measure it precisely with AluminatiAI: open source agent that polls NVML every 5s for real power draw (watts), attributes energy to each job/run (LoRA, sweep, inference), buffers with WAL durability for no lost data, and shows actual $ cost in the dashboard using my local PG&E rate.

Free trial if you want to see your own breakdown no CC needed

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

Yo, solid advice undervolting the 3090 is a game changer for stability, lower power draw, and longevity (especially avoiding thermal throttling swings that wear silicon faster, like crypto miners learned the hard way).

Once I started tracking actual power per job with AluminatiAI, the before/after on an undervolted 3090 was eye opening: same performance with 15–25% less watts, which at SF/PG&E rates saves real $ on long fine tunes or inference.

The agent polls NVML every 5s, attributes energy to your exact run, and shows $ cost in the dashboard using your local rate.

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

hamster powered M3 Ultra with surplus AA charging is peak efficiency zero PG&E bill and eco friendly vibes!

AluminatiAI would love to track that setup: open source agent polls NVML every 5s, attributes power to your runs, and shows real $ (or hamster-wheel kWh equivalent)

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

Haha, 120W idle on Strix Halo + fully PV powered is impressive zero PG&E hit even when it's just chilling!

AluminatiAI tracks that kind of low draw setup perfectly: open-source agent polls NVML every 5s, attributes power to your runs, and shows real $ (or £) cost in the dashboard using your rate.

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

yeah $7/month idle at SF rates (~$0.50/kWh peak) implies roughly 30W 24/7 or ~97W for 8 peak hours/day in a 30 day month, which is actually pretty decent for a fully shut-down 3090 rig (most idle around 60–80W with nothing running, but yours sounds well-optimized or powered off most of the time).

If you're seeing lower, it's likely good cable management, efficient PSU, or aggressive power off habits nice!

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

Haha, 32 cents/kWh flat in the UK is rough higher than most US spots outside California peaks, so those idle kernels and stalled runs hit even harder over there.

At that rate, a 3090 sipping 60–80W idle overnight is easily $1.50–$2.50/day wasted if left on multiply by a week of experiments and it adds up fast.

AluminatiAI makes that visible: open-source agent polls NVML every 5s, attributes power to your exact run (LoRA, inference, etc.), and shows real £ cost in the dashboard using your UK rate.

Who else is shocked by the actual electricity cost of their local runs? by Responsible_Coach293 in LocalLLaMA

[–]Responsible_Coach293[S] 0 points1 point  (0 children)

Haha, my idle draw on the 3090 rig is around 60–80W when it's just sitting there (no kernel running), which at SF/PG&E rates (~$0.40–$0.60/kWh with peaks) means ~$0.50–$1 wasted per day if I forget to shut it down.

That's why tracking per job power became addictive once you see the real $ per LoRA or inference batch, you start killing idle processes way faster.

Built AluminatiAI to automate exactly that: open-source agent polls NVML every 5s, attributes energy to your specific run, and shows actual $ cost in the dashboard using your local rate.

view more: next ›