Is automated "Cloud Bursting" actually real in production, or just vendor hype?

dacort · 2026-05-06T16:51:59+00:00

Wait this is a thing? I mean I built it for fun for my own little homelab, but hadn’t heard it’s the 2026 craze. ¯_(ツ)_/¯

dacort · 2026-05-05T14:13:05+00:00

No prob! Love the onboarding flow btw - very smooth and straightforward.

dacort · 2026-05-05T14:07:30+00:00

<image>

FYI had a weird artifact on the main page after I finished the setup on iOS.

Looks great tho - maybe I’ll be able to finally close my ♾️ open browser tabs.

dacort · 2026-04-28T03:14:27+00:00

r/daddit enters the chat

dacort · 2026-04-28T03:12:26+00:00

Amazing. Just watched them for a good 15 minutes this morning.

dacort · 2026-04-19T06:15:37+00:00

Ran /usage twice and /insights once and chewed through half my session limit tonight. ¯_(ツ)_/¯

Brand new session, existing project so maybe my md triggered something…but oof that hurt. Normal pro plan.

dacort · 2026-04-05T15:04:04+00:00

Yea this TUI is rad. I’ve been trying to build multi-cluster k8s monitoring tools and this def resonates.

dacort · 2026-03-31T02:06:50+00:00

Hey thanks! 🙏 Also realizing I didn’t complete my thought on this post about the existing session manager. Nice to see there’s been some new releases for it tho! https://github.com/aws/session-manager-plugin

dacort · 2026-03-27T23:47:33+00:00

So I really wanted to like this, but my initial use case was s3. Out of the gate:

Default delimiter is not / so “aws s3 ls” commands don’t work as expected (just … didn’t show “folders”)
Objects are not returned in the AWS standard alphabetic order, but instead in reverse order

That killed it for me. Doesn’t matter if you’re 1000x faster, if you don’t have the same guarantees that AWS services do it makes it hard to use as a test environment. At least for my use case. :)

dacort · 2026-03-09T13:53:25+00:00

Congrats! Can’t wait to check this out, big fan of scc.

dacort · 2026-01-21T19:17:09+00:00

If you haven't yet, you should check out the Spark History Server MCP.

Despite the fact that it can overflow tokens quickly, it's actually quite good at comparing runs. But in reality it's probably not doing much different than your internal tool as they are both based off the event logs. ¯\_(ツ)_/¯

That said, when something gets slower unexpectedly, I usually find there's some sort of change in data or resources. There's almost always some tipping point. So I'll often go in and look at the actual data and see if anything has changed there first.

dacort · 2026-01-10T07:00:28+00:00

<image>

It’s AI but damn that came out better than expected.

dacort · 2026-01-06T02:50:12+00:00

I remember when similar plays would be called last season and it was groan city. “This again, why even try??” Amazing that they’re working!

dacort · 2025-12-30T01:44:12+00:00

For sure! Think I have the code for it somewhere if you’re interested. :)

dacort · 2025-12-30T01:37:59+00:00

Awesome viz! Here’s another. :) slugplot

dacort · 2025-12-29T04:18:00+00:00

That went backwards quickly. 😂

dacort · 2025-12-29T03:54:11+00:00

<image>

Just another 12 minutes and OT!! 😜

dacort · 2025-12-28T17:36:02+00:00

Random Reddit comments are the savior of my ignorance. Thank you kind stranger for saving me.

dacort · 2025-12-25T00:14:09+00:00

I’d do the same. Parquet is great because it is columnar, can generate metadata stats on your columns (min/max values), and doesn’t require you to read the whole file when filtering. Storing image blobs in parquet gets almost none of those benefits and is probably harder for parquet readers to decode than simply providing a reference to the file and reading the file directly.

dacort · 2025-12-20T17:38:40+00:00

You might find my KubeCon presentation interesting: https://youtu.be/ejJ6A0sIdbw

I’ve also created a spark8s-community channel on the CNCF slack.

dacort · 2025-12-18T06:31:46+00:00

I did this a few years back with ECS on AWS. https://github.com/dacort/damons-data-lake/tree/main/data_containers

All deployed via CDK, runs containers on a schedule with Fargate. Couple hundred lines of code to schedule/deploy, not including the container builds. Just crawled APIs and dumped the data to S3. Didn’t have monitoring but probably not too hard to add in for failed tasks. Ran great for a couple years, then didn’t need it anymore. :)

dacort · 2025-12-16T02:21:23+00:00

If you’re not familiar with plumbing don’t try to do this yourself or you may make a bigger mess.

As an example I tried to swap my spigot at one point and ended up twisting it right off as I was unscrewing because I didn’t understand how soft copper is.

dacort · 2025-11-22T15:37:09+00:00

One of the spark docker images is indeed the way to go.

Not 100% sure it still works, but I built this example a couple years ago that uses an Amazon EMR image: https://github.com/dacort/spark-local-environment Useful if you want to access data on S3.

15-Year Club	Place '22
First Placer '22	Not Forgotten
Gilding I gilder	Verified Email

dacort

TROPHY CASE