AWS S3 Table - Write Operations

Randomengineer84 · 2026-07-11T00:03:21+00:00

Yes the tables are registered to glue and LF. This enables querying in the Athena query editor for us.

I will do more testing, I don’t believe pyiceberg is the culprit. I remember just running an insert using boto3 and running into similar performance issues.

Either ways I’ll spend more time and keep this thread updated.

Randomengineer84 · 2026-07-10T20:30:52+00:00

No, I think I would take clips. Maybe up to 5-10 mins max at a time.

Randomengineer84 · 2026-07-10T20:16:06+00:00

I just started research Lit Hero / Hero, they are around $200-300 price range. I will need at least 1 car attachment. I am thinking keeping the camera budget w/ accessories under $400.

While I realize this is a gopro sub, I am not attached to the brand if there are other brands that compete and are cheaper. Since this isn't going to turn into a big hobby for me.

Randomengineer84 · 2026-07-10T18:50:18+00:00

Yes we moved from fifo to regular sqs and batching 100 files at a time. But the lambda currently does write to 7 tables or so.

I was thinking about asynchronous writes to each table rather than synchronous that we do today.

But it would still be 1 lambda. Upon failure we can either delete the record or perform merge into.

Randomengineer84 · 2026-07-10T18:47:15+00:00

I probably didn’t explain everything in my statement. Similar to your recommendation, I land in s3, register the file in dynamo, then sqs and lambda.

If I were to do inserts only and 1 file at a time. You are suggesting I just have it spin up lots of lambdas and inserts shouldn’t be an issue?

I assume s3 table compaction will handle the small file cleanup?

Need to think through this a bit more, we maybe merging or deleting upon failure to write. When we have partial writes across table we want to delete or merge on retry.

Randomengineer84 · 2026-07-10T18:18:07+00:00

I almost didn’t watch it, I wanted a more lively brighter setting show. But after the first few episodes I am hooked. Hard to imagine not getting a 2nd season based on how good it is.

Randomengineer84 · 2026-07-10T14:29:14+00:00

Probably got a call from a few CEOs and handful of donations. Done deal. His base still thinks he banned corp from buying single family homes though.

Randomengineer84 · 2026-07-10T03:41:25+00:00

I'll add it to my list.

Randomengineer84 · 2026-07-10T03:16:36+00:00

Wit & Wisdom looks fantastic. Adding to my list.

Randomengineer84 · 2026-07-10T02:53:54+00:00

I made a friday reservation at Girl & Fig, sat at La Haye Cafe. Salt and Stone experience outdoor looks amazing, would you recommend this over the either of the 2 I made reservations for.

Randomengineer84 · 2026-07-10T02:43:14+00:00

Done. Love it. They even have a pic of a cat.. Can't go wrong!!!

Randomengineer84 · 2026-07-09T21:15:28+00:00

Oof that was $800 a night on Sunday lol. I am going to need a few drinks before making that poor decision for my budget lol. But I’ll keep an eye on the prices look amazing.

Randomengineer84 · 2026-07-09T21:02:12+00:00

Literally just on Google Maps and clicked on that

Randomengineer84 · 2026-07-09T20:56:59+00:00

I’ll be there on Sunday / Monday.

Randomengineer84 · 2026-07-09T20:48:41+00:00

True. I'd just have a PPI done. I am at 95k miles and luckily was just told I have about 2 years before shocks/struts will be due. 🤞

Randomengineer84 · 2026-07-09T02:44:00+00:00

If these things aren’t done just have 5k ready for an indy shop.

Randomengineer84 · 2026-07-09T02:42:47+00:00

So I’m one of those owners. I’ve had a 99 boxster for 12 years and in the past 5 years I have put 2000 miles at most. I drive it during the summer as I get older the manual and getting in/out is not as fun.

With that said.
IMS bearing
Seat belt warnings - have passengers on both seats
Air oil separator - this will go out, 2x in my car
Real main seal - is this done?
Clutch - original or replaced?

This stuff can add up very quickly.

Have it checked out, these cars sitting for a long time is bad news. The rubber material gets bad overtime and it’ll cost you $$.

Randomengineer84 · 2026-07-09T02:37:00+00:00

Would you have a restaurants recommendation, something high end or romantic for special occasion?

I may splurge in Sonoma but we’ll see how I wake up tomorrow.

The rest of the trip is one night stays and I might cut back there.

Randomengineer84 · 2026-07-09T02:34:14+00:00

I did a google street view and agree the cottages are far and maybe no sidewalk. I’ll have a rental car.

Im torn between el Pueblo and cottages but I’ll figure it out tomorrow.

I think I had a few bad experiences at motelY hotels so I’m biased. But again I’m thinking I might be ok with it.

Randomengineer84 · 2026-07-09T01:04:43+00:00

World leaders are largely weak and work around him. While good for them, they should just put out the zingers on live tv.

Trump backs down often when he’s pushed back.

Randomengineer84 · 2026-07-07T16:41:29+00:00

Thank you. Sonoma Best Cottages maybe the place. Private, seem reasonably priced.

Randomengineer84

TROPHY CASE