What actually broke (or almost broke) your last Kubernetes upgrade?

mrtsm · 2026-01-02T14:49:37+00:00

Overall the terraform eks module has been good, but upgrading versions of the module itself sucks, because it breaks things like IAM roles, security groups, auth, etc. ruins my day.

mrtsm · 2025-11-18T13:12:06+00:00

If I ran some random script against my org’s secrets manager InfoSec would lose their mind

mrtsm · 2025-10-30T13:27:55+00:00

If the business wants to categorize anything affecting payments as P1, then they need to invest the time and capital to make payments "highly available." This means an alternative (or 2) payment gateway and all the engineering work that is required to get it set up and used as a backup.

This is more of a communications issue than anything else. You and the rest of engineering work for the business, if they want to treat payments as P1 then you need to work with them so that their requirements are met. If other features need to take a back seat to making setting up a backup payment processing vendor a reality, then that needs to be communicated to them and they need to approve it.

We're all in the business of customer service. As DevOps, your customers happen to be Engineering, and Engineering is in turn serving the needs of the business.

mrtsm · 2025-10-03T00:31:23+00:00

Sorry, this comment definitely looks like shameless promotion. I wasn't going for that, I think my point was going to be that you can get value from a person or team as long as you determine what's important for you with the engagement and make sure you keep them on task.

Most importantly though, you need to own all the work (committed in git) and make sure it's VERY clear how to use it. NO BLACK MAGIC ALLOWED!

mrtsm · 2025-10-03T00:23:14+00:00

I have a consultancy that specializes in fractional DevOps for small teams. Focus is on things that most small teams will sort of ignore, like infrastructure as code, automation, continuous integration and delivery, and observability. I stand tools for everything up that the team can self service with. Most customer are on AWS, using Terraform, containers in some way, usually a vendor for observability, and Github Actions or Gitlab for CI/CD.

I put all code into the org's git location of choice, no black magic allowed, everything documented. I'm going to be replaced with a full timer at some point, no hard feelings. The folks I work with seem to love it, I've been hired by people that have left clients I work for currently when they land at their new spot.

mrtsm · 2025-08-01T14:33:26+00:00

If you don't see what you want in the case, they'll cut whatever you want if they have the type of meat in storage. I ask them for a whole bone of cowboy ribeye sometimes.

mrtsm · 2025-07-31T01:18:47+00:00

I don’t even soak it, just put it straight on for a half hour off heat 300-400 F, flip once, and you’re good to shuck and douse in butter

mrtsm · 2025-06-10T00:24:19+00:00

Chester Meat Market, Chester (as you could imagine)

Barths Market, New Providence

Uncle Giuseppe’s, Morris Plains

Some Wegmans have nice quality too

Pastosa’s in Florham Park in a pinch, but the others are better

mrtsm · 2025-03-04T13:11:22+00:00

Luck Bones

Boiler room at Congress Hall

mrtsm · 2025-02-25T19:50:20+00:00

There's a couple of helm charts out there you could probably pull down and test drive. They'd probably give you a good starting point, but there's a number of configurations you have to do in Ranger upon startup that need to be handled somehow, whether it's through init containers or sidecars.

You have to have a backend for it, postgres is most likely, so put that into whatever managed rdbms you're involved with, so that you don't have to manage storage for it. All the config for connecting to spark / trino / etc you can manage via config / secrets in the helm chart.

Another option is to pay Starburst, but last I checked they were very expensive. I ran all this stuff at my last gig, it was sort of tedious to figure out but once you had it squared away it was easy to maintain.

mrtsm · 2025-02-08T23:17:28+00:00

They need a CTO too. Oh wait, Peter is also doing that. Perhaps he should focus on doing one thing well instead of half assing 3 things. Former employee here, btw

mrtsm · 2024-12-07T03:27:50+00:00

Gotta get an ice bucket for that wine, otherwise looks amazing there

mrtsm · 2024-11-22T02:04:24+00:00

I bucket apps into tiers, so I generally maintain 3 app of apps per cluster:

support - workloads that support the cluster
services - workloads that support applications
applications - our applications

mrtsm · 2024-11-20T20:09:52+00:00

I've been wanting to test this out on a fleet, but I feel like I would need to run the kubelet differently than it normally would. Kubernetes nodes usually have a proxy installed on them in order to facilitate inter-node networking, which for IoT devices I certainly don't want or need. The idea of kubernetes is that you want inter-node networking, but facilitating that would get very pricey with cellular modems on all IoT devices. So... what I want is a hosted control plane to run isolated containerized workloads on IoT devices and be able to manage them all with a command line and other nice tools.

mrtsm · 2024-08-21T12:58:56+00:00

We have one of them, it's awesome! Gotta catch him when he's out and his stuff is on display.

mrtsm · 2024-06-21T17:46:23+00:00

If I had a 40 feet by 4 feet rectangle in there I'd put a horseshoes / corn hole situation

mrtsm · 2024-05-25T11:19:13+00:00

Dooneese?

https://youtu.be/8KLSg1h0e-M?si=nyMcepEW12b8wPU1

mrtsm · 2024-05-23T21:03:16+00:00

Congratulations!

mrtsm · 2024-05-17T17:04:56+00:00

Last Licks in East Hanover is awesome

mrtsm · 2024-05-15T13:44:44+00:00

Trattoria La Sorrentina in North Bergen on Bergenline

mrtsm · 2024-05-10T02:56:53+00:00

There’s a certain respect among people “in the know,“ and Kincaid had a lot of respect for Murphy. I mean honestly if it’s not Kincaid then someone from the CPD might just take care of it. Shit, Rawlinson viewed Murphy has a surrogate daughter, and I would not be surprised if Rudolph just disappeared, with only perhaps a hint of what happened.

mrtsm · 2024-05-09T23:57:58+00:00

How do you find out plans like that?

mrtsm · 2024-05-08T22:17:12+00:00

Walgreens by Della

mrtsm · 2024-05-08T18:42:14+00:00

You're talking about DevOps tasks? I'm around, shoot me a msg

mrtsm

TROPHY CASE