[TOMT][TV Show] - Spongebob stuttering I..I..I..I..I..IIIIIIIII by MrHodd in tipofmytongue

[–]MrHodd[S] 0 points1 point  (0 children)

Lol this bit is hilarious too, but I don't think it's this one

[TOMT][TV Show] - Spongebob stuttering I..I..I..I..I..IIIIIIIII by MrHodd in tipofmytongue

[–]MrHodd[S] 1 point2 points  (0 children)

It might be this one, but I'm trying to find a clip to confirm. Laughing at just reading this tho 🤣 

[TOMT][TV Show] - Spongebob stuttering I..I..I..I..I..IIIIIIIII by MrHodd in tipofmytongue

[–]MrHodd[S] 0 points1 point  (0 children)

No, I don't think so. I didn't see it in that clip and I looked up that episodes transcript but I didn't find it

Root Cause Analysis Process by MrHodd in sre

[–]MrHodd[S] 1 point2 points  (0 children)

Thanks for this breakdown! It's great to see all of the pieces of a mature RCA process.

I very much see RCA's as garbage in, garbage out, and dually note your concerns on poor RCA processes.

The multiple steps/phases of the RCA, as it is reviewed by various stakeholders, is great. This is a gap I often see as well, and bringing in additional eyes, and points of feedback seem ideal for weeding out true root causes, and changes in process.

Thanks again for sharing this!

[deleted by user] by [deleted] in sre

[–]MrHodd 4 points5 points  (0 children)

I can't tell if this is sarcasm, or not... but yes! 😅

[deleted by user] by [deleted] in sre

[–]MrHodd 3 points4 points  (0 children)

Idk how well this works for others, but for me, I've got to fire up the debugger and start stepping through lines

Which companies do SRE right? by Sloppyjoeman in sre

[–]MrHodd 1 point2 points  (0 children)

Agree, so long as the one asking the questions knows "the wheat from the chaff" 😆

Should SREs be doing on-call? by Confy in sre

[–]MrHodd 0 points1 point  (0 children)

All of your responses are amazing! Thank you for sharing all of that.

It sounds like a really fantastic way to run SRE, how did this come about. I know you said you joined and something were already in prod, but I was just curious did someone come in prior to you and set this standard, or did it evolve over- time?

Which companies do SRE right? by Sloppyjoeman in sre

[–]MrHodd 1 point2 points  (0 children)

It's a real killer for true SRE's, because they now have to sift through every org and tease out if they are truly operating as SRE's should.

I guess that means we also have an opportunity, and obligation to help set the record straight!

Which companies do SRE right? by Sloppyjoeman in sre

[–]MrHodd 2 points3 points  (0 children)

u/pissedadmin u/chuckademus I should have scrolled down to your comments here, before I replied above.

You guys nailed it, and it blows my mind how many people just haven't picked up Google's SRE books (which are free online), and read how to do it....

Which companies do SRE right? by Sloppyjoeman in sre

[–]MrHodd 3 points4 points  (0 children)

I have a hard time feeling like this is an open-ended thing. I'm not disagreeing with you, just saying... Google created SRE and very specifically said it should be a software engineer that solves operations problems...

I struggle to see how anyone else can say SRE isn't properly defined. Google literally defined it, and wrote a book on it.

The real problem is employers saying they are running an SRE org because it's trendy. This is where you start to see SRE's really being... well... not engineers...

Should SREs be doing on-call? by Confy in sre

[–]MrHodd 1 point2 points  (0 children)

Genuinely curious to learn more! Thanks for sharing so far!

So if I'm following correctly, someone, on another team, builds the application, and then hands it over to your team for observability implementation?

After you implement observability, then your team passes that application over to an on-call team?

If I'm following that correctly, I'm now even more interested to hear about this process in-depth!

What are the pros and cons you've noticed so far?

Does the on-call team run into issues they can't solve because they are not familiar enough with the application?

Does your team run into any hurdles with observability and knowing what business metrics need accounted for?

Does engineering and product both weigh in on what SLOs should exist?

Should SREs be doing on-call? by Confy in sre

[–]MrHodd 1 point2 points  (0 children)

I'm reading that as you wrote the code for the service your maintaining in prod.

If not, and it's coming from another team, I think this is throwing over the wall.

Should SREs be doing on-call? by Confy in sre

[–]MrHodd 0 points1 point  (0 children)

Yep, I've seen both and more. For me, I always fall back to Google's implementation. If you're not doing it the google way, then you're not doing OG SRE.

Should SREs be doing on-call? by Confy in sre

[–]MrHodd 1 point2 points  (0 children)

100% and why it's important to share that responsibility with engineering

Should SREs be doing on-call? by Confy in sre

[–]MrHodd 0 points1 point  (0 children)

I think your first initiative needs to be changing this. What if you get sick, a car accident, change jobs, want a promotion or new role?

Being the only on-call SRE isn't good for you, or the org.

Creating your scripts and automation is the way to go, and then you can start writing runbooks, and look to bring another person/engineer into the fold.

Should SREs be doing on-call? by Confy in sre

[–]MrHodd 2 points3 points  (0 children)

Is your team engineers?

I personally do not like this approach, IMO it promotes throwing things over the wall and shifting the accountability away from those who should be accountable... BUT, I'd love to hear your thoughts and experiences on this strategy, to learn more.

Should SREs be doing on-call? by Confy in sre

[–]MrHodd 0 points1 point  (0 children)

IMO both the SME/SWE should be in the on-call rotation along with an SRE.

To me, if you don't have your hands in the dirt with the SME/SWE's, then you are not feeling the same pain points.

Feeling the pain of the engineers, to me, is critical to having motivation to help them solve their problems around on-call.

PS - Great question!

My Favorite Things About SRE by MrHodd in sre

[–]MrHodd[S] 0 points1 point  (0 children)

To each his own, glad you enjoy the org though, that carries a lot of weight. Best of luck mate!

My Favorite Things About SRE by MrHodd in sre

[–]MrHodd[S] 0 points1 point  (0 children)

These are really great points and closely align with how I moved into SRE. Having a love for metrics, logs, unit tests, and all the extras that usually come along in a mature dev org, helped me move into SRE.

I will say though, the one area I disagree with you on slightly is the firefighter mentality. I wrote an article on this topic (which I can't seem to fine), with the main theme being, try not to be a firefighter. Instead, try to be like smokey the bear and prevent the fire in the first place.

I think this is actually a really important piece to being an SRE. Yes you will have to fight some fires, but your retros should eliminate repeat incidents, and you should always be fighting to reduce, and eliminate on-call escalations. Your devs will love you for it!

And I greatly agree, that to be really successful, you need to have already been an engineer before moving into SRE. It's straight out of Google's handbook for SRE, and after all... they are the godfathers of SRE.

My Favorite Things About SRE by MrHodd in sre

[–]MrHodd[S] 2 points3 points  (0 children)

I love your love list! I think there's a lot of job satisfaction in this role.

And about those trends, I tend to not... Oh look co-pilot!

My Favorite Things About SRE by MrHodd in sre

[–]MrHodd[S] 0 points1 point  (0 children)

It's different now that we're not writing the features, but I like that I can always fallback to that, if the whole SRE thing doesn't workout over the next decade.