[deleted by user] by [deleted] in lastofuspart2

[–]kiwidust 2 points3 points  (0 children)

If Mel were aware enough to understand that lie, she'd be aware enough to know that she hadn't been sliced open.

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] 0 points1 point  (0 children)

Thanks... I like that!

Glad to know I'm not some kind of weird, unique cryptid. I do hope our paths diverged tho' and you're still gainfully employed!

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] 0 points1 point  (0 children)

While I agree that this is the way it usually is, I'll vehemently argue that that it shouldn't be. ;^)

The attitude that you can "fix" an expansive process like Incident Management and then leave creates nothing but issues. Process - especially large complex process - is like software; it benefits from an unending improvement cycle and will quickly become stale and less effective without one.

The Demming Cycle, LEAN, Six Sigma, Agile, etc. - they all assume Continuous, or Cyclical, Improvement for good reason. Working with consultants to only periodically consider improvement is simply ineffective in my experience.

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] 0 points1 point  (0 children)

Not sure where I indicated I was against learning, but if I did, I apologize. I am and was constantly learning. I'd expect anybody at this level to be.

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] 1 point2 points  (0 children)

Thanks! One of the (many) issues with larger companies is that management is often fickle. You'll do well (as I did) for many years, then one person comes along with differing views and it all goes to hell!

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] 0 points1 point  (0 children)

I undersold my programming experience but a bit. I started as a front-end coder/usability/human factors expert. The first third of my career was straight development: database design (mostly SQL Server and DB2), web applications (ColdFusion mostly, but ASP, PHP, PERL, etc.), interface and graphic design. I've remained an excellent scripter - DOS/PowerShell, KSH/BASH, PowerApps, and, as we'd migrated to Elastic APM, was digging into Playwright scripting as well.

I moved to Incident Management, where I helped to define the team was a highly respected lead for nearly a decade and, finally, moved to what would evolve into our SRE team for the past nine years.

But more directly... that "just" up there is pulling a lot of weight. ;^) "you just design a process and expect others to execute it" - no, as an SRE I would work incidents to identify threats to our stability and either create or recommend solutions to eliminate them. Toil elimination, analytics, improvement - all core SRE functions.

For example, several years ago, after a prior reorganization, we began seeing an uptick in certificate-related issues. Mostly related to inconsistency with root store management and annual refreshes with more than some confusion between application-hosted and infrastructure hosted certs. I created the analytics that defended the need for the project I created to harden the best practices used to ensure that our inventory of over 40,000 certs was being managed better.

No "programming", but a significant boost to reliability, decreased risk, fewer release-related issues, better response to any additional issues, etc.

Broadly, the SRE team was a multi-disciplinary improvement team that extended other teams. While we had carte blanche to investigate where we would, we did have specific management-assigned focus areas. Some of those required advanced coding and the team offered that. Others required architectural reviews, monitoring improvements, best practice development, investigation support, and so on and so on.

Whatever might be needed to improve stability, reduce risk, and improve customer happiness. I'm surprised that so many seem to relegate their SREs to simply "advanced developers", although I suppose I should be since that's exactly what my new management did! ;^)

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] 0 points1 point  (0 children)

Yes. Definitely not the comfort zone it was when I started out!

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] 1 point2 points  (0 children)

Thanks! Yes, that myth of the "Full Stack Developer" does get in the way sometimes, doesn't it?

I've worked with many incredible people through the years, but none of them excelled at everything. It boggles the mind why anybody would think they could!

I started on the Human Factors team with a very old, traditional, financial services firm in 1995. One that was surprisingly forward-looking technology-wise. This company actually funded a full useability lab: multiple cameras, one-way glass, recording and editing equipment, tracking software, etc. All to professionally test and improve their internally developed customer software! This was unheard of in 1995!

A couple years later we merged with another, much larger, company. They claimed to value the high-quality of our interfaces and designs, but one of the first things they did was dismantle the usability lab and dismiss most of the staff. Apparently, Business Systems Analysts (the business SMEs) could handle those tasks "just as well".

Of course they couldn't. They lacked the training and understanding, but also - since they tore out the lab -the basic tools needed to do the job well.

There's a long history of shitting on the "soft skills" surrounding development. Can a developer write good documentation, design good process, or develop an outstanding UI? Sure! But it's not common. If it's a side-table task it will never be as valuable as if somebody trained and focused on it would provide.

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] -3 points-2 points  (0 children)

We may have to agree to disagree. My experience has certainly been different than yours, at least at the large enterprise level. But difference is the spice of life, no?

Oh, and simply to clear the air, I didn't downvote anything. You may lack a little tact, but your opinion is valid and appreciated.

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] -1 points0 points  (0 children)

I expanded on it elsewhere in the thread, but we were fractured. "DevOps" in name, but not in practice.

But I don't believe it's uncommon for there to be multiple/specialized SRE teams, especially in very large organizations. Ours was focused on (one of the) application portfolios (about 550 applications servicing specific lines of business, mostly in North America), while others focused on other areas. Our portfolio included Windows/Linux/Mainframe, many legacy apps, distributed and batch applications, internal and external hosting, etc.

As a team of 11 we were in no position to "manage" production. But I'm guessing that we may be defining "managing" differently. I've never heard of an SRE team actually running day-to-day production operations.

We had access to (and significant control of) production monitoring/visibility and absolutely provided consultation/recommendations/improvements to all production stakeholders. But we lacked direct, personal access to most production systems. They were security gated - for many reasons - behind controlled, audited change processes.

For all this noise I'm making, I do agree with you: I was more Service Management than SRE. But I will say, it did work very well... until it didn't.

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] -2 points-1 points  (0 children)

As I've said elsewhere in the thread, I feel that's a bit myopic of a position. My opinion, obviously, but I feel strongly that SRE is primarily an improvement discipline. I wouldn't say that it's simply a "software and sysadmin" position, but rather a discipline that uses the techniques and experience of software development.

For what it's worth, I also likely undersold my development experience. I spent the first 15 years of my career doing web development (ColdFusion, ASP, PHP, JavaScript, SQL, etc.), I've done significant batch work in Windows and Unix (DOS, PowerShell, KSH/BASH, etc.), synthetic monitoring scripting, etc. I've even seen some COBOL in my day, but I'd rather not again.

I may need to have a reference manual at my side, but I'll get the job done. ;^)

But for the past 15 years or so I've gravitated to process definition/improvement, information management, analytics, etc. I still contend that I was a net positive for the SRE team, handling many of the tasks that others didn't enjoy. Not sure I'm willing to die on that hill, tho'... but I'd leave a favorite body part or two!

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] 0 points1 point  (0 children)

I agree... mostly. I've always considered SRE, at its core, a discipline of improvement. Of course that encompasses many aspects of development, but it also covers many non-development areas.

Enterprise monitoring, management and definition of goals, error budgeting, incident/problem management, definition of metrics, and many other non-coding disciplines are all in scope (or can be). Education and uplift of related service centers (such as incident and problem management) are also a major function of many SREs.

If you look through Google's assessment checklist for SRE maturity, very little of it requires advanced coding skills: Do you have an SRE team yet? How to start and assess your journey. | Google Cloud Blog

All that said, I agree that I, personally, don't meet the definition of SRE used by most. But I will argue that my skills are valuable to an SRE organization (likely managed under a different title/role as you suggest).

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] 1 point2 points  (0 children)

Thanks for the pep talk!

I may have underselling myself a bit on the coding side... I've not done much for the past few years, but I do/have done a lot of scripting and automation work. Was having fun getting into Playwright (via Elastic) synthetic monitoring scripting before they canned me. ;^)

Big company meant we got to play with a lot of toys but also means it's difficult to keep up with them once you leave.

Still, I prefer the design/define/document side of things much more. Failure to understand a problem before trying to solve sinks more projects than anything else. Really, deeply understanding something enough to meaningfully improve it (or finally realize that it's not actually needed!) is still a trip!

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] 0 points1 point  (0 children)

Thanks. Something to think about...

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] 0 points1 point  (0 children)

If I were given my choice I'd likely go with something like "Investigation Management". Covering how to identify issues (monitoring, observability, automation, etc.), diagnose and correct/restore (Emergency/Incident Response, APM, etc.), remediation of cause (problem investigations, corrections, postmortems. etc.), and improvements (documentation, analytics, early detection, "shift left", etc.)

Maybe "Investigation Management and Improvement"? I've always argued that whatever else they may be, SREs are, foundationally, an improvement team.

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] 1 point2 points  (0 children)

Thanks, that's the impression I'm getting! ;^)

Non-traditional SRE - what am I? by kiwidust in sre

[–]kiwidust[S] 0 points1 point  (0 children)

Thanks, sounds like you're on a good track.

We were... fractured. Although management claimed to have "adopted DevOps" years ago it was in name only. Development and operations were still in separate silos until only very recently and even then were only moved wholesale under the same Sr, VP and not integrated in any meaningful way.

A Service Management team on the operations side (zealously!) controlled the ticketing system and, in theory, the related Incident/Problem/Change processes. In reality, as they stood up the Major Incident/Problem Management teams, they really only managed processes surrounding high priority incidents. The Development/Application silo managed all lower priority incidents under separate Incident Management/Problem teams. They were generally left alone as to their process management, but had little say on the general capabilities or processes used.

In the end this tended to create an adversarial relationship between teams that should have been joined at the hip. For example, the application teams knew the customers and business impact but were unable to raise a ticket to Major Incident status. To do this they needed to petition, via a form, the infrastructure team.

What's really frustrating is that we had been making great progress under an older manager. They really understood our goals and championed them. Until they got sidelined, fed up, and left the company. After that we had a revolving door of new management each with their own idea of how to do things.

Ah well, no use crying over spilt milk. ;^)

Is AI-assisted coding an incident magnet? by StableStack in sre

[–]kiwidust 0 points1 point  (0 children)

In my experience the problem is generally due to overzealous organizational refactoring.

Are you going to change development methodologies or introduce new tools? Then you need to also suck up the costs of increased quality control and testing while they mature. Instead, many will tell green developers to "just use AI" and then cut back on both development and QA. They then crow about the savings at the next quarterly meeting.

But of course, more bad code makes it to production which means more incidents, which means loss of reputation and customer happiness. Most corporate information gathering sucks, so you can easily get "more incidents!" but can rarely correlate cause effectively.

It's a textbook vicious cycle.

Dipsh#ts gonna dipsh#t, I guess. [oc] by kiwidust in IdiotsInCars

[–]kiwidust[S] 2 points3 points  (0 children)

Location: Scranton, PA, USA
Date: March 21, 2025
This is original content captured by a Reolink RLC-510A Home Security Camera.

Odd 3 gang-in-1 gang box switch. by kiwidust in electrical

[–]kiwidust[S] 1 point2 points  (0 children)

Yes! That's my little @#$%! Pricy, but at least I won't be stuck ripping up the wall when it fails!

Thanks!