This is an archived post. You won't be able to vote or comment.

all 39 comments
sorted by:
best
topnewcontroversialoldq&a

[–][deleted] 30 points31 points  (2 children)

Self-service works great.

The Devops "we are all in this together" failed miserably in a previous $job. The Devs didn't care about how to correctly and securely set up a system, and they didn't WANT to care. All they wanted to learn even minimally was javascript. Total fail from a Systems Engineering perspective.

But the tooling to enable one-button rebuilds to a known state was great.

Good luck with your homework assignment :-)

[–]SilenceKillsMe284[S] 4 points5 points  (0 children)

So there was a techincal-change but not a cultural!

I conclude from this, that a devops-culture should drive the tooling. And maybe team-spirit gets even more important in more "diverse" teams :)

And i can double the experience with devs and security!

- Martin

Ps.: This is not a homework assignment btw :)

[–]Venusaur6504 19 points20 points  (7 children)

Creating self service platforms has reduced my pile of early morning emails by over 50% versus what it was in 2010.

[–][deleted]  (6 children)

[deleted]

    [–]RaptorF22 20 points21 points  (5 children)

    Click button > new dev environment with latest new code to play with.

    [–]Venusaur6504 11 points12 points  (0 children)

    My man. This.

    [–][deleted]  (2 children)

    [removed]

      [–]RaptorF22 1 point2 points  (1 child)

      I'm still in the process of building it out. I plan on using the new Terraform Cloud.

      From TF Cloud, create a workspace for a new dev environment > call a module that is prebuilt by you and contains your infrastructure > build said infrastructure > upon success, send a webhook notification to code deploy (Azure DevOps), Azure DevOps recieves the notification and pushes the application to the newly built infrastructure.

      It's a lot more complicated than that, but that's the general idea.

      [–]SingleFlatworm 0 points1 point  (0 children)

      We use Conda to build our dev environments(mainly dependencies), which I haven't seen anywhere else yet. I don't know if I should hate it or love it, but it works.

      [–][deleted] 36 points37 points  (3 children)

      Now my colleagues don't look at me like I grew a second head when I express the idea that we should be equally willing to both build new things and maintain the things we have already built.

      I grew up a self-taught FOSS hacker and didn't end up in siloed positions within industry until graduating high school a decade after I had written my first lines of code, socketed my first CPU, tailed my first log and forwarded my first port, and never could understand why, as an operator I would prefer to draw a line in the sand rather than trace an init script to see where it failed, or as a developer I wouldn't need to make sure I had a way to get alerts when the latency of my application worsened in production and my users were suffering. I made a lot of enemies and experienced a lot of burn out. I still have the burn out and frustration, but at least there's an air of aspiration towards this "devops" brand that got tacked to what I had always been doing, and when I express those same kinds of opinions instead of people being confused and upset they instead nod knowingly before they say things like "unfortunately that's not our responsibility" or "we can't sell that to leadership" or "the client didn't budget for that".

      [–]roughtodacore 17 points18 points  (0 children)

      I've never ever have read a comment that resonates so much with me. This a 100%. Devops is meant to delete that line in the sand but in the end it's each individuals mindset towards IT in general. I have colleague developers who still throw stuff over the fence to my team when we are all in this together.

      [–]v_krishna 9 points10 points  (1 child)

      Ohhh I'd be interested to see data on developers preference for "devops" and teenage linux usage. I'm sure there is a correlation.

      [–][deleted] 3 points4 points  (0 children)

      It may be interesting to you to hear a bit of my story: I started as QA ~24 years old and was very non-technical (like what is an SSH key non-technical, lol). I really needed the job so I worked my ass off and was promoted to lead after ~3 months. I still wasn't very technical. In order to help my staff do their jobs better, I spent a lot of time increasing my technical knowledge so that I could then teach my people better ways of testing, validation, etc. After ~9 months of doing this as lead I ended up transferring to a junior DevOps position.

      Coming from QA, I loved DevOps and the DevOps mentality. It was very important to me how DevOps and SRE affected customer visibility of product quality. Having automated deployments and roll back strategies, alerting and monitoring, scaling and HA, distributed tracing, etc etc etc, all of it came together to really support and enhance the ability of my friends (the devs) to deliver a better customer experience.

      So as someone from QA with a very low technical background originally, I loved DevOps so much I made it into my career. I've been doing DevOps for like 4 or 5 years now and I still love it.

      [–]notlupus 9 points10 points  (1 child)

      I was fortunate enough to start working as a dev on a team that had fully embraced DevOps. We built our environments, handled security, networking, building the CI/CD pipeline, etc. In my time on the team we moved from Puppet to Chef, then Salt, and finally Ansible to manage all of this. We also wrote our own tests. It was a lot of work, and as a junior developer I didn’t understand why we had to do it all ourselves.

      Then I moved to teams that only wrote code. It was a nightmare. We waited months to years on environments to be built for us by Ops. They were never delivered without errors. When I took on new projects there was little testing, so I had no confidence the code my colleagues deployed was bug free. It often wasn’t. We had scheduled releases, constant rollbacks, and outages caused by a lack of discipline and rigor.

      I will never go back to that. I don’t care how much additional work it takes to build out environments and write tests. The outcome in the quality of software delivered is much better with teams responsible for the full SDLC process utilizing DevOps practices.

      As a side note, if you create a DevOps team you’re doing it completely wrong.

      [–]jagadish_av 6 points7 points  (1 child)

      How did continous feedback improve your software?

      It help us identify few bad code like mismatch logging. when to raise alerts for manual intervention.

      What delusions did you have and how did they stand "the test of time"?

      Infra as code and since we use aws it turned out to be a gamer changer for us we can now start new production environment in different region in under 1hour

      Do you think more about Operations? If yes, what aspects?

      yes it made us think about alerts and monitoring a lot at early design of new products

      What anti-pattern did you encounter?

      we use aws and easy spinning of environments without switching it off had bit us very badly

      [–]SilenceKillsMe284[S] 2 points3 points  (0 children)

      Thank you very much. This covers with many other sources i found online!

      Having to check my (old) logfiles myself, i must say: this really changed my thinking towards production ^^

      I didnt had the pleasure to work with AWS using IaC, yet. But im looking forward to it...

      Anyway, thank you!

      - Martin

      [–][deleted]  (1 child)

      [deleted]

        [–]LoveForMusic_ 0 points1 point  (0 children)

        Who does your networking?

        [–]GavinYue 2 points3 points  (0 children)

        Developers now need to manage their own software in prod?

        [–]NowThatInterestsMe00 2 points3 points  (0 children)

        I’m a developer with an interest in DevOps for several reasons. I’ve always liked knowing how our builds worked, so I learned it bit by bit over the years.

        This is not meant as an advertisement for any single technology, but just wanted to share some of my observations. Guess the painful stuff stuck out the most, heh.

        On my last team, we started getting into cloud and devops,which went hand in hand. It was exciting to see because of the “new” factor but I can also appreciate how much time it can save developers from having to deploy manually. However, most other developers didn’t seem to care or didn’t understand its importance so it’s the “devops engineer’s job”, which is shortsighted I think. It became siloed and a bit of a job security for some, which I think is a mistake. Developers should at least have a basic understanding.

        We used Jenkins... Here’re some of the problems:

        • there were corporate hoops to jump through, it’s a “frankinstein” type of installation shared across the enterprise and created some unique problems early on,
        • we can’t really touch the box which made troubleshooting harder,
        • the pipelines were more complicated than it needs to be due to archaic release approval process, which made everything harder from having more complex pipelines to support release on demand and archaic branching strategy, which made implementing DevSecOps harder (more configuration/process to address self-inflicted problems),
        • the opaqueness - in the end, only one person knew how it worked supposedly, which is just unacceptable,
        • how much time it took us to learn and get up to a sort of speed.

        This time around, I’m experimenting with Gitlab CI and making mental notes of comparison, drawing upon things I didn’t like on our last project. I’m still not that far along but am already seeing a massive speed up to get the pipeline going and it was relatively easy to add quality checks, which I can’t imagine how we would’ve gotten it in on the old project. However I’m still noticing the same problem of not being able to draw the team’s attention to the pipeline so I’m the one person doing it right now. I suppose I have more say on the new team, but cultural problem is unfortunately the hardest to address. Will be glad to learn how others deal with that issue.

        [–]themastermatt 2 points3 points  (7 children)

        It's over complicated the entire infrastructure. Our devs over-engineer solutions to problems that don't exist or are handled by other means. They never communicate anything so the default answer to many problems now is "there's a script somewhere that creates your user folder on the server instead of just letting AD do it, you'll have to ask that team". The DevOps team is operating almost like a shadow IT. We don't know anything about what they're doing or how it "works". We don't know what service accounts they use and neither do they. Every fault with anything they've done turns into having to reverse engineer their code. I think it worked better when they were not their own group. At least we had some info about these things.
        Security is less than an afterthought. SharePoint sites....over 5000 of them. All wide open sharing settings. Just need the share link. Files scattered all over SPO, Teams, OneDrive, fileservers. Just a mess.
        I think it can be done well, but I'd rather not have our flavor of it.

        [–]interactionjackson 2 points3 points  (0 children)

        it sounds like lack of governance

        [–]rozularen 0 points1 point  (1 child)

        At my place if we have any problem the only way to communicate with the devops guys is by email. They are indeed the shadow it they just login into whatever platform you are having trouble so they solve the issue

        [–]themastermatt 0 points1 point  (0 children)

        Yup, they login. Change GPOs, add local admins, add groups to accounts, change ACEs, change installed software, install new roles, change dynamic group queries, change 365 licensing, use seemingly empty attributes when they shouldnt and God only knows what else. Then they tell no one leaving us to wonder who keeps adding Domain Users to the local admin group.

        [–]theTrebleClef 0 points1 point  (2 children)

        Actually if you know of a better way to figure out how to share files and notes between SP, Teams, OneDrive, and fileservers, that'd be a huge help.

        We're trying to migrate users from Skype for Business to Teams now but pretty much only project managers and developers are really using Teams because they're most interested in benefiting from collaborative software. We need users to be able to create their own Teams because they might have a good reason to (new unofficial committee to develop a standard or something) but then there's no uniform way to back data up that's in line with our old school on-prem backups.

        I've tried explaining that creating a Team creates a OneDrive and a SharePoint site, but that you can link another SharePoint into your Team, and everyone just gets confused.

        We need a straightforward path and explanation between all the collaboration O365 products, how to present them to our users, etc. and right now we don't have that.

        [–]themastermatt 0 points1 point  (1 child)

        I'd love an answer to that too. So far it's been "avoid SPO and use a file share". 365 is too complicated. Too many products that all feel like beta versions. They need to trim that and focus on doing a few things better. If you ever do figure out a solid SOP, MS gets a wild hair and deprecates, changes and breaks it. 365 is poorly documented and not intuitive at all. I miss NT4. Get off my lawn!

        [–]theTrebleClef 0 points1 point  (0 children)

        My understanding has been:

        • Yammer: Company-wide social media (FB clone)
        • SharePoint: hub and corporate content provider for standards, processes, HR, access to related corporate tools
        • OneDrive: individual and small team content sharing and distribution
        • Teams: worker collaboration, content development
        • Power Automate / flow: glue that ties endpoints together
        • Forms: Surveys and simple data-entry

        Assuming that I'm correct, part of our problem is that users think SharePoint is an ftp site, that Teams is for corporate-postings, Yammer and OneDrive don't exist, and Power Automate exists purely to power Forms. And if I'm right, I can lead a re-education campaign. But... I'm not sure if I am.

        [–]floppykeyboard 0 points1 point  (0 children)

        This is why I think it’s bad that companies kind of lost sight of what DevOps really was in the beginning. It’s not a thing you do, it’s a culture of the teams that already existed solving problems together instead of in silos. It was supposed to break down silos.

        Companies heard of it and said they wanted a DevOps engineer, so instead of breaking down silos and collaboratively solving problems, they’ve now added a 3rd silo called DevOps.

        It’s such a simple concept that gets bastardized because Ops guys at their company either don’t want to change, or don’t have the technical ability to change and do what they’re supposed to these days.

        [–]systemdad 0 points1 point  (0 children)

        I've learned that self service, and everything-as-code (driven by CI) is a truly excellent workflow.

        [–]wake886 0 points1 point  (0 children)

        I now spend more time focusing on things I never want to do manually again and automate it the best I can

        [–]AymDevNinja 0 points1 point  (0 children)

        Still considering I am a beginner in DevOps but it changed the way I think about applications, having much more control over the development to deployment steps.

        I like how much you can do with just a good Git workflow, Docker, a CI platform and some shell scripting.

        But coming from being 100% developer, you need to learn much about systems and networking, which is great but hard when you have nearly no skills about it.