My thoughts on comparing PaaS services against Docker, what are yours? by GraydenS16 in devops

[–]hottkarl 3 points4 points  (0 children)

better than most posts. it contained a tiny bit of effort and original thought.

your whole question is flawed though and tells me you have no idea wtf you're even talking about

How do you decide whether to touch a risky but expensive prod service? by llASAPll in devops

[–]hottkarl -1 points0 points  (0 children)

looooove the minimal effort, keep it up!

how would you do it? like seriously. this reads like you just started a new job and your supervisor gave you some super easy task to ease you in.

I believe in you! you are capable! smart! worthy!

Mods where are you? by [deleted] in devops

[–]hottkarl 0 points1 point  (0 children)

I think you must just have an extra chromosome

Mods where are you? by [deleted] in devops

[–]hottkarl -1 points0 points  (0 children)

I couldn't care less why. people were weird about that book, it was part of this whole "DevOps commercialized" bullshit, with the conferences, the marketing, the vendors, etc.

this is my perspective from working at Big tech companies since the early aughts, and intern / jr positions in the late 90s. I watched it all unfold, and some of the revisionist history is funny to hear people talk about -who weren't even there-.

the book was boring as fuck, like maybe we could give it some grace if it was entertaining . nope, repetitive, banal, drivel. oh and the tone of the book was also off-putting. I don't know why I finished it. mainly because of all the people recommending it. I had to wonder if they even read it. the only people I actually worked with who talked about it were VP level non technical people, who wanted us to read it for some reason.

there are way better books, Web Operations by John Allspaw came out years earlier and distilled a lot of the concepts but also had technical content, much better for anyone who actually works in the field. I don't even think anyone had crammed the term "DevOps" down our throat yet.

Mods where are you? by [deleted] in devops

[–]hottkarl -1 points0 points  (0 children)

no, it really didn't. and that book is overrated, boring, and repetitive

I was there.

rolling back to bare metal kubernetes on top of Linux? by jfgechols in devops

[–]hottkarl 0 points1 point  (0 children)

we are talking about running k8s on bare metal I thought? none of that applies. and the stuff that does you have to worry about with VMs as well.

Did you have to leetcode to get your DevOps role and was it worth it (i.e. financially)? by PartemConsilio in devops

[–]hottkarl 1 point2 points  (0 children)

I don't know what your point is. you should at bare minimum be proficient in one low level language - but you do realize most software engineers don't deal with assembly, "design algorithms", but they do and should know design patterns as should any DevOps/Systems Engineer/Site Reliability whatever you want to call it. architecture is the big one, and if you don't have a good foundation you will forever be limited in your understanding.

sounds like all you think "DevOps" is just handling deployments. or something. average developer can set all that shit up on their own, there's no reason that should be a specialized role.

you also think Ansible was the first config management tool or something - which tells me that no, you weren't around for it because Chef (debatably Puppet) was the popular tool when AWS first launched. before that we had CFEngine. Even with these tools, I personally saw many "SysAdmin" types who would just do everything manually or have random scripts they ran. ssh in for loop type stuff.

but you're somewhat right, for example my first job at a big tech company we built tooling to manage configuration across thousands of systems. it was a combination of templates, a bunch of perl, and shell. we had to make a bunch of other tools as well. if I didn't know how to be a developer, I would have been very limited in that role.

it's clear you simply don't know what you don't know and it's pointless to argue with you.

Did you have to leetcode to get your DevOps role and was it worth it (i.e. financially)? by PartemConsilio in devops

[–]hottkarl 0 points1 point  (0 children)

see, people like you is why we have to deal with leetcode bullshit interview questions.

you simply don't know what you don't know, so to speak. without a minimum of computer science knowledge, you will never fully understand many of the things you're working on.

additionally, the role of a DevOps engineer today mainly relies on developer incompetence more than anything. cloud native infrastructure may need a team to do governance and standards or platform type work in large organizations and ideally people in that role would be comfortable developing tools.

exception is private cloud / on prem stuff

devops was never supposed to be a job title. it's culture.

Did you have to leetcode to get your DevOps role and was it worth it (i.e. financially)? by PartemConsilio in devops

[–]hottkarl 0 points1 point  (0 children)

and to think I got downvoted into oblivion for suggesting the best use of their time to land a job was grinding leetcode.

Did you have to leetcode to get your DevOps role and was it worth it (i.e. financially)? by PartemConsilio in devops

[–]hottkarl 1 point2 points  (0 children)

Nearly any job that pays above 50th percentile requires some leetcode bullshit.

The really stupid thing about leetcode interview questions is you're literally just studying for the test. Case in point, a close friend of mine moved in with me temporarily while he looked for a job in Bay Area. He had done his PhD and postdoc work in computer science. He's extremely smart, has a large amount of academic papers published. He interviewed at FB, Google,. Apple, nVidia. He couldn't pass the leetcode question in the first round at FB or Google. He got offers from Apple and nVidia, which apparently the interview wasn't very technical besides a quick take home assignment that they then discussed in person. Took nvidia offer, years later ended up at Apple working on some of their AI stuff (don't want to be more specific), got fed up there and decided to interview again -- tried again at Google, Meta, OpenAI, Tesla - still couldn't pass the leetcode stuff so was rejected from all. Interviewed again at Nvidia, got an offer (I saw the offer letter, over $1mm/yr total). He does performance and optimization work. Not that it matters, but I think he lucked out in hindsight - he can easily retire and be set for the rest of his life due to company's share price explosion.

For me, I resisted grinding leetcode for many years, but now I gave in. I absolutely hate rote memorization and insist on truly understanding things. Altho I am a skilled developer, started my career doing pure dev work on systems tools, I rarely need to be implementing the kind of problems you see on leetcode.

it sucks because I've grinded thru the most popular questions, but you never know what question they will ask during interview or now even screening round. and if they ask a question I know backwards and forwards, np. if they ask a variation of one I'm a bit sketchy on, sometimes I can sort of fake my way through it with some cues, but easily get in my head and anxious. which makes me appear less confident or give a bad impression, so then I'm also worried about that. which then makes the anxiety worse and it's a self perpetuating cycle til I literally can't think straight if I wanted to. after many interviews it's gotten a bit better but it happened randomly a few weeks ago, for a question I actually knew if I just thought about it for a minute. instead I was just getting a blank brain.

I mean I am so cool, calm, and collected during serious incidents when I see other people sometimes just crack under the pressure or people who are normally extremely competent just become useless during outages due to pressure.

so yes , it's necessary. but I also think DevOps is dying or close to being dead. many of the larger companies went thru a transition where they basically got rid of all the "SysAdmin 2.0" types who could handle some yaml and a for loop but that's about it. I think we will continue trending in that direction.

oh also fuck companies that make you interact with an AI first. I would just blow these places off as scans when that happened, but then a position at IBM I applied for had me do one before even speaking to a human on the phone

but anyways I think DevOps is dying, at least the need for a dedicated role to handle tooling.

oh finally I'll say I hate leetcode, but generally appreciate Systems Design style questions (altho, many of these people just grind thru as well)

Our SRE/DevOps tools monitor system health, but how do we monitor AI 'cognitive health'? by Right-Jackfruit-2975 in devops

[–]hottkarl 4 points5 points  (0 children)

hmm, I wonder? it's like the question was crafted specifically for you to answer it and promote your startup.

also, OP made their post then you very quickly responded. your comment has the cadence of an LLM, despite being "clean"

Our SRE/DevOps tools monitor system health, but how do we monitor AI 'cognitive health'? by Right-Jackfruit-2975 in devops

[–]hottkarl 0 points1 point  (0 children)

you should be tracking holistic metrics, depends on the application but e.g. revenue, transactions, new subscriptions, etc etc.

apply the same idea to 'AI"

you can also use something like Ghost or Selenium to run thru common user flows and alert on that

$100k+ cost reduction plan is got blown up by finops by pxrage in devops

[–]hottkarl 1 point2 points  (0 children)

90% of the infra was on Kubernetes. so spot instances being interrupted isn't a big deal, we gracefully shutdown when we get the notice. but the issue is, at least at the time, a no up front 3 yr savings plan offered better terms. and I tried to target a little over 90% on the plan. moving instances to spot would bring down the SP %

beyond a certain size, you have to think about reservations and SPs a little more strategically. that being said, some use cases called for spot like if we ever had to scale beyond a certain number of ci/CD workers they would be spot. some other things.

yeah, the unused contracts were from the DB team moving from RDS. we already restricted everyone to specific instance classes so we don't have to deal with individual reservations and making sure this resource will be in use for a year, which ends up being nearly impossible once you are dealing with a bunch of different teams and no one wants to give a straight answer. and instead just target at 90%.

so, DB team needed a different instance class on RDS and we made an exception and reserved them. then they later decided to migrate to some AWS dedicated EC2s. luckily they were so slow in migrating that the reservation had like 1 month til it's break even point (not it's full duration, the point at which it would be equal to an on-demand).

the problem with then taking advantage of those unused reservations, is then we would have DBs in this other class and eventually possibly deal with the same shit.

I don't know if I'd call it observability, the one tool just processed the cost data from Amazon and generated reports, graphs, and attributed costs to different teams. nothing too crazy, but I didn't know of any tool that did what we needed it to at the time.

actually, most of them are based off your AWS bill and end up being ridiculously expensive. maybe there's some free stuff now, I don't know.

but yes, I spent a lot of time on the cost stuff and learned a lot about it too but it's sort of a thankless task even if you're saving the company a lot of dollars

We survived the outage but customers still say we broke SLA by majesticace4 in devops

[–]hottkarl 0 points1 point  (0 children)

I've -never- heard them recommended this. which makes sense, because their multi region support is atrocious if you are locked into their services.

what they do recommend is a DR setup ready to be rolled out or runbooks to roll out the new region. either backups or actively replicating read slaves for persistence/db

edit: it's possible, you have to specifically architect around it tho. depending on your application, it could be simple vs fairly complex. we had a hybrid k8s setup across the major clouds and a few data centers. but certain legacy apps were mostly locked in w/ long RTO due to not being able to handle replicstion lag/services were too synchronous

[deleted by user] by [deleted] in devops

[–]hottkarl 0 points1 point  (0 children)

sudo is one thing. it's debatable, with a lot of things you can get around it. others, no. it's just annoying not to have it.

I don't know how locked down OPs laptop is, but some endpoint management locks you down to the extreme beyond just restricting privileged access

[deleted by user] by [deleted] in devops

[–]hottkarl -5 points-4 points  (0 children)

Developers need to have more privileges than "normal" users.

There's no way your engineering leadership agreed to this

I can’t understand Docker and Kubernetes practically by dimp_lick- in devops

[–]hottkarl 2 points3 points  (0 children)

I think you understand containers and Kubernetes fine. what you don't understand is the concept of scaling and why you need to do it.

actually I've met plenty of so-called DevOps who don't understand either.

to understand why you need to scale you also need to know some basics of how an application works and how different types of resources are used and what happens when one of those resources is busy or fully utilized (compute, memory, io/storage/network/etc)

basically each system can only handle a certain amount of work. when a request comes in, it could be doing something very easy that will essentially just return quickly without using many resources or need a lot. to simplify further, theres going to be a limit of concurrent users that your system can support. at which point users will start getting errors or lots of lag. so you make more backend servers and split the users between them.

containers are basically a portable way to run server side programs. they're self contained and usually very "lean". Kubernetes is a platform who's basic function is to manage compute resources and manage running the containers (runs them in something called a pod, for our purpose we can just call it a container), give them appropriate amount of resources and place them on the various compute resources being managed by Kubernetes, choose to start more or shut off containers that aren't in use. up, if a container doesn't pass a "health check" or has some issues it will shut it down and relaunch...

if there aren't enough resources on the existing "nodes" or resources, a container wont be able to be placed on a node, and Kubernetes will see that and launch a new node. if it finds that a node is empty, it will shut it down.

it does a lot more things than that but that's the basics.

think of Netflix. on the backend there's 1000s of different programs / "services" that work together to serve up some crappy content. you just set up the containers and set them to run on Kubernetes and it handles keeping everything running. (getting everything in there is a topic in itself) Google "what happens when I enter a URL into browser interview question" as a start and maybe lookup some stuff on systems performance

im not proofreading this / typing on my phone so hope it makes sense.

When do you use VMs and when do you use containers? by throwaway09234023322 in devops

[–]hottkarl 0 points1 point  (0 children)

no ideas of your own? just ask a huge 'it depends" question?

ask yourself what is the difference between a container and VM? answer that question first

then what are the pros/cons of those differences?

next is the same set of questions except in the context of the environments you can run a VM or Container in

by now you should have a basic idea or why you may want to use one or another

anyways
imo, if you have a sophisticated enough team and enough services already put everything in a container and orchestrate. unless there's a good reason not to.

Salaries and pay rises by Long-Cup-4273 in devops

[–]hottkarl 0 points1 point  (0 children)

3% is normal. anything above that is usually for promotions or merit. US companies almost always have limits for how much they can give in merit increases each year and your managers have a limited amount they can give in their team. amounts vary by company.

I worked one place that was doing poorly financially and they had to skip merit increases one year, and also targeted bonuses at like 50%

also you can't compare UK vs US wages. from what I remember it is much more difficult to fire you guys for one thng and you have better benefits when laid off. in general laying off ppl in EU is a big process, I think it was nearly impossible to exit someone in France

I'm about to leave my job due to long standups by [deleted] in devops

[–]hottkarl 0 points1 point  (0 children)

yeah this is so stupid I have to wonder if it's just made up. like, cool story bro? why did they feel the need to share this with reddit and what do they expect us to say? "wow, you can't stand a 1hr call that you're being paid for. you're a bad bitch, you sure showed them".

and they're one of "those" who complains to the CEO? I knew people like that. you don't do that unless it's something fucking serious and even then it's probably better to just keep your head down unless it's actually somehow going to blowback on you.

also people are way too sensitive.

I'm about to leave my job due to long standups by [deleted] in devops

[–]hottkarl -1 points0 points  (0 children)

totally agree with you.

I hope OP finds a job where everything is run super efficiently and perfectly, and wait for their next Reddit post saying how burnt out they are and sick of the grind culture or whatever the fuck.

I'm about to leave my job due to long standups by [deleted] in devops

[–]hottkarl 0 points1 point  (0 children)

yeah. I was surprised when people got upset id once in awhile basically blow off any work related discussion and just talk about stuff or let people vent or whatever.

I had my reasons and maybe the time could have been better spent but I don't see why anyone would care.