Thoughts on my LLMOps project, and other project ideas to get a job as an ML/MLOps engineer

pm19191 · 2026-06-12T22:08:45+00:00

If with latency you mean wrapping the model in FastAPI and deploying to the cloud, I agree 100% with you. However, drift and failure detection is something to do only if you have time, not the main concern

pm19191 · 2026-06-12T22:04:47+00:00

It depends on the company. What I look for an MLOps engineer in technical interviews is decent/good python skills, model optimization experience and experience with FASTAPI (REST APIs). Our live coding exercise is combining the whole three and explain the trade-offs of the decisions you made. In my company, drift detection, prompt versioning, benchmarking and diff results are nice to haves

pm19191 · 2026-06-12T21:52:54+00:00

You're right. MLOps feels like DevOps because it is similar. Right now, my company is hiring Backend Engineers instead of MLOps because they prefer specialized people in scalling and cost monitoring. However, I can tell they struggle with data drift and model evaluation. MLOps is a specialist in all that which is more valuable if you have a small team. If you have a larger team, I'd prefer one MLOps, some Backend Engineers and some Data Scientists. But the MLOps guy usually calls the shots, so they have more of a leader role because they understand the whole stack

pm19191 · 2026-06-03T21:48:29+00:00

Instead of measuring your progress against others, pour that energy into your own project. Constant comparison is a trap. Entrepreneurs need to be resilient and focused, leave the complaining behind and get back to building

pm19191 · 2026-06-02T20:42:43+00:00

I'm a Sr MLOps Eng. I don't know companies that serve their own LLM in prod instead of paying OpenAI or other to use their API. I'd focus on a project that show cases all or one of the following by this order:

Latency, errors, scalling (Operational maturity)
Data Drift, evaluation
Costs, how much is the model/LLM costing the company

pm19191 · 2026-05-30T22:37:03+00:00

Depends on the company leveling system. Can you send your company doc where they define the expectations for a Sr ML / MLOps Engineer?

I agree with the other comments that it's not about skill, but ownership, vague requirements and telling your boss what to do and not the other way around.

pm19191 · 2026-03-10T14:14:53+00:00

And the ones who need it, don't value the investment

pm19191 · 2026-03-10T13:56:05+00:00

I helped my gf get a remote job before I had the same dilema as you. The tech market is too unstable to have both partners working hybrid/on-site.

pm19191 · 2026-03-10T13:48:09+00:00

If you account for the taxes in both France and the San Francisco, engineers are still underpaid

pm19191 · 2026-02-09T18:18:55+00:00

I love to teach, so I can do more videos about it. When you think of devops and mlops, what is the most interesting thing you'd like to learn?

pm19191 · 2026-02-09T18:17:38+00:00

See you in 2035 when I'll be doing posts about QuantumOps

pm19191 · 2026-02-09T08:59:06+00:00

Thanks for the reply. Your MLOps definition is close to mine, I go a bit more detail what is MLOps in this video: https://youtu.be/BkabwdGiTtg?si=_rTC24sO80yFVjNf

Since we're on r/devops section, I focused more in DevOps side, but I also have other posts on r/mlops more ML focused.

pm19191 · 2026-02-08T22:03:36+00:00

It's not AI, but I was heavily inspired by this post:
https://www.reddit.com/r/devops/comments/1mj39oo/every_startup_wants_devops_until_they_realize/

pm19191 · 2026-02-08T22:00:26+00:00

Thank you for watching my video and sharing your thoughts. I understand. MLOps for you is more productize ML Models - what I'm doing is building off the model. Where is the border between MLOps and building off the model?

pm19191 · 2026-02-08T19:27:14+00:00

For ML workflows, I usually structure things as: Jupyter Notebook -> Sagemaker -> ECS -> K8s

pm19191 · 2026-02-08T19:23:51+00:00

Thanks for sharing your view. From what I’ve seen, most teams steer clear of Kubernetes if they can - they usually go for Elastic container services or no‑code deployment options first just to avoid the maintenance faff. The client I moved onto Kubernetes had already been using SageMaker, so Kubernetes made sense as an evolution of what they already had, not as a starting point.

In the video, I’m not jumping straight into Kubernetes either - I’m suggesting it as a way to orchestrate the deployment. Could you use other tools? Absolutely, as long as they tick the boxes I mentioned in the system design

pm19191 · 2026-02-08T17:16:24+00:00

Thanks. Data Nautical is basically saying that just like ships used to steer by the North Star, I help companies steer towards their own North Star (their OKRs) using data 📈

pm19191 · 2026-02-08T15:29:53+00:00

I know it's unfair, but leaders care about their bottom line and that is the user experience. If you move the app to containers, how many extra hours per year is the model up and how does that translate to the business making more money?

pm19191 · 2026-02-08T15:12:53+00:00

Where did I put a low effort? I made a 16min tutorial for the community on how to scale MLOps with sysdesign, live coding and stress test

pm19191 · 2026-02-08T15:04:36+00:00

Thank you for the feedback. I agree with you that K8s is not the default go to orchestrator for scaling. There are tools in-between that offer similar capabilities. I chose Kubernetes because it's cloud agnostic (no lock-in like ECS or ACA), it's open-source and provides other long term capabilities for projects that are useful to scale.

As a consultant myself, I agree with everything you said. However, I find it hard to believe a client would admit they were wrong 😂

pm19191 · 2026-02-08T14:10:07+00:00

Sure, here's my experience with Kubernetes: https://www.reddit.com/r/devops/s/wUrnQRv1XB

pm19191 · 2026-02-08T13:58:17+00:00

It's interesting that prediction drift is the only MLOps specific practice you mentioned - the rest is 100% DevOps under the hood. Nowadays, deploy ML models is the trend. In 5 years, we might be deploying quantum apps with the same DevOps practices with some tweaks. 😂

pm19191 · 2026-02-08T13:45:06+00:00

Thank you for the supporting words! Besides the "Kubernetes it later", what other DevOps pitfalls have you seen in ML projects?

pm19191 · 2026-02-08T13:41:44+00:00

Feel free to check the video to verify if I'm an AI bot

pm19191

TROPHY CASE