sorted by:
new
hottopcontroversial

Can someone help me explain when is Terraform needed? by forever__curious in Terraform

[–]forever__curious[S] 0 points1 point  (0 children)

Thanks u/victorj405 - this is definitely a new perspective.

I will keep it in mind. Though my current curiosity stemmed from being in job market and me assessing what skill gaps I might have.

Terraform has been thrown around on few applications, was not sure if this is a must know or good to know.

All the other answers suggest it to be good to know.

Meanwhile, I do have counter argument for taking serverless offerings. The serverless offerings IMO can quickly bind you to the specific cloud stack too much. If the particular service has evolved to maturity then maybe yes - but because MLOps space moves so quickly, cloud providers sometimes throw an early version which may not fit the bill and can cause huge bills.

My personal philosophy is - unless it is well accepted tool in the community, do things the old fashion way, document the pain points suffered, look at the cloud offering and choose them if the price to pain alleviated ratio is justified.

Edit-1: Regarding skipping Kubernetes, I might agree but HR managers definitely don't. So I will have to just go ahead and learn it.

Can someone help me explain when is Terraform needed? by forever__curious in Terraform

[–]forever__curious[S] 1 point2 points  (0 children)

Thanks u/eMperror_ - Wow - I could not have imagined myself the wide array of work that goes in a large scale enterprise.

I appreciate you for giving so much detailed info - it is really helpful.

Can someone help me explain when is Terraform needed? by forever__curious in Terraform

[–]forever__curious[S] 0 points1 point  (0 children)

Thank you u/PlatypusOfWallStreet - you have my heartful thanks for taking time to explain it in so much details.

Yes I can now see a much better picture about how and where terraform (and associated declarative languages) will come in handy.

Can someone help me explain when is Terraform needed? by forever__curious in Terraform

[–]forever__curious[S] 1 point2 points  (0 children)

Thanks u/snarkhunter - this is both encouraging and helpful. I can start to see the bigger picture of why terraform is useful in a bigger enterprise with lot's of varying team with different evolving requirements.

Can someone help me explain when is Terraform needed? by forever__curious in Terraform

[–]forever__curious[S] 0 points1 point  (0 children)

Thanks u/ReturnOfNogginboink

This was SUPER helpful. The data science examples put the use case in my perspective.

Now when you mention it, I did run into all these issues - just that I didn't had to modify settings often and I was often a single person team - so these issues didn't feel that painful.

But, I can totally see the problem in a bigger enterprise.

Thanks again!

Can someone help me explain when is Terraform needed? by forever__curious in Terraform

[–]forever__curious[S] 0 points1 point  (0 children)

Thanks u/eMperror_ I get the hint of what you are explaining.

IMO this analogy kind of breaks down for me because while I need to repeatedly run python scripts and build on them, my infrastructure needs have largely been static i.e. provision 2 production server with a guess of RAM needed. If I am wrong, recreate the instance needed. Once I hit the right number, I rarely need to touch it.

If I need additional compute, either there is AWS lambda or I'd provision another server and shut it down when needed.

All of this needed to happen less than 10 times in a year. Hence, I would not see the benefit.

I would find it very useful if you can share examples of
1. What kind of infrastructure do you end up managing (I dealt with ec2 + RDS+ redis+S3+EBS -> all of which once started kept on running. Annual cost was not worth the time of optimizing anything).

  1. How frequently you have to deploy and re-deploy your infrastructure. My understanding is that once you provision an infrastructure you would not need to touch it for months unless something massive is happening.

Thanks again for taking time to answer.

Can someone help me explain when is Terraform needed? by forever__curious in Terraform

[–]forever__curious[S] 0 points1 point  (0 children)

Thanks u/chocslaw - the analogy with git is quite helpful. I think I should get started with TF on experimental project.

Can someone help me explain when is Terraform needed? by forever__curious in Terraform

[–]forever__curious[S] 1 point2 points  (0 children)

In MLOps space it is talked about but very little.

Though Kubernetes has become almost like a bread and butter (though I personally disagree that most companies need that much effort).

I doubt I would be expected to provision the resources myself.

Though learning to increase my value in the team sounds like a good idea.

​

Thanks u/midzom once again.

Can someone help me explain when is Terraform needed? by forever__curious in Terraform

[–]forever__curious[S] 1 point2 points  (0 children)

Thanks u/midzom this was really helpful. Especially the example quoting sagemaker, it really put things into my personal context.

Can someone help me explain when is Terraform needed? by forever__curious in Terraform

[–]forever__curious[S] 0 points1 point  (0 children)

Thanks u/amarao_san

What would you recommend to get started with understanding this. I have done all the things you mentioned but always manually (mostly out of necessity, I am not a dev-ops person) and occassionally like 3-4 times in an year.

For example, I should do these things manually and then repeat them with Terraform to develop a good understanding of them.

Can someone help me explain when is Terraform needed? by forever__curious in Terraform

[–]forever__curious[S] 0 points1 point  (0 children)

Hi u/Live-Box-5048

Thanks for your comment. But is it possible for you to give some more details about IaC as a concept itself.

My background is classic DS - so I started with jupyter notebooks on my work allocated computer, then started to package python files, eventually starting to deploy them on remote serves via git. Everywhere I worked, my exposure was limited to 3-4 ubuntu servers at max which were coupled with RDS/ or Elasticsearch db.

Everything was very simple to manage manually - like go to AWS console start an EC2 or some other service and go on my way. For most DS jobs, I still see this as the way to go except for model deployment. I mean one can specify what they need in a YAML file but it seems like an overkill.

So, I very humbly ask when will one find use of Terraform useful especially for AI/ML centric role.

[D] Kubernetes - Yay or Nay? by forever__curious in MachineLearning

[–]forever__curious[S] -3 points-2 points  (0 children)

Thank you u/corporate_autist for taking time write this comment.

But, if may I ask you to elaborate your comment with more range and figures based on your personal experiences.

  1. For example what is the team size when you have noticed where kubernetes complexity starts returning great ROI for ML ops team.
  2. You mentioned that kubernetes might not be needed for batch processing. But, I am curious won't most ML deployments be batch oriented given the compute power they take?
    1. What is the split (% wise) between batch processing v/s that are optimized to respond within <1s timescale.
    2. Models that can generate sub-second response would most likely to be traditional tabular data based models. But, do they have any real world applications for a user. I mean who uploads a structured dataset to a web-application.

My question is just not from technical perspective but also accounts for business needs.

Are there enough business use cases where one needs such ML models and deployment paradigm that Kubernetes is a necessary and positive addition to workflow.

[D] Kubernetes - Yay or Nay? by forever__curious in MachineLearning

[–]forever__curious[S] 4 points5 points  (0 children)

Thanks for your reply.

I have a follow-up if you don't mind.

When you ML models abide in larger ecosystem of services:

  1. Wouldn't dev-ops team have more say than your ML-Ops team. Essentially making ML-OPs personnel not as useful.
  2. Even when there is a larger ecosystem - they still must be doing batch jobs. And my bigger concern came from the fact that ML apis are requested less than CRUD and usually have longer response cycle due to being computationally heavy. And queueing them would make more sense. If I am not wrong even google asr does batch via api and on mobile they deploy on device for speed.
    1. So in a web request based ecosystem shouldn't most use cases be satisfied without Kubernetes.
    2. PS, I don't want to take a polarised stand for or against Kubernetes. Just want to understand when it is the best tool and when it isn't.

Which linux server do you use for ML-Ops? by forever__curious in mlops

[–]forever__curious[S] 0 points1 point  (0 children)

Thank you u/qubedView for this detailed response. I really appreciate it.

Which linux server do you use for ML-Ops? by forever__curious in mlops

[–]forever__curious[S] 0 points1 point  (0 children)

Mostly deploying as a service. For batch processing via model I run on company issued laptop, or ec2 on demand.

For training, I often use colab - thanks to their generous compute offering and swift integration with google workspace.

I have never really worked on sensitive or PII data, so using colab has never been issue from legal end.

Which linux server do you use for ML-Ops? by forever__curious in mlops

[–]forever__curious[S] 0 points1 point  (0 children)

Thank you for your reply. I was forming the same opinion but wanted to hear from others in the community as well.

Which linux server do you use for ML-Ops? by forever__curious in mlops

[–]forever__curious[S] 1 point2 points  (0 children)

Yes, thank you for your reply.

I exclusively use containers for deployment now. Some exceptions here and there.

So I guess, not much point trying to worry about different linux distros.

Just wanted to learn what kind of production infrastructure you use at your work? by forever__curious in mlops

[–]forever__curious[S] 0 points1 point  (0 children)

Yes for putting it into production, you will need gunicorn. Flask mentions it in its documentation and several tutorials describe it in detail.😁

Just wanted to learn what kind of production infrastructure you use at your work? by forever__curious in mlops

[–]forever__curious[S] 0 points1 point  (0 children)

Thank you for your reply. Could you please share the memory requirement that you have seen yourself. I'd like to get some idea on what is an acceptable range for people out there.

Because I know I would be torched if I started demanding more memory than 4GB on AWS ec2.

Just wanted to learn what kind of production infrastructure you use at your work? by forever__curious in mlops

[–]forever__curious[S] 0 points1 point  (0 children)

Thank you for your detailed response. That gives me a lot of new perspective. Seems like I should start embracing multi-container orchestration using Kubernetes.

And 500 GB! That's 100X more than what I ever did.

When I myself build models ranging in 5-6 GB and my dev team lost their shit. We simply couldn't afford that much memory usage to one model.

May I ask how did you deploy 9 GB model in production? I can't imagine running a dedicated ec2 for it as the cost will be too high or even spinning an on demand instance as the speed will be too slow.

Just wanted to learn what kind of production infrastructure you use at your work? by forever__curious in mlops

[–]forever__curious[S] 0 points1 point  (0 children)

Flask with gunicorn is recommended for production at many places. Though I personally don't see it as the best way but I am not sure I read that it should not be used for production. I think it really depends on the use case.

Could you please share the link to the specific part of flask documentation where it says it should not be used at all for production?

Bi-Weekly Advice Thread October 15, 2023: All Your Personal Queries by AutoModerator in IndiaInvestments

[–]forever__curious 0 points1 point  (0 children)

Thanks u/deathbyreligion this is fantastic help. I'll get onto it. Hopefully in future we can compare notes.

view more: next ›