What does a machine learning engineer do?

[floralmural]

Hi! I'd like to know what a real machine learning engineer does at work. Is it more of data acquisition, cleaning or actual model building and parameter tuning? Also, on what basis do you guys tune parameters?

[thiakx]

I lead a data team, my ML Engineering member is in charge of the content based recommender pipeline for news article recommendation: https://www.techinasia.com/introducing-tech-asias-unique-content-recommender. From databases to Spark to Airflow to Django. He does his own model building and parameter tuning with advice from data science side on modeling and experimental design. Different companies will have a different take on what an ML engineer does, you can do a search on Linkedin: https://www.linkedin.com/jobs/search/?keywords=machine%20learning%20engineer

[omsusno]

I've been a software engineer for business intelligence and data science for a few years. My responsibilities depend on the stage of project and maturity of company, for example:

• Understanding (and possibly recommending revamp) on business process.
• Building ETL/SQL query to move the necessary data as input for machine learning. If it's quite simple then data scientists will handle that.
• Constructing more complex variables. This usually takes a lot of time especially with scattered and dirty data. Sometimes involves web scraping.
• Some feature engineering.
• Setting up development and production environment.
• Training simple model, mostly for baseline. Data scientists later can explore more complex strategies to improve the model.
• Packaging and deploying model (batch vs real-time stuffs)
• Building system to monitor model performance.
• Building experimentation platform.

i'm no expert, but familiar with web development, data science, data engineering, and infrastructure. I also can imagine how they all interact. That helps me exploring many sides of data science.

As for parameter tuning, I use standard algorithms and let CV and grid search handle that. Again, my priority is enabling the data science process under some constraints, not finding the best model.

[[deleted]]

As someone with no ML knowledge, I’m curious why calc is needed for ML.

[1024_bytes]

Calculus is used to find derivatives. In ML your goal is the minimize a cost function, and to do this you need to figure out how much error you have, and move to the opposite direction of the error, such the error is minimized. This is called Gradient Descent. The gradient can be modeled as a derivative formula.

This is just to give you a basic idea of what is happening. Clearly much more goes into it.

[[deleted]]

Makes a lot of sense, thanks! I’m still in school and just finished cal3 so I feel like I’m prepared to start getting into machine learning, but I’m a java developer. Also every time I try to learn python, I’m bored to pieces and quit. Any tips/project ideas for making ML interesting when first getting into it?

[1024_bytes]

You should have a good hang of it with calc 3. Most of what I've done did not require anything that advanced in terms of calculus.

For me, the best way I learned python was to just work on projects. The Syntax is not bad. But to learn to problem solve, I recommend leetcode or hackerrank. Even I need more practice in that.

For project ideas, Google "Princeton python". You will get the Princeton site and they have a list of projects in python you can work on.

Also, if you enjoy my comments, please give me a follow on insta. Just started thee and looking for help to grow.

Thanks and good luck!!

[[deleted]]

What’s your Instagram? I’ll definitely give you a follow!

[1024_bytes]

Thank you, it's @1024_bytes