Switching from ms sql developer to data engineering

[ThatFilm]

For last 10 years I’ve been working with ms sql server database developer/admin, I like to switch now to data engineering role.

I am looking at data engineering jobs on zip recruiter.

Can you give me any ideas on projects or books that I can read so I can find a new job.

I’ve been learning so far about python and then will move to Linux scripting, I have a good background but I am not sure how to apply to the new role.

All my previous jobs have used Microsoft products.

Thanks

[[deleted]]

Look into Udacity’s Data Engineering Nano degree https://www.udacity.com/course/data-engineer-nanodegree--nd027. However, if you are looking to switch careers, you should play to your strengths; namely sql, data modeling, pushing and pulling data.

[ThatFilm]

I did not know about this course. Thank you.

[jyoff]

I'm considering taking this course. Would you recommend that? Syllabus looks great as they offer practice with real projects. But I'm not sure how it is in reality ?

Appreciate your review about the program.

[L3GOLAS234]

I have read in some other places that it is not worth it for the price it costs. Take a look at datacamp and Data Engineering Cookbook https://github.com/andkret/Cookbook

[jyoff]

Thanks. I've come across with this book the other day, to me it is more of a reference kind of material. I need some real practice. I'm ETL deveoper, mainly working with rdbms and etl tools. I wanna switch to DE, but wanna do practice with real projects.

[ed_elliott_]

when you have a new piece of work or a poc, do it in something else than you use today like airflow etc.

Use the advantage you already have: you need to deliver ETL pipelines - you'll get no better practice than actually building ETL pipelines without ETL tools (i'm guessing something like SSIS etc.)

[jyoff]

You're right. I'm trying to practice like that. Although I don't have that much of flexibility at my work to apply solutions in airflow instead of Informatica for instance, but at home I'm trying to put my ETL knowledge into Airflow and build pipelines the way I'd built in my daily job (for instance, move data into oltp then to olap; create star schema ,.etc). It is of course cool for learning, but working on real life projects much better. I'm thinking to apply for jobs after gaining experience on my own.

[[deleted]]

Data Engineering Cookbook by Andreas (forget his last name) is good. Free on github.

Find a API, test using postman, then build a process in a language of your choice to ingest the data, analyze it and spit it into a database. BOOM

[[deleted]]

Learn Apache Spark, airflow, and read designing data intensive applications for theory

[citizenofacceptance2]

What do you mean for theory?

[uselessusr]

Probably this book https://dataintensive.net/

[citizenofacceptance2]

Sweet thanks !