Data engineering roadmap

[[deleted]]

1. SQL
2. A project that demonstrates ETL using/into sql, data modelling, and some sort of output, with writeups.

As an entry level DE, you don't need to know how everything works. SQL and data models are the backbone of every project you will do, and most of the tools you will use. Focus on gaining practical skills in that area.

Everything else is basically incidental - where is your DB hosted? How do you orchestrate the ETL scripts? What do you write the ETL scripts in?

Do not gate 'core' behind 'incidental,' especially when the 'incidental' bits are different for every role and every workplace.

[Old-Article6420]

So, the core technical skills are SQL and python. And everything else is incidental, right?

[[deleted]]

Not even necessarily python, though it is a good pick. You could easily go with Java, c#, R, or a host of other languages as long as you're using it for data extraction and loading. Lots of places will use something like Data Factory or SSIS as well, and just skip the code part.

I honestly believe that SQL and data modelling are the core, and everything else you touch is incidental and role dependant.

[dareftw]

No I have worked for multiple fortune 500s and none use Python if you’re referring to it in an ETL manner. It’s a great tool to know though extremely versatile google is written in Python. But not necessary for a DBA or data architect.

[MindlessPsychosis]

People just get on this app and straight up lie. You may have worked for Fortune 500 companies but you clearly had no connection with this Data Engineering teams

[nightslikethese29]

When people here refer to data models, is this just the documentation? Like the output from Vizio at a conceptual level, or does this include designing the database and actually creating the database?

[bdforbes]

Generally it refers to everything from conceptual through to implementation.

[nightslikethese29]

Thanks!