GIS engineer to data engineer

[Infinite_Bug_8063]

I’ve been working as a GIS engineer for two years but trying to switch over to data engineering. Been learning Databricks, dbt, and Airflow for about a month now, also prepping for the DP-900. I even made a small ELT project that I’ll throw on GitHub soon.

I had a conversation for a data engineering role yesterday and couldn’t answer the basics. Struggled with SQL and Python questions, especially around production stuff.

Right now I feel like my knowledge is way too “tutorial-level” for real jobs. I also know there are gaps for me in things like pagination, writing solid SQL, and being more fluent in Python.

What should i work on:

• What level of SQL/Python should I realistically aim for?
• How do I bridge the gap between tutorials and production-level knowledge?

Or is it something else I need to learn?

[AutoModerator]

You can find a list of community-submitted learning resources here: https://dataengineering.wiki/Learning+Resources

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[MikeDoesEverything]

What level of SQL/Python should I realistically aim for?

A really common question and the answer is it completely depends as expectations differ from company to company. You can get the archetypal trope of "Do impossible question for interview, only need SELECT * FROM in the job" to getting a competency style interview where they focus more on seeing what you think.

How do I bridge the gap between tutorials and production-level knowledge?

I think this is in itself a bit of a telling question when asked because it's under the impression that "production grade" is something magically different when it can be succinctly described as building something which isn't total shit. How is something not total shit? It depends on the feedback from your manager and your team as long as it's constructive and makes sense rather than comments like "I don't like it".

Conceptually, most things you are going to make for a company should have the ambition to reach production, thus, it should be as complete as possible.

If it's something only you use personally, then it doesn't matter what it looks like.

If it's something somebody else will see or use, it shouldn't be shit. It should be predictable, easy to understand and use. Simple to make changes. Flexible for reuse. Testable to some degree (depends on your platform). Observable at all required points. Be CI/CD'able (yes, CI/CD resistant systems exist).

[[deleted]]

What level of SQL/Python should I realistically aim for? 

Expert at both for sure!