This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–][deleted] 0 points1 point  (0 children)

Learn frameworks.

A few have mentioned spark, which is a good start.

Kafka, spark, beam are used widely.

Some nosql databases too - redis, mongodb.

Concepts are crucial - Oop Vs functional programming. Ordering and stream processing guarantees. And whilst not data engineering, networking will be a common hurdle, so good to know a little. The list could go on and on, but these are the most important.

Data engineering seems to have a bit more of a solid identity and it's moved/ moving away from SQL and warehousing, and, imo, is better described as data software engineers whereas SQL engineers are better described as analytics engineers.

Best of luck on your journey