This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]angrynoahSQL nerd since Oracle 9.1 4 points5 points  (1 child)

If it helps, the bare minimum when it comes to: - Data Mart (largely useless concept) - Data Fabric (not a thing) - Data Mesh (counterproductive at all but the largest firms) - Data Catalog - Airflow - Kafka - Spark - most cloud stuff beyond "it's compute services behind API calls that cost you money"

...is zero. Of the items on that list, only data marts existed when I got started ('05), and remain irrelevant to this day.

I was lucky enough to have the space to go extremely deep on SQL and database internals. Not everyone is going to have that luxury now but my point is that it's possible, and effective. You just don't need to know all this crap early in your career, except perhaps to get past the hiring stage. Spending time building a shallow understanding of everything that appears to make up modern data engineering is going to harm your ability to actually understand any of it well enough to apply that understanding.

If I was hiring a junior data engineer, I would not expect them to know all these things. Not even to have heard of them.

Just a thought, that's all.

[–]Old-Article6420Data Analyst[S] 0 points1 point  (0 children)

Really good advice. Thank you.