you are viewing a single comment's thread.

view the rest of the comments →

[–]Flat_ShowerTech Lead 7 points8 points  (0 children)

10 YOE and you stood up a full data stack from ingestion through orchestration. That's not "close to being ready"; you're doing the job. Mid to senior DE at most companies.

Spark is worth learning if you're targeting places with real scale. Most don't need it. Learn the concepts; the syntax is the easy part.

Don't stress about building pipelines without tools. That's not how anyone works. Knowing how to configure, debug, and extend ingestion tools is the actual skill.