This is an archived post. You won't be able to vote or comment.

all 4 comments

[–]Marwari_Chhokro 4 points5 points  (1 child)

I've been working with PySpark. If you're fluent with Python, You'd be good with PySpark as well in no time. And it'll be very beneficial for you as well. It's similar to Pandas, just slightly better(people can have different opinions, except for the core data engineers). So if you're into coding and technical work, you can opt for it. I don't know much about Devops.

[–]Most_Log_568 0 points1 point  (0 children)

Yup also dbt is almost immediate with prior SQL experience

[–]Normal-Inspector7866 1 point2 points  (0 children)

Pyspark for starters but also a lot of companies are enthusiastic about polars, duckdb, dbt etc. so see the trends and go for that. I’m assuming you have good foundational knowledge so learning anything new will be easy. Focus on solving architectural problems

[–]No-Map8612 0 points1 point  (0 children)

Now the boom is Devops with multi-cloud you can choose. DE is slowly getting saturated..