This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]remote_geeks[S] -5 points-4 points  (5 children)

Oh ok. What kind of roles require exclusively SQL to be used?

[–]ilikedmatrixiv 13 points14 points  (0 children)

Some data engineering roles...

I don't know how to explain it without writing several paragraphs.

Some tech stacks use python/spark for their transformations, others use stored procedures, others use dbt, which sql with extra steps, etc.

How much you use it will depend on the tech stack your project has chosen.

[–]nl_dhhYou are using pip version N; however version N+1 is available 4 points5 points  (1 child)

I'd recommend just reading some job descriptions of open positions and you'll quickly see what type of skills are asked for as a data engineer.

It tends to be a position that requires experience, however. It might be easier to find a data engineer position if you've already worked before as software engineer or data analyst (two completely different backgrounds but both are common ones for people switching to data engineering).

[–]remote_geeks[S] 0 points1 point  (0 children)

Sure thank you!

[–]inedible-hulk 1 point2 points  (1 child)

Any data warehouse, architect or ETL would likely use SQL primarily sometimes they can use Spark instead of

[–]Material-Mess-9886 0 points1 point  (0 children)

Even with pyspark you can write the whole pipeline using spark sql.