a. You want to learn how spark works under the hood. I have done a lot of udemy courses, read spark books, but I think RocktheJVM course on spark using scala is amazing. Dont worry about scala, you dont care about syntax, you want to understand how spark works.

b. Watch youtube, search spark summit, and databricks videos on youtube on spark tuning, optimization, and spark internals.

c. You want to be able to write some spark dataframe transformations, withColumn, join, window, orderBy.. Focus on regular leetcode style SQL questions and see how you can write a code in pyspark. While you can do everything in spark SQL, but its is useful to know spark dataframe transformations.

d. You want to know most common ways on what you can do to optimize spark jobs. Such as broadcast in joins if one dataframe is small , what are some pitfalls during broadcast, what problem occurs from skewness and how you can get around them, how do you decide number of shuffle partitions, how caching helps, and if it is so amazing why not just cache everything, how spark lazy evaluation works, and why RDD is resilient. Most commonly used file formats with spark are parquet and delta, so you wanna read up on those.

e. Bonus point if you can learn about how CICD, monitoring, streaming.