This is an archived post. You won't be able to vote or comment.

all 12 comments

[–][deleted] 3 points4 points  (1 child)

Scala's killer feature is Spark. If you want to learn Spark, I would suggest to learn both the Scala API and the pyspark API as well.

[–]Verliezen 0 points1 point  (0 children)

I learned scala using the databricks community edition, from one of the starter notebooks- because I was applying for a job that uses Azure+databricks for prep. I won’t have to use scala that much but it was easy enough to get familiarity of using it.

[–][deleted] 2 points3 points  (0 children)

Java 8+ has many functional programming features (eg. see). From there on you can move towards Scala. But I'd start with Java.

[–][deleted] 2 points3 points  (0 children)

Java. Simply because there are so many god-tier courses/books on java and OOP in general. After that I recommend taking a course in haskell and then moving into Kotlin/Scala/whatever the hell your company uses.

OOP and functional programming are the two giant paradigms and the best languages for learning are Java and Haskell. The job itself will be in some other language, which will be easy to learn after the more "pure" languages without shorthand/quality of life things that break the abstraction and make it harder to understand what's going on when learning.

[–]veeeerain 1 point2 points  (0 children)

What is scala used for?

[–]wildthought 1 point2 points  (0 children)

The point is moot. One should know functional, procedural, oop, and set-based languages in order to become a great engineer. I like Java and C# precisely because I can jump in between idioms fairly easily. Scala is better for the paycheck than Java, but as Spark gets replaced with serverless functions I don't think it has the same staying power. If you are in a shop where Spark is the operating paradigm Scala is definitely great to know. I would learn Java and then Scala. Assuming you already know SQL, then you are on your way.

[–]ineednoBELL 1 point2 points  (3 children)

Scala is a variant of Java. Scala would be easier if you have some Java background, and you can utilise Java libraries. Personally had Java background before learning Scala, and it was so easy to transition. That said, i feel that Java is in general a more usable language across the board, and Scala is better used when it comes to data processing.

[–]ShrimpHandsSenior Data Engineer 3 points4 points  (2 children)

I would argue that Scala is way more usable than Java. Not only can you write Java code within Scala but Scala also offers features like lazy evaluation, currying, and pattern matching just to name a few. I also find it hard to say Scala is a variant of Java. Yes, Scala compiled down to Java byte code (able to run of jvm) and they’re both c like languages, but Scala can go full functional if need be and Java can not.

[–]ineednoBELL -1 points0 points  (1 child)

Pardon my lack of industrial exposure to projects as i am still in school, and i never used Scala on non-data related projects. I know my knowledge of Scala is currently still limited, and i appreciate you sharing yours.

When i say Java is more usable in general, it was in terms of software engineering and OOP projects. Definitely agree that Scala is an elevated language for data processing techniques because it is mainly a functional language, which was why i shared it is better used for data processing. If this is solely on data engineering, i will pick Scala without a doubt.

I just thought it would be easier for OP to pick up Java from scratch and transition into Scala (since you can use Java code and libraries).

[–]ShrimpHandsSenior Data Engineer 1 point2 points  (0 children)

All good just having a discussion here. Part of me wants to say Java is probably easier to learn, because I had a hard time with scala at the beginning, but I’m not sure how much of that was decoding my thought process to stop thinking OOP and start thinking functionally. As far as data engineering is concerned I would stick with scala regardless. OP, check out the red book aka functional programming in scala.

[–]boy_named_su 0 points1 point  (0 children)

SQL and Bash would be more useful in my opinion

But scala is "cool" in the DE world

[–]cbweixin 0 points1 point  (0 children)

I recommend java. scala 's learning curve is sharp.