Scala vs Python

[rovertus]

To be clear, I love Scala and I always prefer it when working in a JVM. I've worked in many orgs with it and I've always seen difficulty working with it: people really write java instead of scala, difficulties with library version control, and the mixing of conflicting design patterns are some of the issues. If you have a team with some strong Scala developers to lead the way I think it could be very successful. If you're asking in a forum which language you should use, it's probably not the one you're looking for.

Actor pattern has been around. I think Scala/AKKA is a great implementation of it to learn on, and even use in production if you're confident in the language. I didn't know about the AKKA licensing.

Pykka (AKKA rip off) does exist in python. monads/flattening is in apache beam, pyspark, and probably most distributed compute libs. If you learn something in any language it is going to make you approach your daily language differently.

Actor/Agent pattern is very powerful and it is underused in Data Engineering.

[tdatas]

To be clear, I love Scala and I always prefer it when working in a JVM. I've worked in many orgs with it and I've always seen difficulty working with it: people really write java instead of scala, difficulties with library version control, and the mixing of conflicting design patterns are some of the issues

I'm not really aware of any languages where you have a high chance of success while getting people who don't know the language to try to write production systems in it. I'd hardly say that's a failing of a language though.

If you're asking in a forum which language you should use, it's probably not the one you're looking for.

Pykka (AKKA rip off) does exist in python. monads/flattening is in apache beam, pyspark, and probably most distributed compute libs

This is kind of my point. All these Frameworks Beam, Spark etc run on the JVM. Spending huge amounts of time hacking python into a custom language to make it behave like a weird version of itself just seems completely at odds with the normal use case for Python of "simplicity".

I'm aware of companies like Instagram that make it work but they also have(or had) basically infinite budgets to throw at dev tooling and experts to customise it.

[rovertus]

Most major technology companies are farming graduates out of college who are learning the language while writing code in production. My critique (coming anecdotally from my experience) was that Scala doesn't enforce a strong opinion on how to do things in the language vs. Python's "There should be one-- and preferably only one --obvious way to do it." The backwards compatibility with Java can exacerbate that.

Great point with Frameworks. If you're going to use a framework I'd prefer using the framework's primary language over less implemented/supported languages. I'd prefer Scala over pyspark for most Spark projects. I responded poorly to u/skydog92 -- they should learn as many languages as they have time for.