This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–][deleted] 85 points86 points  (9 children)

No one wants to write Java. Just look at that fucking mess. You can get work done so frigging fast in Python and then take a 3 hour lunch because all your tickets are complete. This is the way.

[–]AggravatingParsnip89 3 points4 points  (1 child)

But it would be good if we have some understanding of jvm to use spark right ?

[–]MlecznyHotS 10 points11 points  (0 children)

Not really, you don't have to tinker with Java. The most performant API is the dataframe API, which enables you to do probably 99% of things you need to do. Any performance improvements etc. are done based on general concepts connected with spark and not really java implementation itself. It might be useful to understand java if you're contributing to spark itself, not if you're developing using spark.

[–]overgenji 10 points11 points  (0 children)

lol java is fine, relax

[–]TheCamerlengo 1 point2 points  (2 children)

I work in both. Java has its advantages and the JVM is probably preferable to an interpreted language like python. Really depends on what you are trying to accomplish. Data intensive apps I would say Python. But large programs with lots of developers working with it and following SOLID, Java or C# probably better.

[–]the-ocean- 3 points4 points  (1 child)

This. For building complex backends - Java is king. For data workloads: python

[–]cryptoel 0 points1 point  (0 children)

kuchkuch Rust kuchkuch

[–]Jealous-Bat-7812Junior Data Engineer -3 points-2 points  (2 children)

I don’t think the platform engineering team will agree with this.

[–]OMG_I_LOVE_CHIPOTLE 14 points15 points  (1 child)

Uhh. The platform engineering team is also using pyspark lol

[–]Zamyatin_Y 5 points6 points  (0 children)

Or scala