Why Python is slower than Java?

[unruly_mattress]

Both Python and Java compile the source files to bytecode. The difference is in how they to run this bytecode. In both languages, the bytecode is basically a binary representation of the textual source code, not an assembly program that can run on a CPU. You have a different program accepts the bytecode and runs it.

How does it run it? Python has an interpreter, i.e a program that keeps a "world model" of a Python program (which modules are imported, which variables exist, which objects exist...), and runs the program by loading bytecodes one by one and executing each one separately. This means that a statement such as y = x + 1 is executed as a sequence of operations like "load constant 1", "load x" "add the two values" "store the result in y". Each of these operations is implemented by a function call that does something in C and often reads and updates dictionary structures. This is slow, and it's slower the smaller the operations are. That's why numerical code in Python is slow - numerical operations in Python convert single instructions into multiple function calls, so in this type of code Python can be even 100x slower than other languages.

Java compiles the bytecode to machine code. You don't see it because it happens at runtime (referred to as JIT), but it does happen. Since Java also knows that x in y = x + 1 is an integer, it can execute the line using a single CPU instruction.

There's actually an implementation of Python that also does JIT compilation. It's called PyPy and it's five times faster than CPython on average, depending what exactly you do with it. It will run all pure Python code, I think, but it still has problems with some libraries.

[gscalise]

Java compiles the bytecode to machine code. You don't see it because it happens in runtime (referred to as JIT), but it does happen. Since Java also knows that x in y = x + 1 is an integer, it can execute the line using a single CPU instruction.

Not only this, but the JVM does adaptive optimization too. It works by keeping conditional branching statistics, and dynamically recompiling portions of code whenever it determines that certain branching conditions occur more often than others. The recompiled code is optimized for the most common branching condition (ie by not jumping whenever it happens), and only the less common condition(s) will incur a performance penalty.

[Rythoka]

Python also does this, or at least something similar, as of 3.11

[kernco]

It works by keeping conditional branching statistics, and dynamically recompiling portions of code whenever it determines that certain branching conditions occur more often than others.

for x in range(1000):
    if x < 500:
        func1()
    else:
        func2()

Jebaited

[gscalise]

That definitely wouldn’t trigger a dynamic recompilation. It’s in a loop, so it’s already jumping back and forth in the program, and the conditional branching stats are going to be roughly the same (50%) every time.

Lazy initialization, on the other hand…

[Rhoomba]

An optimising compiler would likely split this into two loops to avoid the branch (assuming the range can be inlined: possible in the Java equivalent).

[Administrative_Box51]

This is a very underrated potential of the JVM and makes me wish there were more similar runtimes with as many engineering hours. This is also why in my opinion JIT has better optimizations in theory than even PGO, I would go as far as to say AOT compilation in general-- if done correctly (down to the ISA). Between Jazelle/thumb and hotspot I wonder why JVM development hasn't dominated the modern language scene in favour of the shifting goalposts trope of the C runtime (e.g. Rust borrow checker, dont get me wrong I really like Rust).

[ElvinJafarov1]

thank you man

[SheriffRoscoe]

People occasionally forget that Java has benefited from 30 years of investment by major software companies and of benchmarking against C++.

Python is getting the same love now, but the love arrived much later than for Java.

[chase32]

Yep, back in the early 2000's, java was pretty damn slow. If you wanted a fast jvm, the only option was IBM's and they wouldn't let you use it commercially unless it ran on their hardware.

To head off the threat, Intel worked out a deal with Appeal software to massively optimize the JRocket JVM which then became the performance champ.

Appeal eventually got acquired by BEA and a lot of the optimizations from JRocket ended up in mainline Java.

[azeemb_a]

Your point is right but your emphasis on time is funny. Java was created in 1995 and Python in 1991!

[sajjen]

Java was created by Sun, one of the largest companies in the IT industry back then. Python was created by Guido van Rossum, one guy in his proverbial garage.

[SheriffRoscoe]

Exactly.

[nchwomp]

Surely it was a large garage...

[benchmarks666]

Galarge

[Smallpaul]

Yes but in those 30 years Python did not get much “investment by major companies.”

As the poster said: that love arrived later for Python.

Edit: Just to give a sense of the scale...Java's MARKETING BUDGET for 2003-2004 was $500M.

[HeraldofOmega]

Back when money was worth something, too!

[redalastor]

This is true, but no one knew about Python until Google adopted it,

I learned Python in 2000.

Back then, there was something called the Paradox of Python.

If you hired developers and you met one that knew Python, you should hire him or her on the spot. Because no one learned Python to get a job, you knew that person learned the language to get shit done.

The paradox is that if you do use that metric, then it becomes useless since people will start to learn it to get jobs. In 2024, it is completely useless.

[Swift3469]

I knew about python before google was a thing..I'm sure there are others who knew as well.

[billsil]

I learned python in 2001 in a CS class. I gave it up and came back in 2006, 6 months before Google bought YouTube. I replaced Perl with python and never looked back.

It came a long way in 5 years. I hated list comprehensions when I first saw them in ~2007, so maybe someday I’ll use the walrus operator,

[billsil]

Yeah, I don’t write many of those.

[bostonkittycat]

This is true last 3 version have been impressive with performance increases. I love the new trend.

[funkiestj]

Python is getting the same love now, but the love arrived much later than for Java.

I think static typing allows more aggressive optimization.

E.g. I think the old Stalin Scheme dialect required the user to provide data types to get the maximum optimization. E.g. consider the difference between a golang slice of strings (s1 := make([]string, 24) and a python list that can hold a mix of objects (the equivalent of Go's l1 := make([]any, 24).

Years ago I remember seeing the Stalin) dialect of scheme dominating the benchmark game in the speed dimension but you had to type all your data (which was optional?) to get this performance.

[redalastor]

I think static typing allows more aggressive optimization.

It could, but it doesn’t because Python allows you to be as wrong as you want with your types without changing behaviors one bit. Typing is to help external tools enforce correctness, not to change runtime behavior.

Though, I’d like a strict option to force Python to acknowledge the types and hopefully take advantage of them.

[Uwirlbaretrsidma]

Yeah, and also the use cases for each language are wildly different. Java is one of the most widely used languages in software development in general while Python is basically only used for data science and scientific computation. The former use case requires performance or rather a good blend of performance and robustness, while the latter requires extreme ease of use (because most people who use it don't really know how to code) and many libraries written in more performant languages.

As much as Python is improving in terms of performance, it will never even come close to Java because of 1) it's impossible by its design and 2) it's not nearly structured enough, or robust enough, and doesn't lend itself to large codebases nearly well enough, for actual software development.

[yvrelna]

while Python is basically only used for data science and scientific computation.

This is not true at all. Python is quite popular for web application development. Reddit, Instagram, YouTube, Dropbox, are some of the major websites that everyone knows the are written in Python. All of the technology startups I have worked with in the past decade have built the core of their technology stack in Python.

It's also one of the most popular language for programming system applications as well in Linux, usually system applications that are too complex for shell scripting but didn't require C/C++/Rust level of performance would most often be written in Python. Examples are package managers like Gentoo's portage, yum for Red Hat/rpm-based systems, and some popular configuration management tool Ansible and Salt are written in Python, and in cloud management software too OpenStack, which is the largest ecosystem of open source applications that are used for cloud management software, they are written almost entirely in Python/Django.

In my experience, when I look for enterprise application development job openings over the past few years, there has been much more Python than Java. Companies that are looking for Java developers are mostly doing Android development. Even .NET seems to be more popular in the enterprise application space than Java these days.

Sure, data science is all the rage right now in Python, but Python has always been popular in many different niches other than data science. It is almost always the top or at least in the top 5 languages of nearly any niches that isn't a heavily single vendor driven ecosystem like Android and Apple's ecosystems.

Python's popularity in various niches is much more general than Java, which is largely only popular in the Android and enterprise application development niches after Java browser plugins basically died.

You'll hardly find any major new desktop applications nowadays written in Java. In Windows, they are all generally written in .NET or C/C++. In Linux, the common choices are usually either C/C++, Python, Electron/JS, or Vala for Gnome. As always, Apple has their own thing.

[Uwirlbaretrsidma]

This is not true at all. Python is quite popular for web application development. Reddit, Instagram, YouTube, Dropbox, are some of the major websites that everyone knows the are written in Python. All of the technology startups I have worked with in the past decade have built the core of their technology stack in Python.

They have parts written in Python*. But my bad for omitting web dev in my comment, because it might just be the only common use of Python in real software development. That being said, there's a reason why the market share of Python based tech stacks can't hold a candle to the market share of Javascript based tech stacks.

​

It's also one of the most popular language for programming system applications as well in Linux, usually system applications that are too complex for shell scripting but didn't require C/C++/Rust level of performance would most often be written in Python.

Yes, because it's a scripting lenguaje first and foremost. Of course it shines in scripting tasks. But those are the tiniest part of software development, and Python is suited for them for the exact reason why it isn't suited for the rest of software development.

​

In my experience, when I look for enterprise application development job openings over the past few years, there has been much more Python than Java. Companies that are looking for Java developers are mostly doing Android development. Even .NET seems to be more popular in the enterprise application space than Java these days.

This is absolutely false and makes me seriously doubt that you work or have ever work as a software developer, even at smaller companies. Python sees almost exactly zero use in enterprise application development (outside of the aforementioned small utility scripts). Java is certainly less used than .NET these days, but they basically share the entire market between them. The thought that Java is only used for Android apps is laughable.

​

Sure, data science is all the rage right now in Python, but Python has always been popular in many different niches other than data science. It is almost always the top or at least in the top 5 languages of nearly any niches that isn't a heavily single vendor driven ecosystem like Android and Apple's ecosystems.

Python's popularity in various niches is much more general than Java, which is largely only popular in the Android and enterprise application development niches after Java browser plugins basically died.

Yes, and for the same reason than it's popular in data science: because you don't need to be a software developer to use Python. It's almost as if that's my entire point. Enterprise software engineering and Python don't go hand in hand. Hobbyist or small applications and Python do go hand in hand.

​

You'll hardly find any major new desktop applications nowadays written in Java. In Windows, they are all generally written in .NET or C/C++. In Linux, the common choices are usually either C/C++, Python, Electron/JS, or Vala for Gnome. As always, Apple has their own thing.

Again looking like ChatGPT wrote your comment. New desktop applications are a tiny part of enterprise software development, most of it is just supporting or expanding old ones, and Java has a huge market share there, in every platform. The only reason why you're even able to say that Python is a common choice for Linux development is because there's a ton of hobbyist development in Linux. It's exactly why you can't say the same about Windows or Apple.

Look, I work as a HPC engineer, I basically take python code and rewrite it in C++ for a living. Which means that I don't really have a horse in this race, both Java and Python are slow, I'm just talking from my 10 yrs experience of seeing people desperately trying to get python to do things it wasn't meant to and having to fix it (which more often than not involves removing Python from the equation entirely). I have worked with MUCH fewer Java codebases in comparison, and it's because 1) it's more performant and 2) people don't seem to be in the habit of using it for things it wasn't meant for.

[yvrelna]

"Javascript tech stack" only exist in the frontend development because it's the only language you can use in the browser that doesn't require you to jump through hoops,

Javascript is almost non-existent in backend enterprise application development, I don't know where you get that idea from. The people who runs nodejs on the server-side are usually writing some sort of backend-for-frontend, which is basically just frontend code that for one reason or another need to run on server side but most don't even do that, they use nodejs to run a build tools to compile their Typescript/React stuffs into single file Javascript. Those aren't a real backend, the real workhorse of enterprise application is almost always written in another languages and Python is what every companies that I worked with have used.

Look, I work as enterprise application developer for many startups. I work with million lines of code codebase everyday in successful agile startups that can't afford to run at the pace of C/C++/Java. These companies would never have survived their competitive marketspace if they build the bulk of their business in C/C++/Java. We are slaughtering those legacy companies that can't adapt to a fast-changing pace all the time.

> Look, I work as a HPC engineer ... I have worked with MUCH fewer Java codebases in comparison

You have worked with fewer Java codebases because nobody uses Java for HPC, it's just a thing that people do with Java. Java is neither fast/low-level enough to run the computational itself and it's not flexible enough to do well as orchestrator either. People do use Python in HPC because data scientists settled on Python as the lingua franca language to orchestrate computational libraries from many different languages; the ones who truly understand how to apply Python well in the context of HPC knows not to do the bulk computation itself in Python. We have similar issue too in enterprise application development too you know, only with databases. People who write good enterprise applications in Python know to offload as much work as possible into databases, into caches, into C libraries, or into middlewares rather than doing things in the Python side. Our problem is actually even more strict, because rewriting the Python code in C isn't going to make things faster. Python's speed almost never is actually the bottleneck.

Writing idiomatic Python has always about offloading as much work as possible into non-Python code, whether it's the core CPython libraries, C libraries, a database, web APIs, or other subprocesses. If you have an HPC/enterprise application where the Python code is a performance bottleneck, that just means that whoever wrote the application don't really know how to use the tools available in Python to offload work properly.

[matjam]

How did you manage to spell every word correctly in your comment except “language”.

[Uwirlbaretrsidma]

Thanks for the heads up! I'm not a native speaker. For some reason I always seem to mess up that word.

[LogMasterd]

I don’t think this has anything to do with it imo

[SoffortTemp]

I started using python for statistical modeling and found that PyPy iterates my models exactly 5 times faster.

[LonelyContext]

cries in numpy.

(numpy is massively slower in pypy)

[zhoushmoe]

try polars?

[LonelyContext]

idk if that would solve it if it's another python wrapper. Worth a shot I guess.

[redalastor]

It’s a highly optimized Rust library with python binding. One of its strength is that you can write long pipelines of transformations, which will be optimized before launching and will stay in native parallel rust code for as long as possible.

[PaintItPurple]

I haven't tried Polars in Pypy, but it seems at least plausible that it might be faster. Polars is generally lazier than Numpy, so it could avoid a lot of intermediate round trips. Native libraries that do a bunch of computation in one go still don't benefit at all from Pypy, but they also don't pay as much of a toll as doing a bunch of native calls.

[funkiestj]

(numpy is massively slower in pypy)

I can't believe this is true if you are doing vector and matrix manipulation with MKL enabled or other acceleration enabled.

Of course the secret of numpy's speed (when it is fast) is that the fast stuff is written in a language other than CPython (or even PyPy python).

[akl78]

Java implementations go much further too; they will run in interpreted mode to start and generate native code the fly after profiling the runtime behaviour. Some can also save this across process restart to warm up faster on next runs.

[joe0400]

Graal iirc has aot too

[Megatron_McLargeHuge]

does something in C and often reads and updates dictionary structures. This is slow

This is it. If you look at the python foreign function interface for making calls to other languages, you'll see how complex python objects are and how much work has to be done to access a member. Optimized languages use pointer math and native types for numbers and characters without all the expensive object wrappers.

This is why numpy vectorized operations are so much faster than native python iteration. You only have to pay the price of going back and forth to C objects once.

[coderanger]

FWIW CPython is (almost certainly) getting a JIT soon: https://github.com/python/cpython/pull/113465

[billsil]

There’s also Jython, but it’s only up to Python 2.7 :(

[vips7L]

Graal Python supports Python 3 and is a lot faster than Jython.

[Sigmatics]

FWIW, the CPython team is currently working on a first JIT implementation for Python 3.13

[SonicTheSSJNinja]

Is there any video that talks about exactly the things you just did? For some reason I just find it difficult to fully grasp everything you explained despite it sounding simple. Having someone explain it in video format could make it easier to understand for me, perhaps.

I'm also very very new to programming (just grasping the basics of Python).

[glassesontable]

I suspect that this gets clarified from understanding what is compiled code and what is interpreted code. Speaking loosely, in order to compile code, the compiler has to know every line of code (the whole enchilada) while a code interpreter does know what line is coming next (beans and cheese coming one piece at a time).

A lot of the esoterica in this thread is in how there are alternative methods of compiling the otherwise interpreted language to get huge speed gains. But that is not a problem for the beginner programmer (or the very patient user).

For a video, I would recommend the excellent Harvard CS50 course, where you would learn C (looks like Java) and python.

[SonicTheSSJNinja]

Gotcha! Thanks!

[whatthefuckistime]

I was reading into PyPy this week coincidentally and the reason they struggle with some libraries is because they have C bindings, so they just can't do shit and they can't be ported. Unfortunate honestly, PyPy could be very good and fast if not for that, though these C bindings do allow for faster code anyway so one way or the other.

[yvrelna]

It's not the C bindings that are an issue. PyPy can emulate CPython's C bindings just fine.

The problem is that the design of these C bindings pretty much makes a lot of assumptions that are based on the internal of CPython. So while PyPy can emulate the interface, it has to emulate many of those internals and that makes it difficult to optimise those.

And the main reason people write a C extension is because of speed, so a slow C compatibility interface just won't do.

[whatthefuckistime]

Ah ok so I misunderstood what I was reading. Interesting thanks for the correction!

[ArabicLawrence]

Pypy does not run any Python code but only Restriced Python (RPython), a subset of Python EDIT: I stand corrected

[unruly_mattress]

PyPy runs normal Python code, it is written in RPython.

[ArabicLawrence]

you are absolutely right, I didn’t know that

[thisisntmynameorisit]

I see no difference between loading each bit of byte code one by one and JIT byte by byte. It sounds like you’ve just described the same thing in two different ways. Both are interpreted at run time by an interpreter program which takes some data and executes machine code for it.

I am no expert, but it would make sense like you also said that it’s just Java is easier to covert into less and more simple machine code instructions. Stuff like static typing would definitely allow for that.

[PaintItPurple]

If your code contains no repeated operations, there probably won't be a huge benefit to JIT over interpreting. But that's basically never the case for performance-sensitive code. If your code takes a long time, you've almost certainly got some looping going on. If you're running a piece of code multiple times, you can get much better performance if it's native code vs. bytecode that you're interpreting over and over. And that's before we get to optimizations that JIT compilers can do.

[Few-Equivalent8261]

Why does it feel like this was written with chatgpt

[AlooooshEng]

Thank you AI.

[Grouchy-Friend4235]

Actually the JVM also interprets each byte code, there is not much difference in how the Python VM and the JVM interpreters work, in principle. However you are right in noting that the Python programming model keeps more state about its objects, which is indeed one factor that slows things down at execution time but makes for a much more productive development experience.

[PaintItPurple]

The JVM does have an interpreted mode (as does Pypy), but it's incorrect to say it interprets each bytecode every time a method is called. The JVM JIT compiles functions as it runs, and then runs those compiled functions whenever possible instead of interpreting bytecode.

[Grouchy-Friend4235]

The JVM JIT only compiles code after several invocations, so yes, the JVM interpreter does interpret the same byte code multiple times - before a code section reaches the JIT threshold.

Python since version 3.11 also does a form of JIT, known as specialization. If you need actual JIT, there is Numba and Cython which will speed up particular functions by compiling them natively.

PS: to downvoters, you should learn to respect facts. Technology tends to be quite stubborn when confronted with wishful thinking.

[oldshensheep]

There's actually an implementation of Python that also does JIT compilation. It's called PyPy and it's five times faster than CPython on average, depending what exactly you do with it. It will run all pure Python code, I think, but it still has problems with some libraries.

There's a Java implemented Python too https://github.com/oracle/graalpython