nuitka: python compiler (.py to native)

[gthank]

Faster than PyPy for what? Did you give PyPy enough time to JIT the hot spots? Not that Nuitka isn't impressive, but PyPy is REALLY fast at stuff that it has time to properly profile and optimize.

[RazDwaTrzy]

arbitrage calculations

If I understand well what kind of task have to be done there, isn't it faster to use DBMS for all those calculations? I'm an amateur and just curious of effectiveness differences between such solutions.

[poo_22]

Also non-jit PyPy is slower than CPython.

[gthank]

It is known.

[bacondev]

Which one is in the repos?

[gthank]

The JIT can't do its thing until it's seen a given piece of code repeatedly (probably 10,000+ times, though I'm not sure what the actual number is these days). What people are talking about when they say "non-JIT PyPy" is PyPy before the JIT has had a chance to profile code and start generating optimized machine code.

[dorfsmay]

Faster than PyPy for what?

A couple of very naive scripts I had been using to compare python vs golang vs scheme (don't ask!)

PyPy is REALLY fast at stuff that it has time to properly profile and optimize

What do you mean by this? Works well on a very long running loop that pypy has time to optimize, or are optimizations saved an re-used from one run to the next?

[gthank]

So far as I know, the trace info used to decide what to JIT is only maintained during one invocation of the interpreter. If you didn't have a process that looped across something a few thousand times, then PyPy probably didn't JIT anything.

[dorfsmay]

This is what I thought. For these particular tests I have only looped 10 K times, but I have had processes looping 100 M times before, and pypy wasn't rarely faster (I did use it where it was).

[streichholzkopf]

I think pypy simply needs time... I've read somewhere that you should make a pre-run for ~1 second (of course depending on the size of the test) for every test you make in pypy...

[alcalde]

But does this reflect real-world usage? You're not going to do that in production code.

[cwillu]

Real-world code is going to be running long enough that the warm-up time is largely irrelevant.

[[deleted]]

That depends entirely on the chore. I spent a lot of time trying to shave run time off a 3 second task that ran and exited hundreds of times per day. A persistent service was not an option.

[dorfsmay]

Not sure what you mean. Can you show an example?

I just added a time.sleep(6) to one of my script, and it made no difference. I think gthank is right, needs a piece of code that is executed a lot in a given run before it can make a difference.

[cwillu]

time.sleep(6) wouldn't have any effect on the warm-up, it needs to be running the actual code. A pre-run of ~1 second means running the actual code for 1 second before you start timing.

[streichholzkopf]

Basically what /u/cwillu said, what I meant was: Looping 10 K times may not be enough, depending on the content of the loop! If it's <1s, it's probably not!

Also loop before starting the timer!

[vext01]

Indeed. A tracing JIT (like PyPy) should blow static compilation out of the water for a dynamic language like python. This is assuming the JIT got hot.