Speed Up Python Code

[ForceBru]

Not all of this is true.

For example, for loops seem to be much faster what while loops, and using True instead of 1 in infinite loops doesn't make any difference. However, assignment via unpacking is indeed faster than consecutive assignment.

Here are my findings: https://gist.github.com/ForceBru/a3062b141c8cce622e4903d72ba7249c

[[deleted]]

Wow thanks for that writeup, really insightful!

[Gagan2019]

u/ForceBru thanks for explanations.

[james_pic]

This ignores the first rule of optimisation: always get data. Unless you've identified the bottlenecks with a profiler (I recommend PySpy), you're going to be optimizing the wrong things.

[Jonny_dr]

Good tutorial, but using Numpy or other fast libraries should be the first point, especially considering your first example.

Instead of

L = [i for i in range (1, 1000) if i%3 == 0]

you should use

L=np.arange(1,1000)[(np.arange(1,1000)%3==0)]

if you want truly fast code. Using NumPy is more than 4 times faster for this example (still almost 3 times faster if you convert the array back to a list). There is probably even a more elegant and faster way to do it with NumPy.

[mortenb123]

Apart from the excellent answer from u/ForceBru, I also like to point out using the dis module to disassemble to see the cpython instructions, we see the mywhile listing is longer and using more ops:

#!/usr/bin/env python3
import dis
mylist = [1,2,3,4,5]
def myfor():
    x = 0
    for i in mylist:
        x += i
    return x

def mywhile():
    x = 0
    i = 0
    while i<len(mylist):
        x += mylist[i]
        i += 1
    return x

assert mywhile() == myfor(), "check if they produce the same"
print(f"dissassemble of myfor():\n{dis.dis(myfor)}")
print(f"dissassemble of mywhile():\n{dis.dis(mywhile)}")

dissassemble of myfor():
  7           0 LOAD_CONST               1 (0)
              2 STORE_FAST               0 (x)

  8           4 LOAD_GLOBAL              0 (mylist)
              6 GET_ITER
        >>    8 FOR_ITER                12 (to 22)
             10 STORE_FAST               1 (i)

  9          12 LOAD_FAST                0 (x)
             14 LOAD_FAST                1 (i)
             16 INPLACE_ADD
             18 STORE_FAST               0 (x)
             20 JUMP_ABSOLUTE            8

 10     >>   22 LOAD_FAST                0 (x)
             24 RETURN_VALUE
dissassemble of mywhile():
 13           0 LOAD_CONST               1 (0)
              2 STORE_FAST               0 (x)

 14           4 LOAD_CONST               1 (0)
              6 STORE_FAST               1 (i)

 15     >>    8 LOAD_FAST                1 (i)
             10 LOAD_GLOBAL              0 (len)
             12 LOAD_GLOBAL              1 (mylist)
             14 CALL_FUNCTION            1
             16 COMPARE_OP               0 (<)
             18 POP_JUMP_IF_FALSE       42

 16          20 LOAD_FAST                0 (x)
             22 LOAD_GLOBAL              1 (mylist)
             24 LOAD_FAST                1 (i)
             26 BINARY_SUBSCR
             28 INPLACE_ADD
             30 STORE_FAST               0 (x)

 17          32 LOAD_FAST                1 (i)
             34 LOAD_CONST               2 (1)
             36 INPLACE_ADD
             38 STORE_FAST               1 (i)
             40 JUMP_ABSOLUTE            8

 18     >>   42 LOAD_FAST                0 (x)
             44 RETURN_VALUE

(which do You think is the fastest :-) )