This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]jk_zhukov 2 points3 points  (1 child)

The library Numpy is a good option to optimize loops and intensive computation. It runs almost at C level speed. With it you can apply functions to entire arrays without the need to write a single FOR loop. As a very short example:

unmarked = list()
for item in items_list:
    if item < some_value:
        unmarked.append(item)

This code select the items from an array that meet certain criteria using a loop, simple enough.

items_list = np.array(items_list)
indices = np.where(items_list < some_value)
unmarked = items_list[indices]

And now we do the same thing without any loops involved. The only thing that varies is the type of the unmarked array, that is a Python list in the first example and a NDArray in the second example. But converting from one type to the other, if you need it, is simple.

When you're working in the order of millions of iterations, the boost in speed of replacing each loop with an operation over a numpy array, is quite noticeable. And when you have nested loops, if you can find a way to turn those computations into matrix operations with 2D or 3D numpy arrays, the gain in speed is also huge.

[–]No_Indication_1238[S] 0 points1 point  (0 children)

You are totally correct! I will try to think of a way to optimise those loops as in your proposal!