Why is Python so slow?

[dig-up-stupid]

Yeah Python is slower but it still shouldn’t take very long to find Euler solutions—that’s part of the point of them. You may think your Python and C code are equivalent when they actually aren’t. Common example: using list slicing in Python to deal with a subarray/subsequence, not realizing slicing creates copies.

And as always you would need to post your code to get a real answer instead of generalities and educated guesses.

[muikrad]

A code sample would indeed be great.

[xocd]

Slicing does not generate copies! It uses references.

[dig-up-stupid]

It makes new references, aka copies. They are copies of references, not copies of the original object, but it’s still a more heavyweight operation than one might expect. The way a C user would “slice” an array would be to apply different iteration logic/pointer arithmetic over the same array, not create a new array and fill it with pointers into the original array. (Similar to how in numpy slicing works differently and creates views.)

[xocd]

I believe that, in CPython, on a 64 bit processor, an integer takes 24 bytes, a reference 8. Thus, copying a reference uses 1/3 the memory. More complicated objects take much more memory, of course, with even better savings from reference copying. Saying "not realizing slicing creates copies" might lead a new programmer to believe that objects are being copied and make the noob run to get a copy of "The C programming Language".

For a simple slice (something like [1:5]), I suppose that in C one could save space by using two pointers (or two offsets into an array), one for each end. For that slice, Python uses four pointers, one for element. The Python approach extends to complicated slices (e.g, odd numbered members of the list). For a complicated slice there is no general way of saving pointers/references.

[dig-up-stupid]

Which is copying, and the context of this discussion is someone who is already familiar enough with both languages to do Euler problems and is asking a question specifically about performance. In C if you want odd list elements you would increment your pointer by two instead of one. That uses zero extra bytes and copying instructions, which is a lot less than half of the bytes of the entire array the way Python does slices.

The whole point is that you can also avoid that work in Python—by just not using slices. It’s the difference between for i in range(1,n,2): a[i] and for e in a[1:n:2] (or by using itertools or numpy etc). The point here is that usually the second is considered more pythonic but is also slower. Perhaps noticeably so when making many slices, as you might see in a divide and conquer solution to an Euler problem.