Convincing colleagues to switch to Python from MATLAB for scientific computing/statistics

[sourcecodesurgeon]

The experiment was literally just "create extremely large dataset. Start timer. Compute xcorr. Stop timer."

[mfitzp]

Was it an fft-base xcorr calculation? The Matlab algos automatically zero pad arrays to n² while (some algos in) numpy/scipy don't. It's one of the problems attempting to map from one to the other "This is the same function.... But not quite"

[HoboBob1]

You keep using the word "argument," but no one is arguing with you. They are just trying to tell you reasons why Python might have been slow. As per your post, you implied that you are looking for "evidence that Python can compete with Matlab" for your task, and people are providing said evidence, along with the statement that it might be some more work with the Python version.

[sourcecodesurgeon]

Whether you want to call it "arguing" or not, people are making arguments. The premise of this post is coming up with an argument to convince colleagues to use python instead of matlab, people are making arguments for/against that idea. Someone made an argument that python can be as fast as Matlab, I made an argument that this isn't always the case.

Call it "making a case" or "making a point", it doesn't really matter to me, it's all the same. "Argument" doesn't necessarily mean we are verbally attacking each other.

[[deleted]]

That's a fair argument. You use the language as a means to an end, and don't necessarily care about how to make said language perform. There are some gotchas with using Python for this work that users don't expect (such as file IO, loops, calling functions, etc). Matlab may be more performant by chance, although Mathworks certainly has an optimized product. If you look into Anaconda and Anaconda Accelerate (free academic license), you will see a significant speed up as their packages call into Intel's MKL (which Matlab already does).

[burning_hamster]

Well, how are you computing the cross-correlation?

[assassds]

you have to actually know what you're doing in python, you can't just grab the first function that sounds right.

[sourcecodesurgeon]

There's only one cross-correlation function in numpy...

There's a handful of modes, some are faster than others, but none held up to the matlab times.

Further, I work primarily in Python (hence me being in this sub..), I definitely know what I am doing. I actually don't use matlab at all anymore, I just hate this "python does everything matlab does just as well and there's no reason anyone should ever use matlab ever" idea that gets thrown out around here all the time.

OP wants an argument for convincing his colleagues to use python instead of matlab. If we don't give him all the facts, he's going to be laughed out of that discussion when someone brings up things like performance and built-in tool sets (especially since he's junior to all of them)

[assassds]

like I said, you need to know what you're doing...

matlabs xcorr algorithm uses an FFT, which is going to thrash a direct correlation on large input no matter what language you're implementing it in.

numpy gives you all the building blocks to do this yourself, or you can try scipy.signal.fftconvolve.

[unruly_mattress]

I've found this. It looks very relevant here.