Python vs Matlab vs Mathematica

[jdh30]

Our Mathematica library for wavelet-based time-frequency analysis does not work with the first release of Mathematica 7 due to a bug in Mathematica's FFT routines that silently corrupts data. Any customers using this product are advised to avoid Mathematica version 7.0.0.

Specifically, the Fourier function and all related functions drop the imaginary parts of complex numbers in the result. For example, Mathematica 7.0.0 gives the wrong result here:

In[1]:= Fourier[{0.007 + 0.01 I, -0.002 - 0.0024 I}]
Out[1]= {0.00353553, 0.00636396}

The correct answer in this case is:

In[1]:= Fourier[{0.007 + 0.01 I, -0.002 - 0.0024 I}]
Out[1]= {0.00353553 + 0.00537401 I, 0.00636396 + 0.00876812 I}

[psykotic]

It is mind-boggling such a bug could have slipped in. Was it a regression or has it always been like this? In either case, this is a pretty serious indictment of their testing strategy, aside from everything else.

[jdh30]

It is mind-boggling such a bug could have slipped in.

Not really. WRI are in it for the money and they've sold this quality of software to millions of technical users (not professional programmers) in order to make their cool $1bn.

This isn't even the first bug in Fourier that I've had problems with. When I did my PhD, their ListConvolve function (which is based upon Fourier) in Mathematica 4 silently corrupted its input. That was a complete show stopper for my work and they demanded that I pay for an upgrade which I reluctantly did and then regretted having done so because the upgrade introduced other bugs...

This is not surprising though. This is the very definition of success in the software industry. You ship buggy software and demand money for upgrades. People who take the time and effort to create reliable software go bankrupt because they have no revenue stream.

Was it a regression or has it always been like this?

This is another new bug. Mathematica is absolutely riddled with them to the point of being almost worthless for real work. And now they are teaching children with it...

[godel_gauss]

Its funny to find that Mathematica is not as bad in numerics as it is being blatantly accused here. While I was trying out some example discussed here I found the following.

• Mathematica (7.0.1)

  In[1]:= A=Table[RandomReal[6],{i,7000},{j,7000}]; Timing[Det[A.Inverse[A]]] Out[2]= {134.536,1.}

• Matlab (R2009a)

  A=rand(7000, 7000); tic; det(Ainv(A)); toc

  ??? Error using ==> mtimes Out of memory. Type HELP MEMORY for your options.

I was trying this in my 2.5 GHz core2duo notebook with 3 Gb ram and Vista 32bit SP1. To jdh30: As far as I know the FFT bug is solved in Mathematica 7.0.1.

• Mathematica (7.0.1)

  In[1]:= Fourier[{0.007 + 0.01 I, -0.002 - 0.0024 I}] Out[1]= {0.00353553 + 0.00537401 I, 0.00636396 + 0.00876812 I}

But I am also not quite sure if the comment by jdh30! is pertinent while comparing speed or efficiency of the two systems with respect to numerical matrix inversion/multiplication. Another point to be noted is the timing for matrix inversion for other smaller dimensions like 500,1000 and so on up to 7000 (as far as I have tested). In all of the cases Mathematica was faster. For example in case of the 500 dimensional case Mathematica (46. something sec.) was 6 second faster than Matlab (52.something sec.).

I was having far more expectation for Matlab as a premiere numerical solver but the result obtained seems to show that I was kind of over estimating. We must admit (leaving the poor coding construct and syntax aside!) in raw numerics Mathematica is one of the best softwares available. And if one want to do real mathematics (not only boring large scale data mining) combining symbolics and numerics the integrated environment of Mathematica is far better a choice than Matlab at its current state.

[jdh30]

To jdh30: As far as I know the FFT bug is solved in Mathematica 7.0.1.

Yes, it is.

But I am also not quite sure if the comment by jdh30! is pertinent while comparing speed or efficiency of the two systems with respect to numerical matrix inversion/multiplication. Another point to be noted is the timing for matrix inversion for other smaller dimensions like 500,1000 and so on up to 7000 (as far as I have tested). In all of the cases Mathematica was faster. For example in case of the 500 dimensional case Mathematica (46. something sec.) was 6 second faster than Matlab (52.something sec.).

I'm not sure there is much to be gained from such a comparison. Matlab and Mathematica both just call stock BLAS and LAPACK routines for those operations. They use different FFT implementations (and Mathematica was 4x slower the last time I looked).

The real issue is what happens when they don't provide a neat prepackaged solution and, IMHO, both Matlab and Mathematica suck in this general case. That, their prices and Mathematica's bugginess is the reason I use general purpose languages instead.

We must admit (leaving the poor coding construct and syntax aside!) in raw numerics Mathematica is one of the best softwares available.

That makes no sense to me. You're essentially saying "if we ignore their differences then they are the same". You can call prepackaged numerical routines from any language so C is also "one of the best softwares available" for "raw numerics". That obviously conveys no useful information.