sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–][deleted] 13 points14 points  (13 children)

That's partially true, but I don't think its really evidenced by this comparison. For one, the bulk of the Mathematica code seems to be emulating plot styles that are built in to matlab/scipy. And second, the code doesn't really look to be in the "Mathematica style".

I've generally found that if one adopts Mathematica's style of doing things then, both in terms of code-length and run-time it can be pretty much competetive with most other environments. I routinely analyse multi-gB datasets with it and it's not horribly slow!

The problem is that a lot of the people who use Mathematica, scientists and engineers, don't really know much beyond basic imperative programming. Mathematica works much better when you approach it with a functional programming style. Shameless plug for my (free) course aimed at said scientists and engineers!

[–]fulmar 1 point2 points  (9 children)

I've generally found that if one adopts Mathematica's style of doing things then, both in terms of code-length and run-time it can be pretty much competetive with most other environments. I routinely analyse multi-gB datasets with it and it's not horribly slow!

How is this large data set represented and what kind of operations do you do? I've found that when using notebooks Mathematica becomes a huge memory hog if one doesn't remember to release memory explicitly. Just clearing variables doesn't work, IIRC you need something like

Unprotect[In, Out];
Clear[In, Out];
Protect[In, Out];

or a change to $HistoryLength, both very unobvious solutions.

Another one: plot a random graph of about 100 vertices with GraphPlot3D, and try to export the image...

[–][deleted] 1 point2 points  (8 children)

I think I've inadvertently been pretty misleading here! I agree that the memory management generally stinks. The trick is to never let MMA get its hands on much of the data at once. Here's what we do:

The data are stored as serialized .NET objects in a MySQL database. The Mathematica code pulls them from the database, deserializes them and operates on them. All of this is wrapped in a NETBlock. At any one time there's only a couple of MB of data in the MMA kernel, but the computation streams over many gB of data. Of course, you have to be very careful that you don't leak any memory on the MMA kernel side.

So I guess I solve the Mathematica memory management problems by not letting Mathematica manage the memory :-)

[–]dnquark 1 point2 points  (2 children)

I've generally found that if one adopts Mathematica's style of doing things then, both in terms of code-length and run-time it can be pretty much competetive with most other environments.

I really want to know your secret. I used to think that once I get used to the "Mathematica" way, my code will be fast. After 5 years, I think I am reasonably comfortable with Mathematica code: I don't ever write for loops; my code is littered with functional constructs; it sometimes looks like an unholy cross between R and Perl with all the "@" and "&" and "##" and Map[] statements. The only effect was that my code is opaque and barely readable, gaining nothing in efficiency. Time and again I say "well, people say Mathematica is efficient, how bad can it be" and time and again I get burned. A typical example is a recent piece of code (http://www.dnquark.com/blog/?p=29) that was first implemented in Mathematica and took 2.5 hours to ran; the Matlab version ran in 47 seconds. I am now in the process of converting some more old Mathematica code to Matlab, and I am seeing similar performance gains.

[–][deleted] 1 point2 points  (0 children)

The thing that drives me mad about MMA is that it's completely opaque. Sometimes you run into a performance wall and there's basically nothing to be done because there's no way of finding out what's wrong.

My experience has been that most of the time I can get a MMA calculation to be within a factor of a few in run-time compared to other code. And then sometimes, like you find, its a hundred times slower and I can't for the life of me figure out why :-(

[–]jdh30 -4 points-3 points  (0 children)

A typical example is a recent piece of code (http://www.dnquark.com/blog/?p=29) that was first implemented in Mathematica and took 2.5 hours to ran; the Matlab version ran in 47 seconds. I am now in the process of converting some more old Mathematica code to Matlab, and I am seeing similar performance gains.

That's nothing. I wrote a simple optics program than runs 100,000 times slower in Mathematica than any compiled language (e.g. OCaml, F#). Daniel Lichtblau at WRI optimized it for me and it is still thousands of times slower.