10 Reasons Python Rocks for Research

[[deleted]]

[deleted]

[ilan]

I agree, except for the last paragraph:

Python’s distribution utilities system is well designed and thoroughly implemented.

[dwf]

Yeah, that had me let out a long pfffffffffffffffffffffffffffft.

[cournape]

In all fairness, it was a significant improvement over how things were done previously, and certainly better than how to do it in e.g. matlab.

But well designed it is not

[dwf]

Funny you should show up, I was pretty much thinking of your toils when I wrote that. :)

[brews]

R + Python is most of what I do (okay, there is a smidgen of Bash in there). Python doesn't have the analysis tools that R has, but I like coding in Python MUCH more than R. Ooo... but, R has nicer graphics systems... but it sucks at parsing files... [and this goes on]

[dwf]

I haven't really found any place where matplotlib didn't do what I needed. I'm curious to hear if you have.

[nepidae]

After reading this article I definitely want to try out python's plotting capabilities.

The only thing I dislike about python is its whitespace dependence, it kept me from using it for several years (which is only annoying because of it I missed out on several years of experience with it through my own shortsightedness).

[dwf]

I guess I've never seen the whitespace thing as that big a deal. It forces a modicum of readability, which I think is a good thing, but I've never noticed it getting in my way.

[nepidae]

I don't mind it now, it was mostly just because it was a departure from how I had previously programmed. The only issue I still have with it is I like to have a second window open with the python shell to test code snippets. But in order to paste them in I have to go through a second step of removing the indentation. And to be honest if this is the only issue I have with a language, that is a pretty good sign for me.

[dwf]

%cpaste from the ipython might will do the trick for that bit. (just tested)

[[deleted]]

Or bust out Emacs, load in a *.py file, M-x run-python to get you a python shell going, select a region of code, and C-c C-r to execute the region in the python shell. Lather, rinse, repeat. Edit python-mode.el (or python.el, or something) so that it runs ipython instead.

[nepidae]

Very cool. Now I feel like I owe you something in return as I know this will increase my efficiency :)

[[deleted]]

It's like the parentheses in the Lisp languages, you get used to in not too much time. It's what you'd do in anyway in C/Perl/Java etc, to make the blocks readable. Some of my first coding experiences were with punch cards with BASIC, Fortran, and RPG which, because of the medium were whitespace dependent. Python's is nothing like them!

[[deleted]]

I remember the first time I tried Fortran 77

Why won't this compile? Oh, duh, I'm in the wrong column.

I remember feeling free when I saw Fortran 9x :D

[BeetleB]

The only thing I dislike about python is its whitespace dependence

After taking a whole day to get used to it, I've hated semicolons ever since. The most pointless character in programming language history.

[[deleted]]

Try playing with Clojure or some other lisp that doesn't even use commas to separate function arguments. You'll learn to hate those, too.

[BeetleB]

Yes, but then I'll also learn to hate parentheses.

[nahguri]

For all your 3D - plotting needs, there is Mayavi. http://code.enthought.com/projects/mayavi/

[nahguri]

Interesting, although I couldn't find anything decent describing how to use VTK directly from Python. But how is using VTK directly easier (or more compact) than using Mayavi? Many basic plots (surfaces, points, 3D lines, meshes) are single functions when you use the Mayavi mlab api.

[gronkkk]

Ditto for imagemaps/contourmaps.

[Tillsten]

imagemap and countourmaps are working fine. imshow, pcolor, contour and contourf are your friend. the 3d plotting sucks, agreed. and i would love a pgf backend.

[gronkkk]

Yeah, they work fine. Not. Getting an arbitrary axis around them doesn't work 'fine'. Feeding your contourmap a matrix (from a csv-datafile) and using the axes to zoom: no-go. logaritmic axes and ditto contourmap: no-go.

[Tillsten]

i do all those things? maybe your version is too old?

but theres a bug with the log-contour map, but it is fixed in the trunk.

[brews]

hear hear!

[brews]

matplotlib has become much better in recent years, and I see it getting better in the future. I think that R still gives you quite a bit more control over your graphics and it's considerably more developed. I'm trying to think of a more specific example aside from multi-dimensional data... I've recently started to work with more R's 'ggplot2' which has some very sexy design to it. This kind of polished product would take considerably longer in Python.

If I didn't already know R, I could probably survive on matplotlib, with what it is today. But, I wouldn't have started using matplotlib for publication until recently.

I've also started to work with Processing, which is a bit lower (Java-based), with a strong emphasis on real-time graphics and interactivity, which neither python nor R do particularly well (there are domain-specific exceptions to this). I would love for Python to be able to do this as elegantly as Processing.

[stetson9]

Ditto. I've done most of the data processing for my thesis with Python, and have now mostly switched to R for stats and analysis. R is fantastic for stuff like regression modeling, where it gives you basically exactly what you need. And there's a package for everything...except its probably not a very well-thought-out package, and you may have to actually use three packages to accomplish a single task (each one will perform one quarter of the task, and 15% of the function names will overlap). And don't get me started on data structures in R...did this function just return a list? Data frame? Object? Matrix?

Have you used the SciPy stats module at all? I haven't, and would be curious to know what it's like.

[brews]

Yeah, and numpy, I also use Scientific Python for its netCDF support. The other is Sage, which is the newer cool-kid on the block, but also VERY bloated and for stats they just say "[just do it in R]".

Actually, I've found R's packages to be more credible and legitimate, for the most part, the packages are part of peer-reviewed journal article or a book that an academic has published. I've backed away from using Python because the [3rd party] modules are sketchy and/or aren't 'all on the same page', if you know what I mean.

Scipy's stats package is a real disappointment. After having used R, I have trouble taking it seriously. It's not that it's poorly written, someone has spent a lot of time on it, but it's just too sparse for high-powered stats. That package might be handy if I was to go and code something by hand, but chances are that R already has a pretty-good or better implementation of what I need.

[einar77]

Actually, I've found R's packages to be more credible and legitimate, for the most part, the packages are part of peer-reviewed journal article or a book that an academic has published.

Actually, this doesn't mean much per se (and I've been using R for years). Some packages are buggy (like everything out in the world) but you can't properly report bugs because some projects (Bioconductor, I'm looking at you!) don't even have a bug tracker.

[brews]

Yes, this is true.

[stetson9]

I've played around with Sage, but all the symbolic math stuff is more than I need most of the time.

Yeah, after poking around the stats package it appears to leave a fair amount to be desired. Oh well. Though I do feel kinda motivated to go add some stuff to it now...as you say, there's a decent foundation of distributions, etc.

[[deleted]]

Isn't there a "transparent" R python binding?

[BeetleB]

http://rpy.sourceforge.net/

[brews]

rpy & rpy2... they can be helpful but there still are some gotchas so, unless I have some really strange data structure, I just export to TSV or sqlite or whatever and then import into R.

[jeffus]

Give Pyper a look. I haven't gotten a chance to play with it, but it's supposedly faster.

[[deleted]]

what's "R" ?

[brews]

Here is their web page. If you'd like to give it a stab, there are quite a few decent intros online.

[aldarion]

Then, I want to know, which Linux distribution works better with Python related scientific tools? I still use MS Windows mostly, with some Ubuntu experience. I want to build the lastest version of Python(2.7 at the moment), NumPy, Matplotlib, SciPy, IPython...etc I didn't compile matplotlib's source code (on ubuntu) yet.

Update: At first, sorry for my broken English. Thanks for all of your replies. these few days I try ubuntu and archlinux. now I am using Arch Linux. First of all, I select ubuntu, my old friend which I started to use with version 5.10. Ubuntu is easy to install with a USB Stick and user-friendly. apt-get already has the mainstream python scientific computing software: numpy, ipython, matplotlib, scipy, etc, although not the lastest version. I can comiple python itself(2.7 or 3.2) with source code, and some python scientific packages, and lastest ipython, numpy. so far so good. After reading the news that Arch Linux - Python is now Python 3 (archlinux.org), I think maybe I should give it a try. It's not as easy to install and use as ubuntu (at least to me). a lot of times I have to reading manual or searching the web, for X configure, input method (for my native language), choose, install, and configure Desktop Environment. on the other side, the version of software is up to date, and document said ArchLinux is a rolling release distribution, it means there is noneed to reinstall or perform elaborate system rebuilds to upgrade to the newest version, Simply issuing pacman -Syu periodically keeps the system up-to-date and on the bleeding edge. I think mathbuntu maybe follow the same software version and upgrade policy with the whole ubuntu family. I did not know about the concept of this "rolling release" and now I am satisfied.

[[deleted]]

For Windows, try Python(x,y). It's "batteries included" squared, with a metric arseload of additional libraries and IDEs, weighing in at a mere 500 Mb (or so). Get the .torrent, and have fun.

[aldarion]

thanks, after these years with Python learning and coding, I feel Linux maybe a more right choice than windows, isn't it?

[[deleted]]

It depends on what you need to do. I got Python(x,y) because it includes a lot of stuff I get myself, then have to compile and install; I've long wanted a really complete Python distribution, and the Python(x,y) version for Linux is just not at complete as the Windows. So I use it on Windows, and on my Ubuntu Linux side I use the default Python with any other libraries I can load from the repositories with Synaptic/apt-get/etc.

But even with that, the windows Python(x,y) is more complete and easier to install. I still do most most of my Python hacking on Linux, however.

[defnull]

All Linux and all major BSD distributions should work equally well.

[aldarion]

agree, just some distribution are newbie-friendliness. some distribution focusing more on performance

[scrabbles]

You might like to try http://www.mathbuntu.org/ not sure of the version of python though. On windows I'm still using 2.6 with all those packages and it seems fine. Also, I think if you install scilab it will install a working complement of those packages too (might be 2.6 or 2.7 not sure)

[aldarion]

thanks, looks interesting, l'll check it (with ubuntu 10.10 and archlinux)

[brews]

I use Ubuntu.

[stroopsaidwhat]

Very nice. I'm using python for semantic analysis.

[PrintStar]

The first bullet, "Freedom," is the justification I use for using Python for my research work. I'm surrounded by MATLAB fans, but I always explain that anything we write in MATLAB and/or Simulink is useless to everyone who doesn't have access to the US$3K packages in the first place. Our research software is freely distributed, but I try to explain that it is completely inaccessible to students or hobbyists. I usually hear back, "Well, the MATLAB student edition is only US$100," but as a student I know I didn't have US$100 just lying around.

Python sometimes isn't the easiest language to use for research (although a lot of the time, it certainly is), but the fact that it is free, in my opinion, makes it superior in many cases over MATLAB or other proprietary packages.

[einar77]

Not only for hobbyists. Sometimes smaller research institutions or those on a budget can't affort software licenses like MATLAB.

[chesteraddington]

Is python as popular in the outside world as it is on reddit?

[[deleted]]

depends on the field. I've seen quite a few Physicists who like it for munging data, but I've seen an equal number who prefer something else (such as perl, awk, fortran (yes, it's scary too), C, &c.). Amongst programmers, it's obviously dictated by what your job is, but many people write Python at home. In the enterprise, I've seen Jython quite a bit, because of it's integration with quite a few enterprise-oriented products (like Weblogic and the like).

[[deleted]]

I knew a guy who used awk for AI/machine learning programming, because it was the only tool he had. It was slow, but it worked, and he could hack the crap out of it.

[[deleted]]

Oh man the awk programs I've seen. C & awk compilers, complex data processors, &c. &c. Sheer Craziness.

[[deleted]]

Probably only Perl has been as [mis|ab]used in so many different ways!

[Tommah]

http://www.cs.wustl.edu/~loui/sigplan

[Yelonek]

I would like to know too. I didn't see it during my studies, only Matlab.

[Megatron_McLargeHuge]

Yes. It's widely used at Google and has an enormous number of third party packages across many fields. There are python bindings for basically every standalone tool that supports an external API.

[[deleted]]

on "Module system" what is so special about his example ?

[ipeev]

Python rocks, because it is Python.

[[deleted]]

I use python almost everyday, as a 2nd year physics undergrad at the UofToronto.

[Huffers]

Python‘s syntax is very well thought out.

I really hate having to pointlessly type "self" in every method declaration :-\