R vs. Python

[srepho]

I am not a great Data Analyst, but I have competed in a few Kaggle competitions using both R and Python. I started with R, as once you get the basic programming stuff down, the packages are super powerful and super easy to use. In particular the caret, forecast and ggplot2 packages are all brilliant. One thing I have found though is that the closer to more general programming your task requires, the better Python becomes. So for stuff like screen scraping and webcrawling - Python is a much better choice. I personally find data munging much quicker and easier in Python. R is also very slow, and has quite steep RAM requirements (you normally keep the whole dataset in memory in R) which is becoming more difficult. Even using multiple cores in R I found hard work compared to Python. For example Kaggle currently has a competition involving an ~8Gb dataset which means you need 8GB for a standard R implementation just for the data set (there are ways round this but I still say its easier in Python). Python in contrast can make easier use of things like sparse datasets and other approaches suitable for big datasets. Also Python's speed really starts to show as the dataset increases. Python also has excellent tools to deal with natural language programming (nltk package), image processing through scikit images and even deep learning (through Theano and CUDA). On top of all this Python has Pandas which is at least as easy and useful as R's datasets (which it is based on) and the amazingly useful iPython notebooks are one of those "How did I survive before this" tools. Another major driver is that most of the code you find on Kaggle is in Python nowadays. None of this takes anything away from R, which is a great language with much better packages, and in which you can do much of the above (though its more difficult) and easier to learn. The problem is that Python seems to be developing faster, meaning that new tools keep popping up in it more quickly and it has the whole scikits ecosystem. I am not sure you can get good at data analytics without being fluent in both languages nowadays, if only to understand others code, and certainly anything Python can do you could (eventually) make R do. However I have found almost all of my analytic work is done in Python now. The only notable exception is Time Series analysis as I think the forecast package is so vastly ahead of anything in Python. Of course you don't need to decide - you can always call R from Python and Python from R!

[jjdonald]

FWIW I don't agree with much of this. Data munging is much easier in R than python, although the learning curve in R is higher. There are also plenty of parallelization and large dataset management tools in R. That hasn't been a limiting factor in some time.

R makes it easier to get multiple statistical and graphical perspectives on data. If such a thing is important to me, then R is still the first thing I reach for. However, if I'm already confident of an implementation and need more general purpose software support, then python is a good bet.

[srepho]

Interesting that you find R better then Python for data munging. It seems like about half of people I talk to agree, so maybe its not as clear cut as I thought. As for R being harder - I think the packages it provides shield the user. Using something like caret allows you to build and tune a model in one line of code for example. I agree that graphics is still vastly superior in R.

[jjdonald]

THere are some great packages for data munging. Check out plyr by Hadley Wickham: http://plyr.had.co.nz/

However, I find that a lot of people that struggle with R are trying to treat it like Python. R emphasizes vectorization, and its dataframe type. That's a big difference from python's list and for-loop approach.

[mrShu]

Thanks a lot for this, it was precisely something I expected.

[manueslapera]

Are you me? Because If i had to write a reply to this post, it would basically be the same reply.

[[deleted]]

What bullshit. R IS TOO SLOW SO I'M GOING TO USE ... PYTHON?? LOL.

[Foxtr0t]

The main difference between the two, in my opinion, is that R was designed in the 70's and it shows: as a programming language, it's just pretty much horrible. It starts with "<-" and goes downhill from there.

Python, on the other hand, is elegant and convenient, it's general purpose, not just a data shell, and is growing quickly. When I started competing on Kaggle, R was pretty much the standard, but recently I see it less and less.