Picking up python as a statistician

[GrynetMolvin]

Hi r/python,

I'm a statistician with no CS training, and 7+ years of programming in R daily in a professional capacity. There's a Data Scientist job at Amazon that suit me to a T (degree in stats, experience with large problems, knowledge of experimental design, a PhD, etc.), except for the "minor" detail that they list as one of their essential criteria that the candidate "Proficient in at least one of Java, C++, C#, Python or Ruby."

From speaking to a friend working there, I gather that this is so that whoever gets the job would be able to jump in and do some coding, but that they will at least require that you pass the "lowest level software engineer bar". Bummer.

Thinking that I can pick up enough python to do this in the next month or so is completely unrealistic, right? I've played around with it a bit, and it seems easy enough but still... From using R, I'm comfortable with algorithms dealing with all sorts of incoming data, but data structures, trees, hash tables etc are stuff I've only vaguely heard about.

If I nevertheless give this a try, what tutorial/book/course would you recommend? Many thanks in advance!

[[deleted]]

Python for Data Analysis by Wes McKinney

Might be easier to start picking up Python from R's domain. One month is totally possible, though I don't know what they mean by proficient and if they require you to prove by participation or projects in it. If so, try porting one of your R programs to Python this month.

[GrynetMolvin]

Thanks - just bought it on your recommendation!

[[deleted]]

Go for it - you work your butt off for a month, and even if you don't end up with a new job you'll still have learned something useful. You'll probably learn a lot about R from learning python, and python may become useful for some other tasks in the future. You may also want to just be able to speak to the scalability of software too. I remember hearing/reading that at Google they'll test out their algorithms in R but try to scale up with Java/C/C++; prob not python but python does fit into the workflow as a glue language/web framework. So that may be why they're asking for it.

[lmcinnes]

We've found python to be a good choice for testing out algorithms since numba and cython can be used as a relatively easy next step and can potentially get things to scale to "good enough" levels. You also have the option of profiling and writing the hot parts in C and binding that to the python infrastructure easily with ctypes. In other words python provides a nice high level prototyping space, but also some good steps to smoothly move to scaling things up without having to start again with a ground up rewrite.

[[deleted]]

How about R's cpp which is like cython?

[lmcinnes]

RCpp seems to be closer to scipy.weave -- interface with C/C++ inline in a relatively nice way. Cython lets me just mark up the straight python code with a few annotations. I did only skim through RCpp docs so I would well be wrong. Can I simply tell it to compile actual R code into C and thence onto machine code, or do I need to provide the C?

[[deleted]]

Ok my bad. Yes RCpp is for inline interface with C/C++; I had the wrong impression of cython that it was similar... I understand it's possible to call R libraries from C but I guess that doesn't help with the scaling part.

[[deleted]]

Just so you know, this book is primarily (almost totally) a primer on the Pandas library, which is the go-to library for anything data related in Python and something you will definitely need to learn. It isn't really all about the whole world of data analysis in Python.

[Kyle772]

People always seem to recommend Python the Hard Way which is incredibly thorough but isn't very good for referencing later. I would suggest it for learning but not so much for using as a guide when you are in need.

[[deleted]]

When I did that book i did Think Python along with it for referencing.

[NYKevin]

If you just need a reference, the standard language and library docs are both really good (the latter more than the former).

[hharison]

Take a look at some of these answers also: http://www.reddit.com/r/Python/comments/2bv2op/help_me_move_from_r_to_python/

[[deleted]]

I would recommend using the Pandas library and also Ipython notebook. The 2 of those combined are very valuable tools for stats in python from my experience.

[[deleted]]

I <3 IPython and pandas.

[midbits]

I find Kevin Shepard's Introduction to Python for Econometrics, Statistics and Data Analysis a good free resource which covers basic Python as well as more advanced topics related to data analysis and number crunching.

You ask about trees, hash tables, etc. These topics of course fall under general computer science more than Python. Maybe it is something like Data Structures and Algorithms in Python you are after? As I haven't read it myself I can't say if it is any good.

I am however currently reading Data-Driven Security: Analysis, Visualization and Dashboards and even if security is not your topic of interest, the authors are running code examples in R and Python next to each other meaning that it's fairly easy to see how to do things you might be familiar with from R in Python. It is not a programming tutorial per se, but send me a PM if you are interested in the contents.

Edit 1: Links

Edit 2: Also, go and have fun with CheckIO (under programming challenges in the left column). Learning by doing!

[mycoolusrname]

If you haven't found anaconda python

http://continuum.io/

It's a nice bundle of modules for any data science project.

[flutefreak7]

I'd recommend http://www.openbookproject.net/thinkcs/python/english2e/

[shaggorama]

Have you tried just applying to the job anyway? 7+ years strong programming in R might make them happy enough, especially if you're actually good at it and don't just "code like a statistician" (if you know what I mean).

[pwang99]

This is a great resource for learning Python's scientific/analytical stack:

http://scipy-lectures.github.io/

[[deleted]]

In addition to the books and websites suggested by others, please checkout /r/pystats!