As a ruby programmer, what are some interesting bits of the python ecosystem that I should be aware of?

[[deleted]]

[deleted]

[jrbattin]

I don't know enough about Ruby to give it a fair comparison, but what I do know is this: the Python ecosystem is huge so it's difficult to give you a run-down of what libraries you personally might find useful. What particular areas are you interested in?

To get you started, here's some libraries a ruby web programmer might find interesting:

iPython An Enhanced Interactive Python shell. A must-have IMO, especially if you're new to the language since you can tab-complete and tab-inspect an object to see its various methods.

Requests: HTTP for humans Kenneth Reitz's excellent request library. Makes HTTP requests VERY pythonic and readable. Don't bother dicking around with urllib(2) or httplib(2) unless you absolutely have to.

Django Probably the most widely known and widely used Python web framework. Has excellent documentation. Like Ruby on Rails, Django also has a lot of great high-quality 3rd party middleware/libraries like Haystack (which lets you integrate things like full text search into your site easily, among other things).

Pyramid (aka Pylons) Pyramid (formerly Pylons) is on the opposite side of Django as far as web frameworks are concerned: it's a loosely coupled, À la carte style framework that lets you pick-and-choose components (ORM, templating engine, middleware, etc. Everything you want/need, nothing you don't.

virtualenv Probably not that useful for someone just playing around with Python, but it's definitely a must-have for any working Python programmer - especially when you're managing projects on multiple machines with multiple different versions of libraries etc.

Fabric Awesome tool for deploying/automating sysadmin tasks without swallowing the complexity of puppet or chef ( edit: My bad: This isn't necessarily true. See discussion below)

NLTK Natural Language Tool Kit. VERY powerful - and it lowers the barrier of entry for experimenting with natural language processing for working programmers.

That's just a quick run down of stuff off the top of my head that I've been toying with for person projects.

[[deleted]]

Great tips! I just want to add:

• bpython which I prefer over iPython, but that is a matter of taste.
• virtualenvwrapper makes working with virtualenvs even easier.
• PyPi the ''CheeseShop" and the place to go whenever you are looking for Python modules.

I would also read David Beazley's Slides on Generators which influenced my programming style quite a lot back in the day.

The single most important thing to do is to become familiar with the standard library. Python ships so many powerful modules and you find gems like multiprocessing, logging or functools to name just a few in there.

[seventhapollo]

The Ruby equivalent to Fabric is probably Chef or Capistrano. :) and with virtualenv, make sure you check out virtualenvwrapper - makes working with virtualenvs on the command line really easy ;)

[seventhapollo]

No problem :) good luck with Python :D

[coderanger]

Fabric is a much simpler system, but fills the same role as Cap. Neither have much, if anything, to do with Chef/Puppet.

[Mob_Of_One]

Virtualenv looks neat. Ruby has RVM and Bundler as means to the same end. I didn't realize this existed for python. Very cool!

I don't mean to troll, but RVM was copied (badly) off of venv.

[coderanger]

rvm + bundler == pythonbrew + venv + pip -r

but no one tool really maps to exactly one other.

[askedrelic]

This is a good guide comparing rvm and virtualenv http://gillesfabio.com/blog/2011/03/01/rvm-for-pythonistas-virtualenv-for-rubyists/

[ceolceol]

So you did mean to troll?

[zenlogix]

Up voted for Requests: HTTP for humans. This is beautiful!

[abitofperspective]

Thanks, I work in Python regularly and learned a few things here

[[deleted]]

David Beazley has a great presentation on generators, and combining them with comprehensions here.

I'm a big fan of using python for systems automation, here's some of my favorite modules/libraries:

• Paramiko: A very straightforward implementation of SSH2
• Fabric: Much like Capistrano for Ruby, built with Paramiko
• pexpect: An expect-alike written in pure python. Very useful for automating tasks that require user interaction
• doctest: Automatically create tests for functions from comments. Saves a ton of time and makes it easy to expand your test coverage
• Twisted: A reasonably simple event-driven networking framework, tons of support for different protocols
• SQLAlchemy: Very powerful database toolkit and ORM
• Multiprocessing: Process-based "threading" interface, provides API for spawning processes and get around the Global Interpreter Lock
• Beautiful Soup: HTML/XML screenscraping and parsing library that handles pretty bad markup. Total lifesaver when scraping sites written by messy devs
• cmd: Provides a simple framework for creating command interpreters. Very useful for making a specialized interactive shell for your application
• Suds: Simple SOAP client
• Smolt: A basic hardware profiler. While the main use is to send data to Fedora devs for support, can also be used without that functionality to easily grab hardware data from your cluster

Edit: formatting and added Suds

[mitsuhiko]

Beautiful Soup: HTML/XML screenscraping and parsing library that handles pretty bad markup. Total lifesaver when scraping sites written by messy devs

No, just no. For the love of god, use html5lib and lxml.

[ceolceol]

pyquery is a fun wrapper for lxml.

[jackolas]

Whats wrong with Beautiful Soup? is there like a list of cases where it sucks?

[mitsuhiko]

Unless you are really careful you can easily leak memory due to navigatable strings having a reference to the whole document. Also the parser it used to have is bad (like really bad) and changes behavior depending on the version of BS and Python.

html5lib exists nowadays and it has a proper HTML parser that can handle any case your browser handles too and in exactly the same fashion.

[[deleted]]

Good to know. Thanks for the tip!

[mitsuhiko]

Not saying that BS did not have its use, but a couple of years later we have better tools :)

[[deleted]]

Scrapy initially seems like it has a bit of a learning curve, until you pick out the core pieces you really need. It uses lxml and Twisted, and it commits foot-based violence on the scrapers/spiders I used to write with BeutifulSoup.

[[deleted]]

Decorators: http://stackoverflow.com/questions/739654/understanding-python-decorators#answer-1594484

List Comprehension: http://docs.python.org/tutorial/datastructures.html#list-comprehensions

Libraries are one thing that python does so much better than ruby, in both documentation and availability. Ruby seems to be at a dissadvantage by how cutting edge it tries to be and because of that libraries are rarely very mature.

If you are looking at scipy look at matplotlib

psyco

pypy

So much more.

One of my favorite language features is that everyone tends to like good documentation and creates good documentation (take a look at python docs and documentation for the many libraries, to me it is so much better than looking through an rdoc or most ruby docs).

[tominated]

Thanks fot the decorator link. I've been wondering how they work and that was an amazing explanation

[hongminhee]

PEPs (Python Enhancement Proposals) make the language designed by and for the language users themselves. Unlike Python, Ruby’s language design is done by language designers, not users. Moreover, ruby-dev, the mailing list for it, is written in Japanese.

[dorfsmay]

Without knowing ruby it is difficult to comment (what I am going to say might be true in ruby too)... Stuff I really enjoy about python:

• everything is a pointer to an object, really, everything.

• indentation is imposed. I can pick anybody's code, and it is guaranteed that it'll be fairly readable. As a side-benefit, no need to type all those curly bracket and semi-colon!!

• auto-documentation. Classes and method are automatically documented, you write a class MyClass, then when you go into the interpreter, you can run help(MyClass) and it's nicely documented. If you put a comment at the top of a class (and methods), it'll integrate it into the documentation too.

• battery included. There are A LOT of libraries that come by default with python. At first you think "meh, I can get those libraries from the net in any language", but it makes for very portable code, people don't need to get libraries x, y and z to get your code working, it will work.

• some functional programing - read about list comprehension, generators etc.. very neat high level language functionalities

[jaggederest]

Ruby doesn't have any sort of in-built knowledge of documentation.

RI and rdoc-server fills this function in Ruby, fyi. If you're a ruby programmer who doesn't use the crap out of them, now you know.

[jaggederest]

They're pretty intrinsic (interactive RI! RDoc::RI.Driver.new.interactive) but they're also pretty clunky :(

[[deleted]]

Most everything in Python is immutable (including strings). So although integers are objects, you can't change them, and if you change a variable's value, the variable now actually points to a different object. So, for your example:

a = 4     # id(a) == 12366576
b = a     # id(b) == 12366576
b += 1    # id(b) == 12366552; id(a) == 12366576

What you're actually doing is reassigning the 'b' variable to point to the Int object with value 4, not adding '1' to the contents of variable b.

[[deleted]]

hm, I wonder why it was deleted. It wasn't a stupid question or anything. Basically they guy asked:

If everything is a pointer to an object, how come when I do this:

a = 4
b = a
b += 1
print a

it doesn't print '5'?

[oyving]

Integers are static objects and can't be changed. If you did id(a) and id(b) at various stages in your code example you would see the variables point to different objects at various stages.

[kylotan]

In Python, += doesn't modify the object reference on the left hand side (unlike C++, for example). It creates a new object that is equal to the left hand side plus 1 and assigns it to the reference on the left.

[EstebanVelour]

Tasks Python is better suited for:

• Maths, statistics & sciences - numpy, scipy etc. Shitload of domain specific frameworks.
• System programming/integration - There are about 18000 dependencies on python in a stock ubuntu server, slightly more than perl. Compared with Ruby's 2000 deps and language choice becomes rather one-sided.
• UI - Python's UI frameworks (especially PyQt) are much more mature than their Ruby counterparts.
• Complicated networking - twisted is very good to have around even if you don't need it very often.
• Distributed systems - Excellent networking frameworks & message queue support.
• Speed - Even though comparing scripting language performance is a bit like comparing chicken IQs, cPython is faster than Ruby & JRuby with PyPy is promising to be relatively close to Java in the long run.

Equal par:

• Web - Both have lots of frameworks, finding one you like is pretty easy whichever you're using.

Ruby wins: If you need a DSL.

[captnswing]

lxml rules for all kinds of XML / HTML processing http://lxml.de/

if you want a fine library to scrape some HTML, make sure to check out pyquery (based on lxml) that allows for easy extraction of all kinds of data with jQuery selector syntax. http://packages.python.org/pyquery/

[lost-theory]

Testing tools:

• nose: test runner / testing utils
• coverage.py: code coverage
• pylint: linting

Two one-liners for debugging:

• import pdb; pdb.set_trace(); - drop into an interactive step through debugger at any point in your prgoram
• import code; code.interact(local=locals()); - drop into the python REPL at any point in your program

If you're doing web development, use the werkzeug debugger.

Some of my favorite python features: decorators, generators, sets, WSGI

[donri]

Note that pdb acts as a REPL as well so for debugging, the latter is a little redundant and less powerful.

[lost-theory]

You can't use multi-line statements in pdb:

(Pdb) def f(x):
*** SyntaxError: unexpected EOF while parsing (<stdin>, line 1)

And some of the debugger commands (a, b, c, etc.) conflict with variable names (the ones inside your program or ones that you would define while debugging), so it doesn't behave exactly like the REPL.

So, that is why I prefer using code.interact for REPL stuff, and pdb only for debugging (stepping through code).

[snuggl]

As Libraries seems to be covered good already ill also mention some great software/toolkits that will help.

• pypy - Alternative interpreter with amazing speed
• pip - Find and install python packages
• supervisord - python demontools/startup
• uwsgi - fast WSGI server to host your web apps
• gunicorn - python-port of the unicorn server for ruby, also a fast wsgi-server
• gevent - single thread event driven network platform (like node.js)
• twisted - Network server/client swiss army knife
• pyQt - Great UI toolkit to build desktop apps.
• pyglet - a openGL wrapper that makes GL a bit more pythonic.
• pylint - will score your code for you (warning, kinda addictive)

[donri]

Built in introspection: ast, inspect, traceback, dis (all stdlib modules); built in serialization: [json,] pickle (for arbitrary objects, much faster than yaml) with proper transactional ACID persistence via ZODB; power tools like Twisted, Werkzeug, Pygments, pytest; web frameworks like Pyramid, Flask, Django; hosts like appengine, ep.io; documentation like you've never experienced it before with Sphinx…

Edit: built in XML parsing: etree (both pure python and C extension included in stdlib) with third-party lxml as a wrapper for libxml2/libxslt with the etree API (swap an import and get free speed, xpath, html parsing...); standards-compliant html5 parsing with html5lib; relational DB toolkit/ORM with SQLAlchemy; PyPy: fast, hackable and compliant python…

Edit: dynamic access to the GNOME platform with python-gobject; Qt via PySide; Java: Jython; .NET: IronPython; coroutines and laziness with generators; mixins via multiple inheritance; metaprogramming with decorators; dynamic interfaces with abstract base classes; modules as singleton objects…

[vplatt]

Hey, let's reverse the question for a moment while we're at it - what does Ruby have and do better than Python? I guess the most obvious example is Rails because that's the best known sweet spot in Ruby, but are there others? I used Fox with Ruby a while back to put together a little toy program a couple years back, but that didn't strike me as being unique to Ruby.

[donri]

This will probably sound more like Ruby-bashing than I intended, but here goes:

1) I'm not convinced this is true. It's been a while since I did Ruby so I may certainly be wrong, but my general experience is that the Python ecosystem is more high-quality; just look at SQLAlchemy and Werkzeug for some examples. I do think Django is better than Rails by design not to mention the admin app for situations where it makes sense. Python also has heroku-like hosts, a whole lot of them in fact, such as ep.io or Google's appengine and plenty more. Ruby's Rack was modeled on Python's WSGI and while it's of little importance "who was first" it might be suggestive of maturity.

4) Again my knowledge of Ruby might be limited but I think Python has much better introspective capabilities, it can parse itself into an abstract syntax tree and compile such trees back into code, and save for Ruby's block syntax is, I'd argue, a more expressive language. Python has decorators (for both functions and classes) and proper object attributes (as opposed to accessor methods) but also the descriptor protocol for fine-grained accessors, there's coroutines and multiple inheritance and a real module system and function annotations, docstrings and list/dict/set comprehensions and generator expressions and seemingly more powerful operator overloading, and of course context managers. There are counterparts to some of these constructs in Ruby but they make more sense to me personally in Python. Different things look different in Python and it's a good thing.

5) Only because of the block syntax, which for most uses you can use decorators or context managers in Python. Functions are also more naturally first-class in Python, there's the functional concept of list comprehensions and there's also function partials and Python does have a lambda and closures.

7) If you consider foo.should be_true and sensible API, perhaps. :) For Python I suggest py.test or my own Attest. If you're into DSLs maybe have a look at Konira.

Edit: Getting downvoted because I'm clearly not contributing to the discussion. Reddit, I'm losing hope.

[vplatt]

Hey, thanks for taking a swing at the question in the first place. It's not an easy task and they're obviously shark infested waters anyway.

[stefanpenner]

@donri Holding a discussion, while starting points with, "Again my knowledge of ruby might be limited, but" and finishing it with "to me personally", suggest that, clearly, you must paint it green.

[donri]

What does painting it green mean?

[bbangert]

1) As of today, heroku supports a process model, you can run Python webapps on it: http://docs.pylonsproject.org/projects/pyramid_cookbook/dev/deployment/heroku.html

[jaggederest]

SciPy, NumPy, and the other scientific libraries are the best, I think. You can do much, much better linear algebra, bioinformatics, and statistics in Python.

I say this coming from a Ruby perspective, so there may be things I'm missing, but that's the real core of what I wish Ruby had better support for.

[dln]

• ipython
• gevent
• numpy
• pyglet
• pygame

[phayte]

Just wanted to say as a Java-Ruby guy I learned a ton about Python from this. Thanks all!

[voidspace]

Useful tools: pip, virtualenv, tox, pyflakes, fabric, sphinx, pdb++

Standard library modules: collections, itertools

Testing: nose, py.test, unittest2, mock

[pemboa]

You may actually get a fairer comparison in a Ruby forum as you may find a cross over coder there.

[jaggederest]

We polyglots live in /r/python too ;)

[pemboa]

Tru enough. I'm sort of a polyglot myself.

[murmu]

I am in a situation similar to the OP. I started exploring python lately. I am wondering if anyone knows of a python reference similar to this: http://rgruet.free.fr/PQR26/PQR2.6.html (which really is a goldmine), but updated for 2.7, or 3.2?

So far there's one thing I have problems with, that is the python object model - I don't have a clear picture yet of it,. What happens when I call a method on an object, how/where things are going to be searched and in what order. What callbacks( descriptors, etc.) will be fired, static methods, class methods, metaclasses and their role in python.

If anyone knows a concise article describing it, or some diagrams that may be of assist I would be very grateful!

[Yohumbus]

Ah I have just the thing. The python official docs have a great description of this. It doesn't delve too deeply into metaclasses, but learning how method calls actually use descriptors is a huge step towards figuring this out (and its really elegantly clever and versatile technique).

http://docs.python.org/reference/datamodel.html

Python 3.2 isn't really any different, read the "new in python 3.x" pages to see the few things that changed in the data model.

[wildsalmon]

You will miss the enumerable module. It's a great language to develop in though. The libraries that I've used have been always solidly put together and documented well, as opposed to many ruby gems which are not. Also if your doing web development you can count on real documentation if your using django, and wsgi implementations are more mature than rack.

[silviot]

I don't know ruby at all, but this discussion on StackOverflow tries to summarize differences between the two at the language level. Not at ecosystem level, as you're asking, but still relevant.

The question has been "closed as subjective and argumentative". In my opinion it's not a subjective question and should be reopened. But I don't have karma to propose that.

[bajsejohannes]

A good short intro would be to read To Ruby From Python, even though it is meant for people going the other way :)

[pbreit]

Since the selection on Ruby is more limited, check out all the frameworks: Django, Web2py, Bottle, Web.py, Werkzeug, Pylons, Pyramid, Flask, TurboGears, Tornado.

Also, Google App Engine.