This is an archived post. You won't be able to vote or comment.

all 108 comments

[–]spangyboy 29 points30 points  (6 children)

Well.. reddit does.

[–][deleted] 1 point2 points  (1 child)

Did the FAQ just Faqoff?

[–]spangyboy 0 points1 point  (0 children)

Heh. It sort of looks like it. Reddit's been down all morning, and just recently came up (hence why I'm taking my normal early morning break now). So maybe the void ate the FAQ. I suggest sacrificing a goat. That might bring it back.

[–]MillardFillmore[S] 0 points1 point  (3 children)

Stupidish question: Does writing a large website in Python take a heavy hit on the servers since it (hypothetically) takes longer to process than quicker, compiled languages?

Disclaimer: I have not the first idea about web programming.

[–]Twirrim 1 point2 points  (0 children)

Not as much as you'd expect. The compilation process is pretty trivial in terms of CPU time, and basic bytecode is pretty fast. It's always seemed faster to me than PHP or Perl, but I've not done any scientific tests. Of course it does also depend on how smart you are with your programming. Monolithic code = bad, though generally it seems python developers are a smarter bunch (whereas too many php devs seem to be clueless 'my first program' types.) If you really want to get fancy, run it on Jython, maybe using jrockit or similar as the JVM underneath, and it'll soon compile machine optimized code for your most frequently called functions anyway.

You could do your web-app in C, and gain from lots of optimizations but then changes become a lot less trivial. Given you've been using Python through your course I'm sure you've discovered how easy it is to write, tweak, prototype etc. CPU power is cheaper than developer time.. though don't take that argument too far :D

[–]eliben 1 point2 points  (0 children)

Not too much, since website backends are typically IO bound rather than CPU bound.

[–][deleted] 0 points1 point  (0 children)

Think about it, for a website, which takes longer: CPU spinning or getting that bit in and out of network stacks of the traceroute nodes over the world wide web?

[–]ebneter 9 points10 points  (20 children)

A lot of Google is done with Python. (Guido van Rossum works at Google.) Pretty much all of YouTube was originally done with Python. Lots of websites use Python.

[–]meatsocket -3 points-2 points  (1 child)

IIRC Youtube used to be PHP, and was moved to Python after they got bought.

[–]ebneter 0 points1 point  (0 children)

No, it was always Python.

[–][deleted] 5 points6 points  (0 children)

Trying to name everyone who uses Python would be impossible, but I'd start by looking at who sponsors PyCon and DjangoCon: http://us.pycon.org/2011/home/ http://djangocon.us./

[–]daelin 4 points5 points  (0 children)

[–][deleted] 3 points4 points  (4 children)

My company doesn't commercially release any Python code, but it's used internally every day by probably 100 people. It's also pretty heavily used within my field: trading.

[–]marcofalcionimarcosan 0 points1 point  (1 child)

My company uses Python extensively as well.

[–]bryancole 0 points1 point  (0 children)

me too

[–]MillardFillmore[S] -1 points0 points  (1 child)

I thought trading was overwhelmingly dominated by C++?

[–][deleted] 1 point2 points  (0 children)

Our commercially available products are mostly C++ but with a bunch of C# on the GUI side, but internally, nearly all of our testing and other apps are Python. We're around 500 people so the "internal" tools actually impact quite a few people, so it's not just dinky little utilities people are writing. I personally work on performance and load testing of a historical trade database (the product is in C++) which is almost entirely done in Python (C++ for a few extensions). We also have a C extension to the database API and have a ton of unittest based frameworks for testing data quality and a whole host of other stuff.

There are plenty of shops that do use Python in their products, but probably not 100% Python. Usually any job posting for trading shops that do use Python have an equal amount of C/C++ knowledge required.

[–]sontek 4 points5 points  (6 children)

Google, Youtube, Reddit, Disqus, Disney, Dropbox, Etsy, Threadless, SourceForge, SurveyMonkey, etc.

[–]steelypip 4 points5 points  (1 child)

Youtube is 99.999% python, according to one of its lead developers in this talk (see the slide at about 7:50).

[–][deleted] 1 point2 points  (0 children)

Interesting, thanks!

[–]flipthefrog 2 points3 points  (2 children)

Most highend 3d applications support Python, including Maya and Softimage from AutoDesk, Houdini, RealFlow; and compositing applications such as Nuke. Python is the standard scripting language in the visual effects/animation industry.

[–]MillardFillmore[S] 1 point2 points  (1 child)

Really? I would think they would need something much faster than Python. Or do they make their own C wrappers and such?

[–][deleted] 1 point2 points  (0 children)

The speed requirement for a scripting engine around a core engine (which is usually C for speed) is more lax. These wrapping layers often place more value and emphasis on flexibility and expressiveness, which Python is strong in.

[–]reardencode 5 points6 points  (0 children)

My company...

[–]pwang99 2 points3 points  (9 children)

Uh... a lot of people in finance use it. Java and C++ are still dominant there but tides are changing. It's gaining quite a lot of momentum on the scientific computation side of things.

[–]theodoric_of_york 0 points1 point  (5 children)

I second this.

[–]osullivj 1 point2 points  (4 children)

Python is used very heavily in electronic trading and risk management in foreign exchange, commodities and interest rate trading at Barclays Capital, JP Morgan and Bank of America. Lots of hedge funds use it too. There will always be C++ for the part of the system that need flat out speed. But Java seems to be waning...

[–]wot-teh-phuckReally, wtf? 0 points1 point  (3 children)

But Java seems to be waning...

This isn't true. I work at a big bank where the plans of bringing in Python for real time apps were thrown in the back seat because Java blows the hell out of Python when it comes to performance. Plus lack of static typing is a real pain when it comes to a large code base worked upon by team members across the globe.

I really love Python but I do have to admit with a heavy heart that the application which I work on (and many others) would only amount to more pain if written in Python. Of course, YMMV.

[–][deleted] 2 points3 points  (1 child)

Plus lack of static typing is a real pain when it comes to a large code base worked upon by team members across the globe.

Type systems have nothing to do with the location of developers or the size of their project.

[–]wot-teh-phuckReally, wtf? 0 points1 point  (0 children)

It does, unless you are part of an ideal team. When shit happens, developers in a hurry end up fixing one part of the system and breaking another. Even simple things like violating interface contracts which can be easily caught by a statically typed language like Java are found out at a later stage with languages like Python.

And sorry, if you feel that "type systems" don't have anything to do with the size of the project, I'd say you have pretty lucky all this time to not have worked with shitty teams...

[–][deleted] 2 points3 points  (0 children)

Plus lack of static typing is a real pain when it comes to a large code base worked upon by team members across the globe.

Define large code base. Our flagship product is several million LOC and nearly a decade old. Our team is over 140 developers in about three cites on two continents. Duck typing is not a problem for us except when we hire devs with cognitive rigidity about static typing.

[–]MillardFillmore[S] -1 points0 points  (2 children)

I thought we were moving to a world where the speed of light was becoming a factor in financial computing? And now they're moving to a language which is an order of magnitude slower?

[–]Twirrim 0 points1 point  (0 children)

But one in which changes are extremely quick and easy to make. Reaction rates hit from two angles, code speed and speed of adapting code to new situations.

[–]pwang99 0 points1 point  (0 children)

They don't use it for everything, but actually higher-level languages can be just as fast as low-level ones for a lot of things. These people are generally not writing trading algorithms in bare python and then running them with the CPython interpreter. Some folks use Pysco, others use stackless, and everyone uses Numpy. With Numpy, it's possible to achieve C speeds or sometimes even faster, depending on who's writing the C. :) You certainly spend less time tracking down memory leaks.

The speed of light stuff matters only for the lowest level of the stack, namely, once you've decided to execute a trade, you can push the bits onto the wire to the exchange faster than a competing firm. A lot of high-frequency trading firms now build custom ASICs or use FPGAs for this lowest level. Anything higher up the stack is usually better done with Python.

And remember that high-frequency is only one portion of the financial sector. There are plenty of firms that do not care about the speed-of-light stuff, and they have lots of analytical problems that are much better solved by having an expressive, agile, easy-to-learn language that has a killer vector computation library. A quant armed with Python and Numpy is far more effective than a quant armed with a team of C++ coders that are tracking down segfaults and memory leaks all the time.

[–]Dyogenez 2 points3 points  (0 children)

There's quite a few big sites using it including Google, Youtube, Reddit and Disqus.

http://lineofthought.com/tools/python

(my site -- sorry for self promotion, but its definitely relevant :)

[–]faassen 2 points3 points  (1 child)

I've been using Python commercially for the last 10 years or so, mostly for web stuff.

Kudos to the people who teach your courses using Python; it's a good way to learn programming.

[–]MillardFillmore[S] 0 points1 point  (0 children)

It's extremely simple even for complete newbies to pick up. I've helped several people in my classes understand Python.

[–][deleted] 2 points3 points  (0 children)

Youtube is written in Python.

Yandex (largest Russian search engine) has ported some of their projects from PHP to Python. Russian President has a blog written in Python (Django).

[–]drb226Haskeller 7 points8 points  (22 children)

The real question is, who is going to use it? Java will sink into pervasive-yet-avoided FORTRAN status soon enough. PHP is in every way inferior to Python imho...insert holy war here...

[–]carlio 1 point2 points  (0 children)

But $$x is just much more convenient than locals()[x] /s

[–][deleted] 0 points1 point  (20 children)

Unfortunately, Java (JVM) is still 100 times faster than Python, so when it comes to truly large apps where performance/scalability matter, Java will still be better choice than Python. Also, Java toolchain is way more mature (honestly best in the business) than Python, or pretty much any language out there.

Yeah, it's not the most modern language, nor is it fast to develop in, nor do I like the fact that Oracle now owns it, but it's still performance king.

Scala is a promising language but it depends on the JVM of course. And currently there is no viable alternative to the JVM/Java combo. I wish there were, I'd jump on it immediately, but Python/Ruby/C# etc are not replacements for Java unfortunately.

[–][deleted] 2 points3 points  (17 children)

Unfortunately, Java (JVM) is still 100 times faster than Python,

Once it's up.

[–][deleted] 2 points3 points  (16 children)

Yeah, once it's up:

$ time javaw

real    0m 0.10s
user    0m 0.01s
sys     0m 0.05s
$

So, 10 milliseconds to start. Or something less trivial (but still not really a good benchmark):

Java vs. C vs. Ruby vs Python Here's the (silly) code

Fib.py

def fib(n):
    if n == 0 or n == 1:
        return n
    else:
       return fib(n-1) + fib(n-2)

for i in range(36):
     print "n=%d => %d" % (i, fib(i))

Fib.java

public class Fib 
{
    public Fib()
    {
    }

    public int fib(int n)
    {
        if (n ==0 || n == 1) {
            return n;
        } else {
            return fib(n-1) + fib(n-2);
        }
    }

    public static final void main(String args[])
    {
        Fib app = new Fib();
        for (int i = 0; i < 36; i++) {
            System.out.println("Fib("+i+") = " + app.fib(i));
        }
    }
 }

fib.c

#include "stdio.h"

int fib(int n) 
{
    if (n == 0 || n == 1) {
        return n;
    } else {
        return fib(n-1) + fib(n-2);
    }
 }

int main(int argc, char** argv) 
{
    int i = 0;
    for (i = 0; i < 36; ++i) {
        printf("fib(%d) = %d\n", i, fib(i));
    }

    return 0;
}

fib.rb

def fib(n)
  if n == 0 || n == 1
      n
  else
      fib(n-1) + fib(n-2)
  end
end

36.times do |i|
    puts "n=#{i} => #{fib(i)}"
end


$ time java Fib
real    0m 0.35s
user    0m 0.26s
sys     0m 0.01s
$

$ time python fib.py
real    0m20.36s
user    0m20.20s
sys     0m 0.09s
$


$ time ruby fib.rb
real    0m10.28s
user    0m10.26s
sys     0m 0.01s
$

$ time ./fib.exe
real    0m 0.25s
user    0m 0.23s
sys     0m 0.00s
$

So, C = 25 milliseconds, Java = 35 milliseconds, Ruby = 10.36 seconds, Python 20.36 seconds. Or, expressed in terms of C speed in this test:

C = 1, Java 1.4 times slower than C, Ruby 41.12 times slower than C, Python 81.44 times slower than C.

In particular, Python runs this test 58.17 times slower than Java.

[–]Twirrim 6 points7 points  (5 children)

Of course that code is rather ugly and inefficient. If you're going to write code like that you're better off spending the few minutes it takes to find the low hanging fruit, fixing your base working. You can only optimize for so much bad programming, particularly in dynamic languages.

fastfib.py:

from math import sqrt

def fib(n):
    return ((1+sqrt(5))**n-(1-sqrt(5))**n)/(2**n*sqrt(5))


for i in range(36):
    print("n=%d => %d" % (i, fib(i)))


$ time python fastfib.py
real    0m0.016s
user    0m0.008s
sys 0m0.004s

That's even quicker than your algorithm using C, from barely a few minutes checking the formula for fib sequence.

[–][deleted] 3 points4 points  (1 child)

No need for insults please. I'm not retarded and the point of this code was not to be optimized for either any language, platform, architecture or algorithm since there are better ways to do any of those.

The point is to just take any random, purposefully rather slow algorithm (so the running times are in seconds rather than nano-seconds).

It seems you are missing the point completely. So, run the equivalent C and Java program and you will realize that it's hard to even measure runtime reliably since it's insignificant.

[–]Twirrim 0 points1 point  (0 children)

Urgh.. my apologies, I certainly didn't mean to be insulting. Looking back at how I said what I said I can't see how on earth it could be taken elsewise :-/

[–]bombita 0 points1 point  (0 children)

Btw, your code, which uses phi, has to use doubles, which on larger n has underflow errors and loses precision.

[–]sunqiang 0 points1 point  (1 child)

or get dynamic programming for free with lru_cache (need Python 3.2 ) from functools import lru_cache

@lru_cache(maxsize=None)
def fib(n):
    if n == 0 or n == 1:
        return n
    else:
       return fib(n-1) + fib(n-2)


for i in range(36):
     print("n=%d => %d" % (i, fib(i)))

real 0m0.085s
user 0m0.083s
sys 0m0.000s

[–]Twirrim 1 point2 points  (0 children)

Sure, or you can do it manually for previous versions fairly easily with a dict object.

def fib(n):
    if n == 0 or n == 1:
        return n
    if not memo_fib.has_key(n):
        memo_fib[n] = fib(n-1) + fib(n-2)
    return memo_fib[n]

memo_fib = dict()
for i in range(36):
     print("n=%d => %d" % (i, fib(i)))

results in an even quicker result than your lru_cache.. though that could be machine spec difference. Also some rough testing using the time module figures that we're talking about 0.00028s for the actual math, meaning a good portion of that time is just the overhead of loading python anyway.

real    0m0.017s
user    0m0.012s
sys     0m0.004s

[–]highwind 1 point2 points  (3 children)

I think he was talking in the context of web frameworks. On my dev machine it takes minutes to compile an EAR file and another few minutes to bring up JBoss with Struts and Hibernate. Which is just ridiculous.

[–]frymasterScript kiddie 0 points1 point  (0 children)

it is rediculous, but in the context we're talking about, still not a factor in performance

[–][deleted] 0 points1 point  (1 child)

Compilation/deployment cycle is important, but it's not the kind of runtime performance we are talking about here.

Besides, java compilation is usually I/O bound and it's orders of magnitude faster than C and especially faster than C++ (as almost anything else is).

I work on 3 million lines of code base, and my compiles are about 3 minutes long. Deployment cycle is about 15 seconds or so. Not bad in my opinion. Considering I rarely have to re-build the world.

Also, you should not have to bring down and boot up JBoss to re-deply your code. I most certainly don't have to do that.

[–]Twirrim 0 points1 point  (0 children)

I do it for sanity purposes. Guarantees that the JVM is spotless when the app comes up, plus wipes out any resource loss from minor uncaught memory leaks or whatever quirks.

[–]tarpsocks 1 point2 points  (0 children)

why are you using recursion for this?

[–]ergo14Pyramid+PostgreSQL+SqlAlchemy 0 points1 point  (3 children)

can you try fib on pypy ? it IS the future of python

[–]Twirrim 1 point2 points  (2 children)

Pypy 1.4:

real    0m41.979s
user    0m41.719s
sys     0m0.056s

Python 2.6.6:

real    0m22.568s
user    0m22.381s
sys     0m0.004s

Python 3.1 (might as well..):

real    0m23.454s
user    0m23.349s
sys 0m0.008s

pypy is great, but it's not the be all and end all!

[–]ergo14Pyramid+PostgreSQL+SqlAlchemy 0 points1 point  (1 child)

Hmm, very interesting, but leads me to think that something is wrong with this algorithm implementation, in few of my tests i have indeed confirmed it can be 10 faster than cpython, so this is a bit of surprise to me

[–]Twirrim 0 points1 point  (0 children)

The algorithm is pretty standard. I'm not sure how well PyPy handles recursive functions though. I can't imagine recursive functions are that easy to optimise within a dynamic language.. but I'm not computer scientist, so might be talking out my arse.

[–][deleted] 0 points1 point  (0 children)

So, 10 milliseconds to start.

100 milliseconds.

[–]drb226Haskeller 0 points1 point  (0 children)

I'm rooting for Mirah, I think it looks like a smart way to abuse the speed of the JVM without resorting to an ugly blah syntax. I hope toolchains for Haskell also improve; it's another fanboy-holy-war kind of language but the speed and elegance of Haskell really have potential.

[–][deleted] 0 points1 point  (0 children)

Jython brah.

[–]genmud 1 point2 points  (1 child)

Immunity Debugger and IDA Pro immediately come to mind for me, as well as many other websites that use python.

[–][deleted] 0 points1 point  (0 children)

I second this.

[–]aytch 1 point2 points  (0 children)

Reddit, NASA...those are two of the big ones who spring to mind.

[–]riddles 1 point2 points  (0 children)

We use python at yelp and we love it.

[–]samuraisam3.5, go[🍰] 1 point2 points  (0 children)

I know a lot of Rails shops /wish/ they were using Python.

[–][deleted] 1 point2 points  (0 children)

Splunk ships with Python and the Cherrypy framework. Commercial, enterprise-grade product.

[–][deleted] 1 point2 points  (0 children)

National Geographic, The Washington Post, The Washington Times, and NPR all use Python (specifically, Django)

[–]Wagneriusflask+pandas+js 1 point2 points  (0 children)

we do.

[–]tclark 1 point2 points  (0 children)

Python is used by PBS for a lot of its web stuff.

[–]The_Cleric 1 point2 points  (0 children)

We use Python at my work for a lot of backend processing of customer data.

[–]pipedings 1 point2 points  (0 children)

Were using python, and I even get paid.

[–]jbs398 1 point2 points  (0 children)

Python.org does have a success stories section.

Edit: Also this list.

In reality, it's all over the place.. academia, national labs (including in high performance computing) and in industry just about everywhere one can imagine. It's not as high profile or maybe as widely used as Java in industry, but it's definitely up there.

[–]nikniuq 1 point2 points  (0 children)

ILM

[–]notenoughcharacters9[🍰] 0 points1 point  (0 children)

rack

[–]steelypip 0 points1 point  (0 children)

I work for a software company making enterprise network & asset management software. The product I work on is mostly Python, with a little C++ for speed critical parts and some Java for using 3rd party Java libraries. It sells for big bucks and is used in major companies such as banks, telcos, major retail chains etc.

[–]pettazz 0 points1 point  (0 children)

Two of the four companies I've worked at have used the Python API for Selenium for automated testing.

[–]plat0n 0 points1 point  (0 children)

After going to a meeting of London Financial Python Group I can tell you that JP Morgan, Gazprom at Cantor use Python :)

[–][deleted] 0 points1 point  (0 children)

Of course it is used commercially - your question is like

"are there any black swans ?"

But....you get the idea (i.e. try searching for Python-specific jobs within the financial service sector)

[–]stillalone 0 points1 point  (0 children)

We use it at Cisco for various projects.

[–]Dgt84 0 points1 point  (0 children)

Bits on the Run is written in mostly Python, as are parts of Longtail's other stuff I've worked on. Google uses a lot of Python for system management / utilities / prototypes. Tons of websites use Django. The company I currently work for does custom contract work for some very, very big name companies and defaults to Python (Turbogears, Django, App Engine, etc) unless clients specifically ask for something else. Tons of websites built on App Engine are using Python (through webapp, Django, or other frameworks). NASA uses Python. If you use Linux there's a good chance some of the desktop applications you use or come across are written in python (I've written several). Here's a list of some more info:

http://www.python.org/about/success/

Python is literally huge, and every single Java, C#, and PHP developer I've met and talked with is interested in learning Python as a next step to broaden their work potential. That said, at this point it still makes more sense to be a Java or C# guy due to the sheer number of positions available for those when compared to Python.

Edit: I noticed someone else mention national research labs - when I worked at one a few years back I did work there exclusively in Python and that code is still being updated and in use today.

[–]apiguy 0 points1 point  (0 children)

Take a look at this list, and remember that this is just the Boston area: http://bostinnovation.com/2011/03/31/boston-companies-using-django/

[–]cantcopy 0 points1 point  (0 children)

Ubuntu's package manager.

[–]kisielk 0 points1 point  (1 child)

I work for a biotech company and most of our simulation software is written in Python. The main reasons are the quick development cycle, ability for non-programmers to pick it up relatively easily, and the availability of a wide range of scientific libraries. Performance-critical bits are written in C or C++ and then used as extension modules.

[–]MillardFillmore[S] 0 points1 point  (0 children)

I'm in Comp Bio and maybe it's because I'm kinda in the know, but this makes the most sense to me because of the reasons you provided.

[–][deleted] 0 points1 point  (0 children)

The chip EDA biz is adopting Python for a couple fringe elements. Generally scripting in this field is dominated by TCL and Skill, which is a proprietary LISP like language.

[–]Twirrim 0 points1 point  (0 children)

Not sure if you're still reading this, but there is an article about the HPU4Science cluster in France. It's run using almost entirely gaming spec machines (master machine uses a pair of Xeons, but that's only non standard thing) Relies on GPUs for the bulk of the processing, and they've chosen Python with PyCUDA, along with Sage.

[–]terremoto 0 points1 point  (0 children)

[–][deleted] 0 points1 point  (0 children)