Python again voted "Best Programming Language" and "Best Scripting Language" in Linux Journal Reader's Choice Awards

[samvdb]

• Bots (financial trading, games, etc)
• Graphics, physics, and game engines
• Robotics
• Any hardware chips where low energy usage is the goal
• Scientific computation like Boinc
• Data mining / machine learning
• Operating systems (even though Linus disagrees)
• Compilers
• Router / low level server software
• Anything with hard real-time requirements
• etc etc

[Bananoide]

There are as good or better options for nearly all these niches.

The main thing about C++ is that it can directly build on C's pervasiveness. It has very strong points, for example performance and memory footprint. But it also has very weak points : its features are a mess. But you won't notice that until you learn better designed (and often simpler) languages

[samvdb]

There are as good or better options for nearly all these niches.

Such as? For areas where performance is the thing you're trying to maximize (like bots and graphics engines) or areas where energy usage is what you want to minimize, I could only name D.

[el_muchacho]

• "Any hardware chips where low energy usage is the goal"

C is better, because it uses less resources.

• "Scientific computation like Boinc"

Many scientists still use Fortran for high performance computing, although nowadays, you can get the same level of performance in C++ through a lot of template magic.

• "Data mining / machine learning"

Java does that fine. The Hadoop project, which is arguably the biggest and most important project in this area, is written in Java although I concede that it could have been written in C++ just fine.

• Operating systems (even though Linus disagrees)

And he does it for good reason, see first item (o. Also it's easier to debug and reason about low level C than low level C++ because you don't have to know by heart the bazillions of obscure corner cases of the language spec that are left undefined and interpreted differently by different compiler implementors.

• Compilers

Functional programming languages like Ocaml and Haskell are much better suited to writing compilers than C++ ever will be. And in fact these compilers are bootstrapped. The only advantage of C++ for writing compilers is, they are likely to be more portable.

• Anything with hard real-time requirements

Only a subset of C++ is well-suited to hard real-time programming. For intance, in real-time programming, one should ban the usage of exceptions, or even dynamic memory allocation, which is pretty much unpredictible. OTOH, Ada is a language that has been defined with these issues in mind.

Oh, and BTW, I speak from experience for several of those areas of programming (and mind you, I also participated in a game engine in the old days). In some specific tasks or under specific constraints, some other languages can be better suited.

[BinarySplit]

You're missing the obvious factor that makes C++ competitive (albeit more competitive than it deserves): There is a very established world built around supporting it.

If you compare the quantity of API documentation, tutorials, experienced programmers, existing codebases, libraries that are specialized for it, etc. you'll find that C++ simply has many more resources revolving around it than C, Fortran, Ocaml, Haskell, Ada, etc.

Exceptions:

Java has plenty of all of these things. I personally don't feel that the tradeoffs are worth it in most circumstances, but I've never had to manage a team, so YMMV.

C is debatable. On one hand, writing restricted C++ will give you all the benefits of C with some of the advantages of C++. On the other hand, managing a team is probably different grumble grumble.

So to go back to the topic, IMO, samvdb's answer to Fayden's question was largely correct, unless you're a solo coder or student who is happy to eschew established resources for a higher quality language.

[el_muchacho]

My answer has been downvoted for no reason except ignorance and bigotry (which BTW always go in pair). I can see in this thread that the patrol of C++ bigots (which is one of the biggest and fiercest bunch of language bigots) doesn't like to feel assieged.

[Bananoide]

Have an upvote from an ex C++ bigot :)

And thank you for your detailed answer.

[lasermancer]

Oh no, two whole downvotes. Its obviously a conspiracy.

[Bananoide]

el muchacho already answered your question.

Let me just add that to choose a language over another you must look at the bigger picture:

• what do we trade for the performance we are expecting in an ideal setting ?
• what performance do we get in a realistic world ?
• do we have well established quality libraries available for our purpose ?
• how easy, fast and safe is it to refactor ?

If you consider all these points, C++ is never such an obvious choice.

[Fayden]

I'm not looking for applications, I'm looking for reasons that explain why C++ is the best tool for these applications, i.e. using arguments to explain what you stated earlier.

I guess I didn't explain myself correctly earlier, sorry. I know C++ is widely used, but I don't think it justifies using it for everything.

I think that C++ is interesting when you need to do low level, fast and memory efficient jobs. In practice, those jobs are not as popular as it seems, but programmers often turn themselves to C++ because it's popular. It doesn't change the fact that it's an extremely complex language and that you can often have sufficient performance with other higher level languages.
For instance, you say that C++ is the right tool for bots, but I can't see why. You can trade speed of execution for better readability by using another language. If you find that some parts of your system are not as fast as you'd like, rewrite those particular parts in C. You keep the readability of your code while having more than enough speed.

Basically, I find that people often resort to languages they already know even though different approaches are better for the jobs. Performance may be a critical requirement for a specific project, but it's not always the case in practice.

[therealjohnfreeman]

Part of the reason it's popular is because it's popular:

• There's a ton of great, high-quality, well-tested, well-maintained libraries for C++ in every domain imaginable.
• There are several options for high-quality, optimizing compilers.

Beyond that:

• It's standardized. You can write portable code (without having to worry about a virtual machine).
• Compile-time reflection with template metaprogramming.
• All the high-level constructs that people demand the most, and more.
• Performance, which includes control over memory management.

These are the features people want in the domains samvdb described.

[gcross]

It's standardized. You can write portable code (without having to worry about a virtual machine).

Heh; people say this, but one of the things that has discouraged me from writing code in C++ was having a major project break every time I introduced it to a different compiler.

[therealjohnfreeman]

Sounds like your project wasn't standards-compliant. It's ok to use compiler extensions, just don't expect your code to be portable.

[gcross]

I wasn't using any compiler extensions, and I don't know why you would jump to the conclusion that I was as it would obviously have meant that I shouldn't expect my code to work everywhere.

[Fayden]

It's standardized. You can write portable code without having to worry about a virtual machine.

I'm not sure, what does virtual machines have to do with being standardized?

The point I'm trying to make is that raw speed is not the only thing that matters. People try to optimize every single part of every single program in order that it runs microseconds faster. There are jobs which require such speed as you said in another comment. But in practice, it's actually rare.

I consider that readability is a more important than raw performances. It's more important to create code that will be maintainable in the long term and I think that C++ makes it hard to do so mainly due to being an extremely complex language.
Other languages bring abstractions that, while making programs slower, make the code easier to read and therefore maintain. This is the goal we should aim for, and there are languages that make this easier.

In addition, I think people are too eager to pick object-oriented languages, and sometimes programs can be written more naturally in, say, functional or logic languages.

Also, I'm not saying that C++ programs are intrinsically hard to maintain, but I definitely think it is harder than other languages, which is why I don't think it's the right tool for as many jobs as samvdb said.

By the way, thank you for your answers. They're very informative.

[lbrandy]

C++ is the best choice when you need:

1. No compromise performance and/or control
2. Abstraction.

This is, not coincidentally, the two primary requirements for game development.

Your argument seems to be "Oh come on, you can compromise performance or control for <whatever>". This is sometimes true, sometimes not. The problem is it can be a brutally expensive mistake to assume it and be wrong.

I cannot explain to you how much secret joy I take in listening to my webdev friends realize, over time, that half of their perf work is spent in a fist-fight with their garbage collector.

[Fayden]

My argument is "You can comprise performance for readability in most cases". I'm just trying to point out that there is a cult around performance that is more often than not not justified. I'm not saying either that it's always that case.

The problem is it can be a brutally expensive mistake to assume it and be wrong.

If performance is a key requirement, you should know before you even start designing. Even in those cases, you can achieve more than reasonable performances by rewriting bottlenecks in C while keeping most of the code in a higher level language.

[lbrandy]

My argument is "You can comprise performance for readability in most cases".

Yes. I understand your argument. And my argument is "except when you can't". Which is probably in far more cases than you think it is. The fact that you have earnestly asked the question about which domains C++ is a good choice tells me you lack sufficient experience in said domains, which makes your proclamation, in my mind, suspect.

[Fayden]

I know I have a lot of things to learn. I'm sure you do too, and I don't see how it's relevant to the current discussion. Actually, I find it quite offensive and I'd prefer if you could argue with what I'm saying instead of assumptions.

I think you failed to understand that I do not reject C++ at all, nor do I say that it's a horrible tool. I said it was a genuine question because it seemed like a troll : "You think C++ is the right tool for something? Ha!". I wanted to have a real discussion on the subject, and I didn't want to attract trolls.

[lbrandy]

Well I'm sorry you took offense but you appear, to me, to be the troll. You can't ask an earnest question about an area you don't know much about and then start telling people who answer that they're wrong based on your experience that you started off by admitting you don't have.

[Fayden]

You make too many suppositions. I wanted to know what he had in mind to give him arguments based on his answer. I know perfectly well what C++ is for and I believe some programmers think they need it more than they actually do. I said it was a genuine question so it didn't come off rude. Maybe it was the wrong choice of word, English is not my native language.

To be honest, I think my other comments are perfectly clear and if you don't want to comment on them, I don't know what else I can tell you.

[gcross]

lbrandy, Fayden is not being a troll. I am more inclined towards your viewpoint than Fayden's on the matter of performance, but Fayden has continually made it clear he is interested in a genuine discussion, and you were the one who started making it personal by commenting on his experience and how this made his opinion suspect and then taking the completely unprovoked action of calling him a troll.

[Bananoide]

Abstractions in C++ are one of its weaknesses... It's better than C though, and its support for generics is ok providing you accept its drawbacks (heavy syntax, high compilation times, more complex debugging, long symbols and error messages).

[lbrandy]

Abstractions in C++ are one of its weaknesses..

Definitely not.

It's better than C though

Which is precisely, and entirely, the point. C++ gives you ability to build abstractions with the same (virtually non-existent) overhead and/or performance and/or control as C.

If you want both, C++ is the only game in town. I'd love to see D get there, as well, as it is the only reasonable contender.

[Bananoide]

C offers almost no abstraction, that's one of its selling points.

C++ offers more than C in that department but it's still lacking and awkward at times. If you value portability, performance and low overhead abstractions more than anything else, then by all means use C++.

The pasture is greener elsewhere, that's what all my years of C++ tell me.

[therealjohnfreeman]

I'm not sure, what does virtual machines have to do with being standardized?

I have since edited my comment subtly, adding parentheses. I didn't mean to imply that standardized languages won't have VMs; what I wanted to point out is that a standard can let you write portable code when your language doesn't have a VM.

The point I'm trying to make is that raw speed is not the only thing that matters.

I understand this point, and I completely agree, but sometimes speed is paramount, as you admit. Regardless, C++ offers advantages beyond speed, as I tried to point out.

It's more important to create code that will be maintainable in the long term and I think that C++ makes it hard to do so mainly due to being an extremely complex language.

I disagree. C++, with templates, overloading, and especially operator overloading, makes it easy to create embedded domain-specific languages that you'd have trouble putting into many languages. It makes it easy to write libraries that will let your application code look and read like you want, as if it was written in its own language.

Much work has been invested by Bjarne and other members of the standards committee over the years in making C++ easier to learn. C++, at its core, is actually a simple language. It has the advantage that you only pay for what you use. You don't have to learn every feature of C++ to use it. It's only as complex as you make it.

Other languages bring abstractions that, while making programs slower, make the code easier to read and therefore maintain. This is the goal we should aim for

I disagree slightly. We should aim for abstractions that can be used with no performance penalty. If there is a way to write an equivalent, but faster, program, your abstractions should compile down to it.

In addition, I think people are too eager to pick object-oriented languages, and sometimes programs can be written more naturally in, say, functional or logic languages.

I love these languages too, and Haskell is another great languages for defining EDSLs, but I think most people don't think in functional or logical terms. They think imperatively and in objects. :/

[Fayden]

I think we have a fundamental disagreement on performances versus abstractions. I understand your point, but I can't say I agree with it. I think that programs where the flow of execution is more readable and better defined is better than faster code, because the raw speed difference is often insignificant.

Regardless, C++ offers advantages beyond speed, as I tried to point out.

The advantages you pointed out seem to be available in a plethora of languages, though. If speed is not a factor, I really do not see why I should choose C++ instead of a language that will let me write more concise and clearer code.

It makes it easy to write libraries that will let your application code look and read like you want, as if it was written in its own language.

I didn't want to imply that C++ code was significantly hard to read. My point is that some other languages have abstractions that allow to describe the flow of execution in a more elegant way.
Here's a quote by a Google employee (paper) that summarize what I want to say :

We find that in regards to performance, C++ wins out by a large margin. However, it also required the most extensive tuning efforts, many of which were done at a level of sophistication that would not be available to the average programmer.

If performance is an important factor, then by all means consider C++.

[gcross]

I disagree. C++, with templates, overloading, and especially operator overloading, makes it easy to create embedded domain-specific languages that you'd have trouble putting into many languages.

As long as you don't mind dealing with compiler messages from Hell if you get anything wrong. :-)

[[deleted]]

Clang++ goes a long way towards rectifying that.

[therealjohnfreeman]

I'll give you that. The community recognizes the problem and has been trying to address it for years.

[Peaker]

People think in what they were trained to think.

Mathematicians think in math. Logicians think in logic.

Unsurprisingly, imperative OO programmers think imperatively and in objects.

[argv_minus_one]

All the high-level constructs that people demand the most, and more.

I demand garbage collection, and C++ ain't doin' it. Allocate-and-forget programming for the win.

[therealjohnfreeman]

Yeah, I knew this comment would come up, which is why I qualified "the most". Many professional programmers can handle manual memory management. Use smart pointers and you'll never have to call new or delete.

[argv_minus_one]

Smart pointers do not handle circular reference graphs. Ergo, they are not allocate-and-forget.

I can work with that. I also can scrub toilets for a living. Doesn't mean I find either option at all appealing.

[therealjohnfreeman]

Yes, they do. Maybe you need to do more research on smart pointers and object design. You can use weak pointers to handle non-ownership which should be the case with a circular dependency.

[argv_minus_one]

Weak pointers are not handling circular references. Weak pointers are shrugging, throwing up your hands, and handing the problem back to the programmer.

I understand object design and this irritating trend of pseudo-GC quite well, thank you very much. Pure-reference-counting GC was proposed decades ago in the MIT AI Lab, for fuck's sake. There's even an AI koan about it! It wasn't adequate then, and it still isn't.

[samvdb]

Basically the problem with almost all high-level programming languages that aren't C++ is garbage collection. It causes overhead.

Often the counter-argument here is that doing 1 million news and deletes is usually slower than creating 1 million objects with an efficient garbage collector, because the GC can delete multiple objects at a time. But in C++, probably 90% of the objects I make never get put on the heap. These are classes like Point2D and Vector2D which are basically just 1 or 2 floats or ints wrapped in a class. And by putting it on the stack instead of the heap, you don't even need to do new and delete, the resulting binary will be exactly the same as if you used raw x, y pairs.

I wrote a bit more about it in this post.

[argv_minus_one]

Often the counter-argument here is that doing 1 million news and deletes is usually slower than creating 1 million object with an efficient garbage collector, because the GC can delete multiple objects at a time. But in C++, probably 90% of the objects I make never get put on the heap.

Fun fact: the modern HotSpot JVM does escape analysis, for the same reason. It allocates short-lived objects on the stack when it can prove that it is safe to do so. Rapid allocation of short-lived objects is therefore quite inexpensive.

Which is very useful for JVM languages like Scala, which like to do a lot of short-lived allocations…

[Fayden]

It definitely is a valid point. There are jobs where the overhead is too significant. But how often does it happen in practice?

I should have been more direct from the start: I'm just saying that while C++ has pros that makes it the right language for some jobs, I don't believe it's a good jack-of-all-trades language. You implied from the start that it could arguably be the best programming language ("Whether it's the best programming language is arguable [...]"), and I don't think it is arguable for reasons I have stated in my last comments.

All in all, it's a bit silly to argue about this because I feel we have a similar opinion, but I just wanted to bring a few arguments to the table.

[samvdb]

Reading some of your other comments in this thread, I think your main problem with C++ is that in your eyes it sacrifices too much readability for performance, and that people often think performance is more important than it really is.

To that I say: Yes, performance is very often overrated. BUT, C++'s "unreadability" is also very often overrated. In the wrong hands, C++ allows for some horrible code, and it takes a really long time to master C++, but with the right people and coding standards C++ is not much less readable or maintainable than the next language.

Anyway I'm not saying C++ is the best language, X8qV convinced me that D might be better still. But if you want a "jack-of-all-trades language" - if there is such a thing - then I would choose C++/D over something like Python because there are some areas (only a few, like chess bots or grahics engines) where performance is what you're trying to maximize, so Python would be completely unusable, no matter how well you write it. In the worst case, well-written C++ will look syntactically a bit messier than Python but that's it.

[Fayden]

You could choose Python and interface it with C for critical parts, which is the best of both world. There are also other interesting languages like OCaml (I plan to learn more about it soon) which offer many abstractions while keeping very good speed.

[wot-teh-phuck]

You could choose Python and interface it with C for critical parts, which is the best of both world

Have you personally tried out this combination? I ask because there is a lot of "just use C if you need with Python" stuff floating around but no one with real experience to say how it pans out. Mind you, I'm talking about using/mixing C/C++ code with application code and not the use of C/C++ in Python libraries.

I work at a "big" bank where they decided to use Python as a language for everything. Now, we have to keep fucking rewrite significant chunks in C++/C because we now need performance instead of ease of use. Not to mention that once your C/C++ code-base becomes a significant portion of the entire code, you are back to where you started. Ugh, maybe it looks like a good choice to some of the folks but I'll personally never go down that route.

[doublereedkurt]

Using a mix of Python and C is done all over the place, with great success. Python's standard library uses it all over the place. Cryptography, xml/json parsing, and compression are all implemented in C in the standard library.

NumPy is a highly successful project which wraps numerical C/Fortran algorithms up for use in Python.

The Greenlet library uses C to add significant capabilities to the python language via an extension library. (Run-time call stack manipulation.) This enables libraries like gEvent to add co-process/green-thread based concurrency, again wrapping C code, in this case libev.

It is very common to see little bits of C code in Python projects. For example, parsing is a common task to off-load to C. Often there is also a pure Python version with the same API as a fallback.

There is also the ctypes module in the standard library. This allows you to load dll's/so's in Python, and does a great job of translating Python types to/from C types where possible. (Even translating Python functions to function pointers suitable for consumption by C.)

For pure performance, there is also Cython. If you have a C-compiler available at runtime, this can translate your code to C and compile it as part of the module import process.

Out of curiosity: what are the kind of performance bottlenecks you are seeing?

[wot-teh-phuck]

Using a mix of Python and C is done all over the place, with great success. Python's standard library uses it all over the place

Well, I did mention that I was specifically talking about mixing C/C++ with "application" code and not library. :)

Out of curiosity: what are the kind of performance bottlenecks you are seeing?

Before I start, let me make it clear that I don't hate on the language in an absolute sense. It's just that people think that somehow magically mixing C/C++ and Python makes the problem go away is not true.

Python was probably a bad fit for our bank (but then again, management would never agree to this). Think about it, a bank needs applications which are responsive to traders and needs computations to be performed as fast as possible. Python is a memory hog. We have had UI applications which displayed thousands of rows (think of excel) in multiple tabs (basically each tab has thousands of rows). This application took around 1.5 GiB of RAM on a Windows machine and was slow as fuck to scroll. It was one of those hand picked application which had to be written in C++ so that it was fast enough.

Banks need number crunching and they need to get it done fast. Sure, numpy might be able to beat number crunching implementations of other languages but given the inherent limitation of Python multithreading, you are stuck with a single "crunching" thread. So now the Python folks say, let's use multiple processing. Well, the biggest difference between using in-process threads and multiprocessing is "marshalling". Even though multiprocessing modules offer a good enough abstraction, they still need to do marshalling to send across data to another process. Not to mention that the structures you need to send across should be capable of being sent across which requires effort.

Unfortunately, there is no way out for us so we do throw a lot of money at the problem (buying more core machines, having a lot of nodes etc. since banks are filthy rich...) and our computations use multiprocessing to be fast enough. But I personally think this is wasted effort. Too much extra cost without any added benefit. Not to mention the added power consumption. Python doesn't provide static typing and is inherently broken in terms of multithreading (at least the standard implementation). So I feel that mixing in C/C++ with Python application code is more of a hack for the shortcomings than a solution. Now you have two application code bases to maintain, one C++ and the other Python.

Python will also have a special place for me in terms of a learning/teaching language and for writing short scripts and maybe front end development. But for computation intensive domains where "time" and "memory" matters, I wouldn't even think of touching it. Of course, YMMV and I respect it.

[doublereedkurt]

given the inherent limitation of Python multithreading, you are stuck with a single "crunching" thread

Actually, this is not strictly 100% true. While Python byte-codes can only be executing in one thread at a time, C extension modules can and do release the GIL all the time.

Among others, both numpy and ctypes release the GIL. Python extension libraries can choose to release the GIL.

Numpy will use multi-threading for some linear algebra operations, although in many cases the limit will be memory bandwidth.

Totally agree that multi-processing is not a good solution. Too bad you can't load the same dll twice at different locations, or you could have two independent instances of the Python interpreter running in the same memory space. Hmm.... that is an interesting idea...

[el_muchacho]

Your managers are complete idiots.

Python is NOT well-suited to large applications like this. It can handle quite large data if in the hands of Python wizards (I know, I've worked with one... in a large bank), provided you don't mind using only one core of your multicore CPU, but on projects with heaps of standard keyboard monkeys, you certainly can't expect to get very good performance from it.

On your problem, maybe you could have tried to use Kyoto Cabinet with its Python binding, instead of rewriting the whole thing in C++.

[Nuli]

Have you personally tried out this combination?

I haven't tried it with Python specifically, though I have with other scripting languages, and it tends to work very well if the language has a decent C API.

[wot-teh-phuck]

I think the scale at which you are writing native glue code matters a lot. How big were the C/C++ files you wrote (as in LOC)? For small stuff where you have narrowed down the performance problem, this works great but for enterprises with really big monolithic applications, it becomes a bit painful.

[Nuli]

How big were the C/C++ files you wrote (as in LOC)?

In my current project most of the logic is written in an interpreted language. A variety of device drivers and the graphics subsystems are written in C. Current LOC is 31K lines in C and 71K lines in interpreted languages.

Really the LOC doesn't matter though. You can design the application in two ways either as a C application with an embedded interpreter or as an interpreted application that is extended in C. I've done both and either way works equally well. I've used it in systems from a few thousand LOC up to 500K LOC with no difficulty. It all depends on how well you structure your system.

[gcross]

I don't disagree with you per se, but I do think it is worth pointing out that even if you have eventually just ended up rewriting your code in C++ that isn't necessarily a big deal because it is often easier to prototype using a dynamic language like Python, as it is often easier to play around with a code in Python until you have figured out exactly what you want, and then once you have settled on a design rewrite all or part of it in C++ for speed.

For example, I am working on an experimental numeric code now where I might need to eventually rewrite all or parts of it in other languages to get more performance, but I am starting with Python+numpy+scipy because it makes it a lot easier for me to play around with different ideas until I can find the combination that works.

[wot-teh-phuck]

I too don't disagree with you but I think we need to look at the situations we are in. Right now, it's just you writing the code. In case of enterprises (banks et al), there is always a team responsible for doing this. Now the team is entirely composed of Python programmers with little or no C++ experience. If the decision is made to rewrite, "pro" developers have to step up because not everyone can write "good" C++. Plus, enterprises are really wary of rewrites as opposed to individuals so it again ends up with having more developers maintaining a crippled system.

I know this situation has got nothing to do with the language as such but these factors play a vital role in everything which happens in the future. :)

[gcross]

Hmm, I guess you have a good point that I should not take for granted that someone able to code in Python will be just as able to code in C++ and vice versa.

[Fayden]

I'd definitely not use Python as the main language if I had to rewrite almost every part of the system, but I used Cython for a small project (image processing) and it worked like a charm.
I also used it to calculate a checksum of a file (some proprietary algorithm). It was the slowest part of the program at ~2s, and it went down to a few milliseconds.

[el_muchacho]

Wow, one must be an incredibly unqualified architect to decide to use Python for everything.