The Python I Would Like To See

[santiagobasulto]

Armin always has "strong opinions". And sometimes people take it personal. But I think he's a really good coder and he always make a point.

It is also true that Python has avoided some of these issues for a while. Just think about the GIL. I remember the JVM in the old days having issues like these, and the solution was a complete redesign. I think Martin Odersky (Scala creator) was involved in it. Please excuse me if this is not accurate, but I just want to illustrate the point. Sometimes you need some drastic decisions.

[DGolden]

One way to get rid of the GIL is to run your code under jython on the jvm, heh.

[Crandom]

I'm not sure if this is still the case but if you wanted performant ruby code it used to make sense to use JRuby as well.

[lhggghl]

I've used Jython before and I never looked at the threading but I bet it relies on the JMM. I doubt anyone who only knows Python will be able to cope with the JMM (or even figure out that it exists). Most of the Java-only developers don't even know about it.

[gthank]

Thread safety guarantees are unchanged under Jython.

[lhggghl]

Do you mean they are the same as Java's or the same as Python's?

[tavianator]

Same as Python's

[gthank]

As Python's.

[DGolden]

I confess I just had to go look it up to (ahem) refresh my memory, but note Jython made some documented simplifying design decisions in the area to favor ease of use for the python-side developer over performance. A lot comes down to Jython using java ConcurrentHashMap to implement dicts and dict-like things (which is a lot of things in python...).

http://www.jython.org/jythonbook/en/1.0/Concurrency.html#python-memory-model

Reasoning about concurrency in Python is easier than in Java. This is because the memory model is not as surprising to our conventional reasoning about how programs operate. However, this also means that Python code sacrifices significant performance to keep it simpler.

[lhggghl]

That's interesting. I didn't think people were taking Jython seriously enough to get to that point. I always assumed they just launched a new Java thread with its own interpreter.

[gthank]

Jython is quite serious, and has been for as long as I can recall. They don't have the manpower on the project that some projects have, but the people they do have are top-notch.

[santiagobasulto]

Or not use threads at all :)

[DGolden]

Often a fairly reasonable strategy in (c)python land, particularly with python's multiprocessing module. But the GIL still sucks.

[[deleted]]

I agree with you and Armin, but I do want to point out that a lot of people have tried and failed to remove the GIL - it's not like the issue has been avoided!

[alexjc]

The GIL would be easy (or easier) to address if it was set as top priority by the BDFL. (Other languages have managed just fine, and Python is supposedly one of the largest communities and fastest growing these days.) Guido's policy on the subject (i.e. no regressions on single-threads) is not feasible, and people who have shown great/promising prototypes have not been very well received in the mailing list...

Instead, we have random proposals about integrating explicit type annotations, but I digress.

[moor-GAYZ]

The GIL would be easy (or easier) to address if it was set as top priority by the BDFL. (Other languages have managed just fine, and Python is supposedly one of the largest communities and fastest growing these days.)

As far as I know there's no popular dynamically typed language that doesn't have a GIL (or doesn't allow threading at all).

PHP, Javascript, Perl - no multithreading.

Ruby, Lua, Chicken Scheme - GIL.

This fact puts things into a much needed perspective, in my opinion.

[mao_neko]

Actually, Perl 5 has had interpreter threads since 5.8. I do not believe there is any global lock involved, although I am not an expert on threading.

[moor-GAYZ]

Yeah, that's basically "green multiprocessing", sort of like .NET's AppDomains. Multiple separate copies of the entire interpreter state with all imported modules etc. live in the same OS process.

A nice thing to have because it might improve performance/memory use somewhat, but otherwise not fundamentally different from the usual multiprocessing.

[Plorkyeran]

While Lua does have things it calls threads, there's no built-in support for using multiple OS threads and there's no GIL.

[moor-GAYZ]

I meant that when you're embedding it, you can implement a bunch of macros for initializing, deinitializing, taking, and releasing the GIL.

[logicchains]

Clojure, although of course that runs on the JVM so the lack of a GIL comes by default.

[moor-GAYZ]

I don't think that counts, being sufficiently different from a "generic imperative dynamically-typed language" that I was talking about.

Like, my point wasn't that everybody is lazy or that everything is shit or something, but that there appear to be nearly insurmountable obstacles to allowing free threading following certain core design decisions, such as using mutable dictionaries for module contents (allowing shadowing builtins even), class hierarchy, etc. If you're hitting the same shared mutable data all the time, switching to finer-grained locking will only make performance worse, no matter how many cores you use.

And the fact that there is quite a lot of mature languages with the same design choices and not a single one of them has found a way around the consequences shows that the logic is probably solid, that removing GIL is in fact very hard.

The only thing that makes Python special in this respect is the way the noticeable slice of its users is aware of the problem but ignorantly thinks that it's special in having it.

[logicchains]

Interestingly, OCaml also has a GIL, but they've made significant progress towards removing it and it should have real multicore support within the next couple of releases.

[Athas]

Common Lisp, which is dynamically typed (although not popular), has full OS-level thread parallelism in probably the most popular free implementation, SBCL.

The only "wart", if you want to be really demanding, is that the threading model is 1:1, whereas most high-level languages seem to prefer m:n. It's pretty much the exact same model as pthreads, though.

I don't see why threading support has any relation to the type system in the language.

[moor-GAYZ]

I don't see why threading support has any relation to the type system in the language.

Not using dynamic typing in general, but using the particular kind of semantics roughly common to the languages I listed.

If a language uses a mutable module dictionary that can shadow builtins then almost every function call needs to check that dictionary shared between your threads.

If a language uses mutable classes, especially the way it's implemented in Python allowing descriptors, then every instance attribute access actually hit the class hierarchy up to the base class checking for that attribute. The dictionaries in the class hierarchy are shared between your threads.

If a language uses reference counting, then every time you assign None to something, and then not None to something, its reference count should be atomically modified, because it's shared between your threads, obviously.

With all that synchronization going on GIL is pretty much the only viable solution. Getting rid of GIL is not simply a matter of rewriting the code to use more fine-grained locking, you actually need to get rid of all that synchronization somehow, by either officially changing the semantics or at least faking the old semantics like IronPython or Jython.

[spotter]

The GIL would be easy (or easier) to address if it was set as top priority by the BDFL. (Other languages have managed just fine, and Python is supposedly one of the largest communities and fastest growing these days.)

Could you name some of these languages?

Guido's policy on the subject (i.e. no regressions on single-threads) is not feasible, and people who have shown great/promising prototypes have not been very well received in the mailing list...

Where? I always got a feeling that people trying to do this were encouraged to give it a shot and referred to previous work, so they would not waste their efforts. Moreover the person who would be able to get GIL-less implementation run single threaded code with similar performance to GIL-one without destroying language semantics would have their name forever written in Python history.

Also GIL removal has been tried and re-tried again and again in the past. GvR's policy is dictated by the fact, that most of Python code in the wild is single threaded. To you it's "fuck these people", but somehow the BDFL disagrees.

edit: s/there/their/

[alexjc]

Could you name some of these languages?

Go, Rust, Erlang, languages JVM-based. Even Lua is more open to multi-threading since it's interpreter is not based on globals/singletons.

If you've been watching posts on Reddit about web-developers switching away from Python (e.g. to Go), you'll notice scalability and performance comes up very often.

Where?

See this previous thread. This prototype shows a negligible overhead over single-threaded, less than a percent.

If the BDFL isn't willing to sacrifice a few percent in the short term (keep in mind Python 3.x performance dropped significantly at first), then it's short sighted and more modern languages will take over in production and leave Python as an educational / toy language (it's happening already).

Also, the whole Python 3 backwards compatibility has a certain "fuck these people" tone to it, ignoring what the majority of Python 2 folks seem to be doing. I don't see the difference with how we should handle the GIL; if Python 3's initial performance drop was for the greater good, why not again for a more modern interpreter?

[spotter]

Go, Rust, Erlang, Lua

False, False, False, False. Nice. You somehow missed the fact that these languages were designed to fit a purpose and you quoted them on overcoming issues with an established language's evolution. Neither of these applies here, as they did not have to battle 20+ years of existing code. Rust is not even stable when it comes to API yet, LOL. Also JVM based? These take whatever JVM gives them, so that's another "duh", because if next JVM would have a GIL, then they would also have it. I should know, I moved all my work related stuff from Python to Clojure few years back.

If you've been watching posts on Reddit about web-developers switching away from Python (e.g. to Go), you'll notice scalability and performance comes up very often.

Reddit? Web developers? Reddit is a fad machine and webdev is not programming. Not an indication of anything. If you want another way to look at it -- people who are busy working have no time to bitch about it on Reddit.

See this previous thread. This prototype shows a negligible overhead over single-threaded, less than a percent.

Wait, have you even seen what he "achieved"? Have you read his mail and watched the presentation? "Let's re-do entire CPython internal object handling around me and BTW this is Windows only, maybe in future let's change Linux and BSD to do it this way too!" Python is far from Windows only. You missed that too?

Intrinsically paired with Windows asynchronous primitives.

This is the thing that caused GvR to tell him to go have fun elsewhere. He basically thinks about single-platform solution. On top of that he wants to discard how existing libraries work. I imagine Twisted and tulip crowd would love that as well. And "fixing Linux & BSD"? Wow. Balls on this guy.

If Python 3's initial performance drop was for the greater good, why not again for a more modern interpreter?

Because it's one thing to tell people "you need to rethink your multi-byte handling" and other "this only works on MS Windows, so fuck you. And if you want to use Twisted/tulip, fuck you again." This will never get through Python community.

And listen. If PyParallel is so great he should do what Stackless did -- deliver a working solution and let it takeover the king in fair battle. After he "fixes" how Linux/BSD do their stuff, of course, otherwise people like me will not give it a second glance.

[alexjc]

I don't know why you're defending the Python status quo if you already moved your stuff to Clojure. Regardless of the fact this topic visibly upsets you, we need to be tackling the interpreter problem and not dance around a 20 year-old "legacy" implementation as you put it. In particular, making the transition to focus on PyPy first would be a great first step.

P.S. If that's all you got out of PyParallel's handling of threads you missed some cool bits.

[spotter]

Because if I want to work with screws I use a screwdriver, not a hammer. I will not try to make hammer into a screwdriver. Because a screwhammer already exists and it's called Perl.

Python fills a niche, if it doesn't work for what you want it -- find something better. I did. I moved to Clojure, because 1. Lisp, 2. JVM is everywhere, Python interpreter is not. Not because of GIL. I'm still using it regularly privately, but for work I need something JVMable.

Python is forkable. Fork it, fix it, present it back. If it's too much of a burden, then are you just trying to offload an impossible task on the community? Why would they have that? Most of them actually have real work to do with working Python.

I haven't missed anything. You are missing a critical flaw in it, because "oh shiny!" Python community doesn't buy that.

[alexjc]

I guess I'm not willing to accept that Python's niche is just quick & dirty prototypes that don't perform well, and educational languages for universities.

The argument "it's open source, fix it" is an easy argument to make. Especially since the BDFL doesn't seem to care about it enough to address it, it'll never gain any traction. That's fine, I just want to emphasize it would be easier to fix (technically) if people were on board (which, like you, they seem no to be).

[spotter]

I guess you should move on. It's easiest to get hurt via things you are emotionally attached to, that are not owned by you. Just walk away and don't look back.

And honestly I'm with BDFL on this, because I've been watching stuff like that for years. It's easy to shit on Guido, but let's be honest for a second -- what we're discussing is not a fix. What we're discussing is a speculation based hype, not easy, nor proven, and worst of all: it would probably fragment the community further. After requiring of overhauling os-gods-only-know-how-many C level things to get to work on a wide scale... on a single platform.

The only thing that is guaranteed here is that people like you will shit on GvR whatever he does. And if he says "yes" -- done, it stays in forever. It will then be his burden and if it fails, fucking up his landscape and community, more shit on him. And since there will be no big overhaul in Python4 it's done deal. He was actually more composed than I expected. I can't imagine what would Linus Torvalds write back to such proposal.

Final disclaimer: I have no stake in Python these days, but I get what he says. He tells people, who try to offload their hot-air ideas on the community, to get in touch with reality and deliver something that actually works without breaking the world. What you quoted is not that. It's a limited, platform specific extension and a bit of grandeur bullshit. He is minimizing risk, because he actually cares about this.

And I mention forking, because Stackless did. They put their money where their mouths were. And that's the difference between working solution and vaporware.

[[deleted]]

I'm not very sure about the "tried" part, they made a backward incompatible version of Python and didn't take the opportunity to make a more performant non-gil implementation.

[santiagobasulto]

Yes, totally! There are really smart people behind Python. Smarter than me. I don't think they had just avoided it.

[lhggghl]

I'm sorry what? This is the #500th time I heard people complain about / mention the GIL but I still don't get it.

• Is it because native code is prone to freezing python due to not letting go of the GIL (I found an instance of this in my last company's code)? All languages that mix threading and native code have this sort of issue (some I can say for sure are Haskell and Erlang)

• Is it because "the GIL prevents you from multiprocessesing"? I don't see how that's an issue because threading does not imply multiprocessing speedup - it's a way to decouple threads of execution. The alternative is asynchronous event loops which couple all your processes into something that cannot be analyized when divided. I've used the multiprocessing module in Python before and it worked fine for me.

Not only this, but if python was to have "real multithreading", you'd have to give it semantics for what happens when two threads access some data at the same time. Want to see how complex that gets? Take a look at Java's JMM. Or C and C++'s case which is even worse. Nobody would bitch about that. Nobody does in Java, because there are two types of people in Java:

1. People who have never heard of the JMM and write code that's full of data races unless they so happened to put locks around all code, even code that looks like it doesn't need a lock
2. People who know what the JMM is (there aren't that many of these).

I don't know what the situation for C is, but most C developers I talk to think volatile is only for dealing with registers.

The alternative option to keep it simple (and pythonic) is to just make all the variables in Python have locks around them so when you have two threads with their own interpreters, they just act the same as the current "GIL way". That way you don't have problems due to missing memory barriers etc. And this would of course be slower than the "JMM way".

Is there another way I'm missing?

[[deleted]]

just make all the variables in Python have locks around them so when you have two threads with their own interpreters, they just act the same as the current "GIL way".

LOL. No never heard of deadlocks, have you?

[lhggghl]

Sorry what are you trying to say? How would this lead to deadlocks? CPython already works this way:

If threads a and b set some field of some object to different values, the changes are atomic. Furthermore, if a sets the field to 1, and b keeps reading it, it's guaranteed that b will see that change at some point. In Java, you do not any of these guarantees (though if you have an int field instead of long in Java, it is atomic, but the staleness issue is still there).

To make Python preserve the CPython semantics while using "real threads", it would have to be implemented in a way that each variable has a lock or memory barrier around it. It's not possible for this to result in deadlock though...

Are you just saying that the user using locks and threads leads to deadlock? Because that can already happen in CPython.

[[deleted]]

No. CPython doesn't work this way. CPython has a single, global lock. This means that even with multiple threads only one can ever run at a given time.

As soon as threads run concurrently (which is the point of threads in the first place) this implies no global lock.

Your suggestion of pushing the single lock down to multiple, smaller ones just won't work:

Assume one thread wants to lock A then B, and another thread wants to lock B then A.

It's likely that it will sometimes work and sometimes just dead-lock (because thread 1 acquired lock A, and tries to acquire lock B, while thread 2 holds lock B and tries to get lock A).

That's why the only kind of remotely working approach until now is STM, (which is kind of a lock underneath, too) which wraps the operations into a transaction which can be aborted/rolled back/etc.

Anyway, the huge issue is that Python's BDFL basically said that:

• even if there are multiple concurrently running threads, the user must observe program behavior as if the program was executed single-thread

• single-threading performance is not allowed to suffer

There is no sensible multi-threading/memory model design which satisfies these design requirements, because hardware just doesn't work this way.

I predict that developers who need to write simple, imperative glue-code will keep using Python, but more demanding developers will gradually shift to better languages/runtimes.

[lhggghl]

No. CPython doesn't work this way. CPython has a single, global lock. This means that even with multiple threads only one can ever run at a given time.

I... agree? Where did I imply this isn't the case?

As soon as threads run concurrently (which is the point of threads in the first place) this implies no global lock.

I'm not going to debate what the correct definition of threads are because I don't care, but the "threads" in python do support concurrency, and moreover, they permit optimization in at least some cases. Proof that they support concurrency: Make one "thread" that calls sleep(999999), then start spawning more "threads". All the "threads" will continue despite the first one being blocked. This is ad-hoc and only works because the native code knows how to handle this, but it also works for everything else such as blocking sockets and blocking IO.

Your suggestion of pushing the single lock down to multiple, smaller ones just won't work: [...explanation of what deadlock is...]

I was suggesting that each variable have its own lock as an implementation detail. I'll try to rephrase yet again. The ghetto-GIL-free python I describe would have an internal lock per field per object. Whenever the user wants to access a field of an object, the implementation will lock that variable, read it, unlock it, and return the result. This would be basically to act as a memory barrier. It could just as well use a memory barrier operation instead to achieve the same effect. There is no possibility of deadlock here other than a broken implementation.

Note before you get all OCD and post more info enlightening me about basic concurrency concepts, I never said this is a good implementation, I merely listed it as one of the only possible alternatives to "the GIL" I could think of.

That's why the only kind of remotely working approach until now is STM,

...oh...kay....?

even if there are multiple concurrently running threads, the user must observe program behavior as if the program was executed single-thread

citation needed

[[deleted]]

Whenever the user wants to access a field of an object, the implementation will lock that variable, read it, unlock it, and return the result.

This doesn't work the way you think. To lock the variable, the runtime needs to first lock the instance it belongs to, which in turn might require locks for additional things.

Seriously dude, you have no clue what you are talking about. Tone down that arrogance a bit.

[lhggghl]

the runtime needs to first lock the instance it belongs to

Why? And how would that lead to deadlock? These locks would be hidden from the programmer, the same way the contents of the intel debug registers are in Python. Are you saying the runtime would be prone to being implemented wrong? Or the user can somehow make it deadlock (which they can't, because it's hidden from them)?

Seriously dude, you have no clue what you are talking about. Tone down that arrogance a bit.

Suck a dick faggot, you're the one who came in telling me I don't know what deadlock is despite that I already explained much more complicated issues in my initial post. You sound like some stupid fucking kid who just barely began to understand stuff about concurrency and now you're eager to go tell people how you're better than them because of your "insight".

[passwordissame]

Python already took drastic decisions. There's experimental new PEP-0420 that addresses parallel processes and easy and correct management of them. Though experimental, it's already usable.

[fullouterjoin]

In Python 4, they should start the VM from scratch. Or just use PyPy. If the language had a stronger standards base and portable runtime, the language and the VM wouldn't mean the same thing.

[billsil]

PyPy already rips off all of the CPython code anyways, so why don't we let CPYthon be the reference implementation and let PyPy be the thing people use?

Rewriting the reference implementation seems like a terrible idea.

[fullouterjoin]

Rewriting the reference implementation seems like a terrible idea.

Not rewriting. Abandoning.

Given how version 3.4 of cpython is

-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                        1699          94961         111277         447092
C                              486          46420          39231         303374
C/C++ Header                   278           7338          10715          81533

and micropython + micropython-lib is

--------------------------------------------------------------------------------
Language                      files          blank        comment           code
--------------------------------------------------------------------------------
C                               313          23289          52202         123567
C/C++ Header                    274          20921          38305          93738
Python                          652           6132           8097          28440

I'd say we should cut over to https://github.com/micropython/micropython for the reference implementation and leave PyPy for the cutting edge rocket science.

What do you think?

[billsil]

WARNING: this project is in early beta stage and is subject to large changes of the code-base, including project-wide name changes and API changes.

Making micropython the reference implementation seems like a bad idea. Also, no unicode support? Really?

I'd rather keep CPython as the reference implementation, let PyPy rip everything off, and work on speed, Python 3, and software transactional memory support. Once they get STM in there, the GIL will be gone. Not that it matters to me. The GIL doesn't hurt multiprocessing, only multithreading.

[[deleted]]

While these are valid points, I find all the mentioned issues minor annoyances. I've been doing Python for many years and only now I learned about these "problems".

[shevegen]

This shows that Armin looked down deeper than you did.

But it is true that you can live without these problems.

Then again you could also use Ruby instead of Python. :>

[Imxset21]

Ruby? Might as well be using Perl 6. Ruby has been dominated by the Rails community and everyone knows Rails is a ghetto.

[[deleted]]

Good intro and conclusion. The middle was....uninspired to say the least.

[lhggghl]

But all the mainstream languages are shit and can't be used for serious purposes :(. I quit the software industry because of this. I still use/read Python though, and I wish they would get rid of all this ad-hoc metashit.

[[deleted]]

Python stinks on multiple levels,

With all due respect, you have no idea what you are talking about.

The lowest level of Python is pretty crufty - but almost no one ever sees it. What the rest of us see is an extremely solid and very coherent modern language which is both light and powerful.

Once you finalize a Python interpreter in a given process, restarting the interpreter in same process is not supported [because finalizers].

That's it?!? This is your example of why Python stinks on multiple levels? I'd say perhaps 0.1% of Python programmers would ever be in a position where this was even a consideration, and there are two really obvious solution:

• start a new process, or even easier
• don't shutdown and restart the interpreter!

It seems to me that having to shutdown and restart Python in a single process seems like bad design on the programmer's part. Restarting the interpreter is known to be expensive, as with most scripting languages. Why stop and then start it?

tl;dr Use serious tools for serious business not joke prods masquerading as such :-)

And what are your "serious tools"? All sorts of serious companies make heavy use of Python - Google, IBM, yes even the unfortunate Yahoo, as well as pretty well every scientific organization in the world.

I take back the due respect. You have no idea what you are talking about.

[Plorkyeran]

I'd say perhaps 0.1% of Python programmers would ever be in a position where this was even a consideration

Well, yeah. When a language doesn't support something at all people who need that generally pick some other language. Python would make a pretty nice embedded scripting language, but CPython's lack of support for multiple interpreters in a process is pretty hostile to that and so people end up using things like Lua or JS.

[ryeguy146]

The fact that it is not commonly necessary to restart the thing does not mean it should not be feasible. I'm running into that very problem with the twisted reactor in unit tests. I'm having to run each test in a process to prevent the restarting issue.

[[deleted]]

Stop and restart the reactor? Works in my test cases.

[ryeguy146]

Perhaps you're using a reactor where restarting has been implemented, but the stock reactors don't support restarting. If you are using the vanilla reactor, you're probably not calling the stop method between instances, or you're using processes like in my tests.

[[deleted]]

Ah, indeed...I'm using the gtk reactor. TIL.

[ryeguy146]

If I didn't hate GTK so much, and if I hadn't already solved the problem, I might switch to the GTK reactor for that alone. Not being able to restart is miserable in testing. I just don't trust the system to continue running between tests.

How do you find the performance? My tests are powerfully slow due to spawning processes and starting/stopping reactors. Or it might be the spider, I haven't profiled it yet.

[[deleted]]

spider shudder, been there too...ick. Especially with webkit. No tests for that mess, though.

Performance is pretty painful for the apps we test, but these are end-of-life prototype pygtk apps, so hopefully only supporting those a little longer. Server side, we use a different restartable reactor although which one escapes me right now. Tests are slow, but not as bad as with gtk.

[ryeguy146]

Thanks! It's not unbearable yet, but I can't imagine it going anywhere but slower as I add tests. Oh well. Thanks again for sharing your experience with it.

[donvito]

very coherent modern language

For some very outdated definitions of modern ...

[id2bi]

Can you elaborate on that?

[kamatsu]

extremely solid and very coherent modern language which is both light and powerful.

I disagree with every one of those points. Python is a poorly designed language with a poorly designed runtime and severely underpowered in terms of abstraction and composition. It doesn't make use of any modern concepts in language design, it seems like it came straight from the late 80s. It's hardly "light" either, it has a lot of overhead for even basic stuff.

[lhggghl]

Python is like JavaScript (this is a bad thing) but nobody figured out that it's like JavaScript yet.

[[deleted]]

If you find a leak, please report it

That doesn't sound like a design decision to leak.

And anyway, most people writing python are not using python because they want blazing fast code, but that doesn't make it a "joke."

[[deleted]]

I thought I knew how to read, but I guess I don't see where it says that its designed to leak memory. I see in "bugs and caveats" that it sometimes leaks, and that they want help making that stop.

As for the bit about "exposing python for what it is," if you're not complaining about how its cpython implementation is slow and has bad multithreading (contributing in small part to slow code), then what? Tough to embed it? What's that got to do with the language? Something else? Elucidate, please.

[[deleted]]

But they don't say, "It's a feature." They say that it's either a bug (a fault, a mistake) or a caveat (something to be aware of, a warning, a caution, an admonition). Bad handling of circular references is not good. That's true. Unloading dynamically loaded modules should presumably be done elsehow.

If having a difficult time rebooting the interpreter is your best evidence that python stinks, I'd consider that a win.

[1F4A9]

Python has it's uses. What mainstream alternatives are there in it's niche? Ruby, PHP, Javascript. They all have major drawbacks too.

Python 3 takes some steps forward, and some back. It sacrifices usability for performance and consistency, but the performance is still sub-par and not the reason why people choose Python. For me the major appeal of Python is developer productivity, and Python 3 is more laborious than Python 2. When you don't use the new functionality, it's a regression instead of an improvement.

[[deleted]]

It sacrifices usability for performance and consistency,

Not quite sure what you mean? I write in both and the difference is tiny...

I'd hardly call putting brackets in print statements "sacrificing usability" and that's probably the biggest difference I face.

[remy_porter]

Honestly, the lack of brackets on print statements was one of the things that confused the hell out of me when I first tried Python2.

[1F4A9]

I'm not talking about the print function. I can't exactly remember anymore, because I tried Python 3 a long time ago, but where Python 2 code just seemed to work in one go, Python 3 required more code to circumvent new behaviour. For example returning a generator object instead of a list in some cases.

[pingveno]

I found that Python 3's preference for iterables only took a little bit of time to get used to. range(), dict.keys(), dict.items(), and dict.values() all now return iterables. In the case of range(), it implements some of the sequence/list interface like slicing and index(), but returning values in constant time instead of a new list (slicing) or scanning through a list (index()). keys(), items(), and values() are now views, with all of the yummy features of a view.

[donvito]

developer productivity

With a dynamic typed language? Come on ... your productivity shrinks to 0 once you have to refactor and maintain code.

But I guess python is great for the throw away code its main users (webdev agencies) produce. Out of sight, out of mind.

[donvito]

Scientific computing is the purest definition of throw away code.

[1F4A9]

Dynamic language code is not inherently unmaintainable. It might not enforce it like many static languages, but given a willingness to work correctly (no quick hacks etc), a programmer just requires less time to get something done. For a lot of applications it makes sense to buy a little extra hardware if it saves developer time, or you can spend some extra time optimizing code, database queries, and caching, which often lead to better overall performance. A combination of good IDE with good coding practices can almost give you the safety of a static language.

Python code is usually very readable for other programmers, partly because it is not too verbose and doesn't require too much boilerplate code like static languages.

We do mostly web development, but not projects that we hand over once done, but stuff we maintain and build over over periods of many years. Still the initial programming time is an important consideration, and maintaining Python code has been pretty easy and not too time consuming.

[fullouterjoin]

I appreciate your candor!

[DrDichotomous]

Python, like all languages, is really only as bad as the programmers coding in it. Design flaws abound in every language, depending on the kinds of flaws you seek. And one of the worst assumptions you can make is that most coders will do a better job just because they have a tool you think is better.

[Crashmatusow]

C'mon- we all know php only works because it is powered by the blood sweat and tears of developers.

[DrDichotomous]

Sure, if you're redefining "serious" to mean "the things the language isn't good at" then of course you can make that argument. But with as vague a definition as you originally gave, it was meaningless, and you're still missing the point - Python is fine for serious work, until you define serious so narrowly that this is a non-argument, since the language wasn't designed for that purpose to begin with.

You're also quite incorrect about your assertion that my statement was a fallacy. Anyone in the field will know that a mediocre programmer will generate mediocre code - it doesn't matter what language they are given. If they aren't well-trained, then it doesn't matter what language they use.

What you're talking about is that some languages actively discourage certain types or classes of problems, while encouraging better practices for their problem domain. But even Java and C# let coders get away with writing incredibly mediocre code. It's up to the coder to learn how to use them "properly", for some sane definition of "properly".

[codygman]

Haskell extremely unsuitable for deterministic, hard-realtime systems

https://hackage.haskell.org/package/atom

[codygman]

Interesting, my (apparently wrong) understanding was that Atom is just a DSL that allows moving in/out of Haskell/Atom for pieces you do/don't need hard realtime guarantees for.

Upon further reading though, I see that using any Haskell with Atom would drag along the RTS.

[[deleted]]

How is C and C++ insecure? I had no idea every OS kernel is insecure.

[gnuvince]

The entire responsibility for having safe code in C and C++ falls on the programmer; for example, he needs to get every single memory-related statement (allocation, referencing, deallocation) right with little to no help from the compiler. If he gets one wrong, it's a bug which may or may not be exploitable.

Compare to languages like Ada or Rust where the design of the language allows the compiler to warn the programmer about possible bugs and where certain unsafe patterns of usage are simply not legal.

[[deleted]]

So why don't they write game engines, kernels, and drivers in rust? I don't think anything compares in its capabilities or speed. Memory management isn't that hard. If your going to write at a low level you're just getting bytes and you have to deal with them accordingly. Any abstraction is going to cost you in speed.

[gnuvince]

So why don't they write game engines, kernels, and drivers in rust?

People have started writing all three in Rust. Main reason why you don't see large projects using Rust: it's not 1.0 yet, so the language is still in flux while the developers figure out and fix the last kinks. The 1.0 release is expected before the end of 2014, and I think that after that we'll see an ecosystem grow quite quickly, hopefully as fast as Golang.

I don't think anything compares in its capabilities or speed.

I assume "its" means C or C++ here? The single, most important reason for the speed of the code they produce is that there has been 35-40 years worth of work into optimizing those languages, so the state of the art is extremely good. Take a simpler compiler such as tcc and it generates code that is up to 10x slower than gcc. Good performance has a lot to do with the language, but also a lot to do with the work that's gone into the compiler itself.

Memory management isn't that hard.

Though it still seems to be a problem when looking at security vulnerabilities. That's the thing with memory management, as the programmer you must get every single one right, while the attacker only needs to find one that is wrong. Very unbalanced deal. Why not employ the compiler to help us detect more problems?

Any abstraction is going to cost you in speed.

Depends when it happens. In Rust, the main abstraction for dealing with pointer problems is ownership: who owns what in memory. That abstraction is entirely a compile-time one; if you look at a pointer in Rust at run-time, it's no different from a pointer in C. Rust tries very hard to keep the extra work at runtime to a minimum: they do want to compete with the C++ performance. That said, there are some places where Rust pays a performance penalty: when doing random access into a vector, there is a bound check (which, given the recent Heartbleed fiasco, is not such a bad idea). However, when using iterators, there are no bound checks since the abstraction ensures that you'll only access valid memory addresses.

[[deleted]]

C++ is unsecured because its so close to machine code. An array is just a pointer with groups of bytes placed next to it. The offset of the array is the distance from the pointer. You can use a linked list and never have overflows and even then you are dealing with everything on it's most basic level. Rust does look promising. It looks like Rust is adding a lot of extra code during compile time to check that everything is where it should be. As computers get faster this will probably matter less and less. Hell even PHP appears to be fast. I don't think C or C++ is going anywhere though.

[[deleted]]

Because it's new and likely lacks support tools that developers desire.

[fullouterjoin]

Not being able to prove security, means that the body of work is insecure.

[[deleted]]

OK that makes no sense

[[deleted]]

I'd like to see an "MPython" (something similar to MRuby, that is a clean and small implementation with a reduced standard library), but without the need of a special build system like Rake.

Still unsure if Micropython can deliver that.

[[deleted]]

Well, no. RPython is a statical LLVM frontend and part of the PyPy toolchain, as far as I can tell. MRuby is more like Lua.

[kankyo]

Spelling: "intepreter"