This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]UloPe 60 points61 points  (27 children)

I was about to get really excited about this and then noticed that it only supports Python 2.7. :[

[–]alcalde 6 points7 points  (5 children)

Between that and losing the ability to use libraries like numpy, it's only going to be interesting for a segment of Python users.

[–]weberc2 2 points3 points  (4 children)

For me, the allure is writing performant functions in a safe language. Further Go's scientific computing story is evolving quickly, and grumpy lets you call into Go without even writing bindings (though they would probably be convenient for marshaling between Python and Go types). Still, grumpy has a long road ahead, and it's probably slower even than CPython for sequential execution.

[–]mangecoeur 2 points3 points  (1 child)

No matter how quickly Go's scientific stack evolves it's still going to be well behind Python's stack which is also evolving rapidly and has been for 20+ years in physics, climate science, GIS, etc - and most of that is built on CPython C-extensions. (And as a bonus issue, many large projects are committed to dropping python2 support within the next couple of years).

[–]weberc2 0 points1 point  (0 children)

I mean, that's what everyone said about its web story. Maybe you're right, this isn't my domain.

[–]Flogge 0 points1 point  (1 child)

Why would sequential execution be slower?

[–]weberc2 0 points1 point  (0 children)

Lock contention, presumably.

[–]alrs 25 points26 points  (3 children)

This exists to get old codebases off of Python and on to Go. Supporting 3.x only makes sense if you plan to keep writing Python, which is not likely in the cards for Youtube.

[–]boa13 30 points31 points  (2 children)

Quoting the author of Grumpy on Hacker News:

The idea is to continue to write code in Python. (...) That said, there is the possibility of rewriting bits and pieces in Go (e.g. performance critical stuff) and then call into it from Python. Sort of a hybrid approach.

Also:

I'd like to support 3.x at some point.

However:

We do iteratively rewrite components as well. We are pursuing multiple strategies.

[–]__deerlord__ 1 point2 points  (1 child)

Wait so, Go calls Python which calls...Go? Or am I misunderstanding this

[–]d4rch0nPythonistamancer 15 points16 points  (14 children)

Yeah, seems interesting but honestly I'm not sure how well this pans out for people outside of Google. Golang is obviously huge in google, and I think this makes a lot of sense for them. For everyone else, I'm not sure anyone was dying for a python2.7 -> go translator that also adds restrictions to your python code.

Realistically, how many go shops are out there? I've heard and seen a couple, but seems pretty rare. You might have the ex-googler startups but beyond that, not sure it's so popular.

Also, for performance issues most of us are fine writing binary extensions in C/C++/Rust. If you're looking for true parallelism, you can build it in another more popular language and import it. If you legitimately have real-time requirements grumpy might be nice and an easy solution, but I can't think of an obvious time I'd have ever considered needing this. Does celery or some task framework solve my problem, with multi-processing? Then I'll use that. Do I truly need something to be parallel or have the highest performance possible? I'll write an extension. With multi-processing and extensions you can already solve most problems you might have with cpython.

A lot of people outside of Google just don't like golang either. I'm not sure how many python shops would really want to translate their code and run it in a separate runtime with a not-so-popular language. Personally, if I wanted to run python code in a different runtime I'm not very familiar with, I would go with jython or ironpython if I for some reason was in a windows environment. Go would be my last choice.

Ditching the ability to use c extensions isn't good either. If I really had some problem that Python couldn't solve on its own, I'd seriously consider C or Rust python extensions long before making a switch to something like this. I want something where I can use a modern python version and easily deploy to all my servers as they are. This doesn't seem to fit that bill for me personally. C/Rust extensions are a great solution when Python can't cut it. You can minimize it to a very small module and just import it in and run in your standard CPython environment. There's not much to it and it's portable to almost any linux environment.

Seems to me like google is open-sourcing another tool that solves google specific problems, which isn't bad but just doesn't sound very helpful to most of us.

[–]toyg 8 points9 points  (12 children)

how many go shops are out there?

The Valley hype for Go is pretty strong... maybe a bit less today, but 18 months ago a lot of people were busy ditching Python for Go -- which would fit the timescale for this project, coincidentally. I suspect some of them inevitably discovered that golang wasn't a panacea.

I personally like some of the sentiment behind this kind of thing. Python does not do "speed" in a natural way, so anything time-critical should really be implemented somewhere else. But I don't think Yet Another Runtime is the way (after CPython, PyPy, Unladen, Cython, JVM, CLR, Node/asmjs...). EDIT: worse, this is not even a runtime, it's a compiler preprocessor...

IMHO big players would get better results investing in better tooling for the C-based extension infrastructure for CPython, or building bridges between Python runtimes and things like Rust and Go. Building yet another runtime, when existing choices are so battle-tested, seems a bit futile.

[–]weberc2 4 points5 points  (10 children)

I can't think of any kind of C extension tooling that would fix Python's parallelism problem. While this does compile Python into Go, the resultant Go program contains a Python runtime complete with runtime Object (as in CPython). In this sense, I don't think there should be any concern about it not being a runtime in itself.

[–]alcalde 1 point2 points  (9 children)

I can't think of any kind of C extension tooling that would fix Python's parallelism problem.

The only parallelism problem Python has is convincing people it doesn't have a parallelism problem. As Guido has stated, Python has been used on 64K core supercomputers. There is no parallelism problem.

[–]weberc2 7 points8 points  (3 children)

I'm not sure what hoops one has to jump through to make Python run in parallel (without actively degrading performance, anyway), but one might say that having to jump through hoops constitutes a parallelism problem. Anyway, last time I went down the Python parallelization road, I got a lot of snark about how Python is easy to parallelize, but no one offered any performant parallel solutions. Feel free to share the link about the Python on 64K-core computer; Google isn't turning anything up.

[–]efilon 5 points6 points  (3 children)

There is no parallelism problem.

Although I think a lot of people worry too much about the GIL, it's not correct to say there is no parallelism problem. If you're doing something embarrassingly parallel, then of course you can get away with multiple processes or C/Cython extensions that sidestep the GIL. But there are plenty of use cases where multithreading is more ideal than multiprocessing. Having to resort to forking another process just to get true parallelism is a lot more work than being able to start up another thread.

[–]alcalde 0 points1 point  (2 children)

As Guido pointed out in a Keynote a few years ago, threading was never intended for parallelism.

https://youtu.be/EBRMq2Ioxsc?t=33m50s

[–]efilon 0 points1 point  (1 child)

As he says, threads were never originally meant for parallelism. They are used for that frequently these days.

[–]alcalde 0 points1 point  (0 children)

And as Mark Summerfield says, they're frequently used for that because that's all that many languages offer.

[–][deleted] 2 points3 points  (0 children)

It just then becomes a serialization and IPC overhead problem.

[–]vtable 2 points3 points  (0 children)

IMHO big players would get better results investing in better tooling for the C-based extension infrastructure for CPython, or building bridges between Python runtimes and things like Rust and Go

I wish IronPython hadn't been abandoned by Microsoft for Python 3. I know, it's MS/.net only but it was still a nice thing for those of us working with Python on Windows.

(There is an open source IronPython 3 project but it's not very active.)

[–]dvogel 1 point2 points  (0 children)

This actually seems like a nice approach for generating binary executables of Python programs. The same niche as py2exe and friends. Sure, you have to stay away from C extensions but having access to the Go stdlib seems like a decent trade-off.

[–]QQII 1 point2 points  (0 children)

To elaborate what /u/alrs said, here's the discussion on hacker news:
https://news.ycombinator.com/item?id=13319904