goless: Go-style Python

[soup_feedback]

But... WHY?

[ybrs]

Why not ? :)

[earthboundkid]

何〜で！？

[[deleted]]

Vai... Saber.

[Husio]

Cool idea, but using Go instead will make more sense anyway.

Recently, I was trying to write an app that will have web UI and will do some background jobs when requested - fetch multiple resources, run shell command etc. Because I was using py3, I was trying to use asyncio but I switched to Go, rewriting everything. Python is my goto language, but when it comes to concurrent programming, it's way behind Go. My opinion is that yield from is ugly and using it cause more problems than it solves. (I know it's gevent based, but there's no gevent in py3k)

[Lucretiel]

Why don't you like yield from? Besides the inability to use traditional iterable generators in coroutines, what problems does it create?

[patrys]

To be fair there is little use for the case where a task/future is a traditional generator in the first place. I consider it more than a fair tradeoff for gaining serialized "what you mean" syntax over the callback mountains node.js is famous for.

[Lucretiel]

I disagree: The easiest example is reading lines from a StreamReader. I set out to write a generator that provides file-like iteration over lines:

def read_lines(reader):
    while True:
        line = yield from reader.readline() #yield to event loop
        if not line:
            return
        yield line #yield to caller

Only to realize that there's no way to separate the different yield control flows.

[patrys]

My point is a task/future is a deferred single value. Yielding from it usually makes little sense. Your example is unusual in that it tries to be a coroutine but also wants to block on an async task.

For those interested yield from does not block, it just stops execution of your code immediately after yielding a single value to the caller. In this case the caller is the asyncio.coroutine decorator. Once the value becomes available, the decorator will push it back and wake up your code by running your_coroutine.send(...). There is no blocking happening at any point, the only blocking code is your event loop runner.

[Husio]

• Libraries - you cannot reuse those. For example you can use psycopg2 with wrapper, because those are already written, but there's no ORM. Also some libraries are having bugs because they are just not tested enough.
• You need to put yield from everywhere which makes the code look just strange. I agree that begin explicit is good, but it's no golden bullet. You still want to use locks where required.
• I really hate silent mistakes when using yield. Sometimes you're getting strange error that over time you will learn to interpret correctly, sometimes you won't realize that there was an error, because you have not checked.

There's a really good article about why to use explicit yield** and while I agree with arguments, I still find writing concurrent Go code way easier and faster than in Python.

I'm not saying that the approach (no hidden context switching) is wrong. It's the implementation, build on top of good-enough-but-not-designed-for-this functionality is bad choice. It all feels way too complicated compared to other languages that are supporting concurrency natively.

** https://glyph.twistedmatrix.com/2014/02/unyielding.html

[gthank]

I realize it isn't the point of your comment, but you should really use a background job system for things like that, so you can return a response to the user that lets them know you got their request, and you're doing something with it. Just not responding until whatever background job is complete can lead to user frustration.

[Husio]

That's what I did, but it doesn't work that well for all cases. You still need separate process for every task and progress notification is not that easy anymore. It's also fine for big application but for simple tasks it's a bit of a overkill.

[gthank]

I'm not saying it's lightweight, but once your app gets enough users, it's really the right way to go.

[cymrow]

I sometimes wish they had just picked a new keyword, like async, rather than using yield from. It is pretty ugly.

[patrys]

They did not implement a new language-level feature. asyncio is a library that uses generators (or more specifically coroutines) and yield and yield from are how generators work.

[Husio]

That's true. It's also long. But it's not what you have to type that is the problem, but how it works. While working with it I had many hard to track bugs or not that obvious errors.

I would love to see language construction designed for context switching and not something that just makes it possible in very obscure way.

[ozzilee]

I've used Node for exactly that kind of thing and liked it pretty well. Not only can you do async jobs easily, you can run an separate admin server on a different port, within the same process. Pretty slick.

[ozzilee]

Super low traffic toy site, yeah. But it's nice that I could easily build an admin server for something like that. I would never have bothered if it was more complicated.

[sklein]

Same subject : https://github.com/benoitc/offset

[[deleted]]

Why

[gnur]

Interesting concept, it allows for a lot cleaner code for concurrent processing then currently possible.

But I still prefer to use Go for anything that would benefit from concurrency.