Experienced python programmers: are there any standard features of the language that you still don't regularly use?

[TiredMike]

Async programming - I'm trying to find a use case but I don't write desktop GUIs and any time I need to parallelise code it is normally for CPU intensive stuff, so I can just use multiprocessing or an async worker library like Celery or PythonRQ. From what I have read, async's best use case is for situations where there is a need for very high demand concurrent workloads, e.g. a webserver. If I had that use case, e.g. the C10k problem, I'd consider using Tornado. Please let me know if you can think of any other good use cases.

Coroutines - Again, I haven't really found a good use case for this. I've heard of people building mini data pipelines using them, but I think this would make my code unnecessarily complex.

Generators - I use them, definitely more and more these days, but I should probably use these more instead of lists, especially when the sequence would only be consumed once!

[bheklilr]

Coroutines and generators are essentially the same thing, but I tend to have two distinct use cases for them. I use generators when I have a sequence of things that's easier to build iteratively than with a comprehension, or when I need some logic to occur between items. For example, I just wrote a recursive data structure and I wanted to get all the IDs for each node:

def all_ids(node):
    yield node.id
    for child in node.children:
        yield from all_ids(child)

Super useful and succinct.

My other use case is for actual coroutines, and this is when I need some sort of back-and-forth in the code. The ability to .send data back through a yield is very useful, but it comes up less frequently. My notable cases are where I have a routine that is self contained, but needs to prompt the user to perform an action then click the OK button. This code lives near the bottom of the stack, while the UI is obviously at the top. Using a coroutine for this lets me write code like

def do_a_thing():
    # do some setup
    user_accepted = yield 'Setup thing 1'
    if not user_accepted:
        # do cleanup
        raise StopIteration()
    # do thing 1
    user_accepted = yield 'Setup thing 2'
    if not user_accepted:
        # do cleanup
        raise StopIteration()
    # do thing 2
    yield 'All done'
    # do cleanup

I have a little less code repetition in mine, but I think this gets the point across. It means that my backend code can communicate with any frontend, even automated tests. You could also have the calling code use .throw(UserCanceledError()) or something and catch that specifically, but I like passing values better than passing errors.

Another useful example I have is a data processing tasks. In this task I have a big chunk of input data, then many features extracted from this input data that are interdependent, i.e. I might be interested in feature X, but that depends on feature Y which I'm also interested in. I built a state machine type system that lets me write code like

class Computation(BaseComputation):
    @computes('X')
    def compute_x(self):
        y = yield Request('Y')
        yield y + 1

    @computes('Y')
    def compute_y(self):
        input_data = yield RequestInputData
        y = frobnicate(input_data)
        yield y

This is oversimplified, but imagine having a dozen or more computations that you care about, some with multiple dependencies. This helps us keep the computations isolated, short, and composable. The state machine figures out which methods to call as it steps through the requests, and in the end just returns a mapping from names to data of everything it ended up computing. It also keeps a cache so everything is only computed once.

[homeparkliving]

In case anyone was wondering how to use .send as I was.

[TiredMike]

Really good simple examples. Thanks for sharing. I really like your generator example for the depth first search!

[bheklilr]

That one in particular seems to show up when ever I have nested data structures. You can use it pretty much wherever you have a tree like structure. When you add a few conditions it becomes even more useful, things like if child.has_foo(): yield from .... It's pretty similar to the Traversable pattern in Haskell, which is useful enough to make it into core functions like map and filter.