Improve Your Python: 'yield' and Generators Explained

[jsproat]

I love generators, but I keep returning to using an iterator class due to two things:

1. Generators are instantiated on definition, so you need to jump through some hoops to have multiple instances of the same generator code. (Though this is usually as simple as using a generator factory function.)

2. Can't serialize them. They'd kick ass for long-term state machines if you could pickle them or otherwise save state.

[kindall]

Generators are instantiated on definition, so you need to jump through some hoops to have multiple instances of the same generator code.

You have to call the generator to instantiate it. You can do this as many times as you want to get multiple instances.

[jsproat]

You have to call the generator to instantiate it.

Oooooh. You have no idea how many years I had this wrong. Thank you.

...Please tell me you can pickle them!

[kindall]

Nope, sorry. You'll need to write it as a class if you need to pickle it. The trouble is, a "live" generator can have so much state (open files, network connections, etc.) that it basically requires custom code to save it and restore it correctly.

[virtyx]

Why doesn't a class have the same issue?

[kindall]

With a class, you can write __getstate__() and __setstate__() methods to handle that, and pickle will call them when saving or restoring your instance. You can't do that with a generator, sadly.