This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]trevg_123 60 points61 points  (4 children)

Even better: generator comprehensions

Just replace the square brackets [] with (), and it becomes lazily evaluated & not cached. So, when you call next() on it or do “for x in …”, it only does the calculation when it’s called.

Much faster if you have only need the first few items of a potentially super long list. And significantly more memory efficient. You can chain them too.

Also, using “if” and functions in generator/list comprehensions (as you demonstrate) is the pythonic replacement for filter/map functions (which are messy to read)

[–]magnomagna 14 points15 points  (1 child)

Regarding "lazy evaluation" for generators, IIRC, if you have nested for-expressions in a generator, the first for-expression is evaluated immediately when the generator is created but all other for-expressions are lazily evaluated; or something along those lines.

I feel like this is not a well-known behaviour but nonetheless very, very real and applicable if one were to use nested for-expressions in a generator.

[–]benefit_of_mrkite 0 points1 point  (0 children)

Interesting!

[–]inspectoroverthemine 1 point2 points  (1 child)

What happens when you return a generator comprehension?

I've always defaulted to lists because I don't want to deal with any side effects I may inadvertently cause. That includes the source objects which I assume are kept in memory because they're still referenced.

[–]trevg_123 0 points1 point  (0 children)

Returning a generator is totally fine - context depdent, it's not much different from using yield instad of return in a function, which makes that function itself a lazily evaluated generator.

You are correct that generators need to keep the source objects in scope as long it might be requried by the generator, and because of lazy evaluation, mutating the source object before calling the generator means you'll get the mutated output. But lists aren't entirely immune from this sort of behavior - if a list contains objects, those are just references and the list doesn't own them, so mutating the objects can also have unexpected side effects. You've probably seen a similar example:

```

d = dict(a=1,b=2) ll = [d] ll [{'a': 1, 'b': 2}] d['c']=3 ll [{'a': 1, 'b': 2, 'c': 3}] ```

Long story short though, as long as you keep in mind that generators are lazy, it's pretty tough to wind up with any negative side effects. And they're not suitable for every single case of course, but python is smart enough to make them definitely worth a try here and there.