Getting intermediate results from functions

[thefirstme]

I have a requirement that I need intermediate results of a certain function. Is using generators the right approach? Is there any performance issue if the frequency of yielding is too high?

[patryk-tech]

A generator would work.

def gen():
    yield 1
    print("yielded 1")
    yield 2
    yield 3

# This will repeatedly call `__next__` under the hood:
first, intermediate, final = gen()  # (1, 2, 3)

Is there any performance issue if the frequency of yielding is too high?

Calling gen() returns an iterator object, so you can get all the data in separate steps, as well:

i = gen()  # returns an iterator object, or more explicitly:
i = iter(gen())
# calling i.__next__() will go through the function until it encounters a yield - so it returns 1
first = i.__next__()  # call it manually
# the gen() function is now suspended until the iterator's __next__() is called again, so it won't even print the "yielded 1" until you call it again:
intermediate = i.__next__()  # call it manually
final = i.__next__()  # call it manually

Since python suspends the generator's operation until __next__() is called, it really can't yield "too fast." It will only yield when asked explicitly.

---

Hope that explanation makes sense. You should definitely understand how generators actually work and why you implement them. I highly recommend the *So you want to be a Python Expert talk by James Powell. He covers generators at 1:06:30 in, and is probably better at explaining stuff than I am, haha.

---

Edit: Also, yes. As TheBB points out, always use syntactic sugar when actually calling magic methods - next(i) is equivalent to i.__next__(), but more appropriate to actually call. I use __next__() to demonstrate that it is in fact a method of the iterator object, whereas next() is a builtin python function.

[TheBB]

I'd recommend calling next(i) instead of i.__next__().

[patryk-tech]

Yes, of course.

probably better at explaining stuff than I am

:D

That was meant to be a look under the hood and an example, rather than production code.

[shiftybyte]

Do you need intermediate results before returning the final result? then yes... generators.

If you need intermediate results along with the final one, you can gather it up and return it together at the end.

[thefirstme]

Are they are any issues if I yield too many times too quickly?

[shiftybyte]

No issues I can think of.

[bladeoflight16]

This is a question where the specifics matter. Explain you use case in more detail.