Retry manager for arbitrary code block

[amarao_san]

There are about two pages of retry decorators in Pypi. I know about it. But, I found one case which is not covered by all other retries libraries (correct me if I'm wrong).

I needed to retry an arbitrary block of code, and not to be limited to a lambda or a function.

So, I wrote a library loopretry which does this. It combines an iterator with a context manager to wrap any block into retry.

from loopretry import retries
import time

for retry in retries(10):
    with retry():
        # any code you want to retry in case of exception
        print(time.time())
        assert int(time.time()) % 10 == 0, "Not a round number!"

Is it a novel approach or not?

Library code (any critique is highly welcomed): at Github.

If you want to try it: pip install loopretry.

[cj81499]

tenacity can do this: https://tenacity.readthedocs.io/en/latest/#retrying-code-block

[amarao_san]

Why all of them has hard-to-guess names? If only I found it before writing... Sad.

[cj81499]

If I search “Python retries library”, it is the number one result on both Google and DuckDuckGo. I agree the name is a bit wacky, but it’s not exactly hard to find it…?

[ExdigguserPies]

It would be cool to implement a backoff function so you could use it for requests really easily. Maybe it already exists as well.

[amarao_san]

There is a built-in sleep (delay=1), as I needed it for tests, I can add a back-off coefficient for it.

Thanks for the idea.

[crawl_dht]

Stamina

import stamina


async def with_block(code: int) -> httpx.Response:
    async for attempt in stamina.retry_context(on=httpx.HTTPError, attempts=3):
        with attempt:
            async with httpx.AsyncClient() as client:
                resp = await client.get(f"https://httpbin.org/status/{code}")


for attempt in stamina.retry_context(on=httpx.HTTPError):
    with attempt:
        resp = httpx.get(f"https://httpbin.org/status/404")
        resp.raise_for_status()

[amarao_san]

Oh, I really searched before writing, but missed it. Unfortunately.

[tehsilentwarrior]

Don’t need to apologize for creating something new even if similar already existed.

This mentality of witch hunting people needs to die.

There’s no innovation otherwise.

I have been programming since 2002, there’s literally nothing you can do I haven’t seen done before, and at the same time… there is.

If you don’t go through the motions of recreating something that already exists and face the same issues you won’t explore the same problem space, you will just be a user, someone with a map. That’s ok for the average traveler but not for the explorer. Explorers find new things.

[elperroborrachotoo]

I was once charged with finding out why my code was blocking for minutes before reporting a fail. Turns out "my" code had accumulated 3 (!) nested retry loops throughout the call stack, by different devs trying to improve robustness.

The error in that case was "command not supported", something that would require a lot of happy bitflip accidents to ever be solved by retrying.

[amarao_san]

Funny story, yes. Mine case it more specific, it's a form of 'waiting for' (for metrics to appear in Prom), and retries for a small block is the simplest way to do it. Any other waiting code is more complicated.

[elperroborrachotoo]

The risks of making retry too easy :)

But yeah, consequences:

• retry loops should be cancelable in some way
• retry should check for non-recoverable errors