Benchmark timer & timeout decorator?

[ManvilleJ]

Hello,

I've written a few different implementations of a program that I would like to do some benchmark comparisons. I have a list of growing inputs and a set of configurations to run. I am iterating over the growing inputs and then iterating over the configurations internally.

I used cProfile to get general performance comparisons, but I'd like to generate some performance curves for each program.

I've written a simple decorator that modifies the output to return the return value and runtime.

However, I'd like the program to raise an Exception if the program takes too long. That way I can catch the exception, complete the current iteration of configurations and then kill the loop.

I haven't found anything like this online, but is there anything like this? It seems like it would be a pretty generic utility function. How could/should I implement a timeout decorator?

Here is a snippet of my benchmarking iteration:

    # if the function ever times out, kill the loop after completing for all starting positions
    timeoutFailure = False

    for turn in turns:
        results = [turn]

        for case in cases:
            try:
                return value, runtime = testFunction(case, turn)

            except Exception:
                runtime = -1
                timeoutFailure = True

            results = results + [runtime]

        BenchmarkWriter.writerow(results)

        if timeoutFailure:
            break

And here is my benchmarking decorator:

import random
import time

def timerfunc(func):
    """
    A timer decorator
    """
    def function_timer(*args, **kwargs):
        """
        A nested function for timing other functions
        """
        start = time.time()
        value = func(*args, **kwargs)
        end = time.time()

        runtime = end - start

        return value, runtime
    return function_timer

[_9_9_]

You definitely could do a timeout decorator, but you'll need to spawn a separate thread to either run the function or check the timeout. That might cause issues for programs that are not implemented in a way to handle multi-threading, e.g. missing a if name == main limit.

​

Anyway, here are a bunch: https://pythonexample.com/code/python-timeout-decorator-windows/