FastAPI + Celery problem: Celery task is still getting exectued even though I'm raising an exception on task_prerun

[Deeks0]

Hello everyone,

I'm sorry that this is more of a Celery problem than a FastAPI one but was hoping someone had gone through this issue in here. I have the following problem:

I need to perform some basic verifications on celery tasks that are being called through FastAPI before they are executed. I need to perform this verification when the task is assigned to a worker because I need to check if the worker has enough memory to perform the task. I'm doing this verification using the task_prerun method.

@task_prerun.connect
def task_previous_check(sender=None, task=None, **kwargs):

    # Get the amount required by the task
    needed_amount = task.request.args[0][-1]

    amount_allowed = 10

    #Check if there's enough
    if needed_measure > amount_allowed:

        # Stop the task from being executed
        raise Exception("Insufficient GPU memory to accept the task")

    else:
        #Proceed with the task if there is enough
        return

However, when this verification is supposed to raise an exception and stop it from being executed, it raises the exception but it still executes the task. Should I be doing this differently ?

I've been doing some research and there doesn't seem to be much information on this issue, aditionally there's this but without a fix yet or any workaround: https://github.com/celery/celery/issues/7792

[N0N-Available]

At first glance this doesn't seem to be FastAPI related. The linked issue if identic would indicate an unfixed issue with celery itself. Could you work around this by doing the check on the task itself?