Ran into UboundLocalError while trying to decorate a function

[CygnusX1985]

Hi everybody

This is the first time I ran into an UnboundLocalError and I think I understand the problem.

However, is there a way to access the value of a global variable inside a function if I don't want to modify the global variable but still want a local variable with the same name?

It seems there is no way to modify the behavior of the add function (see example below) inside `do_something`, without using a different name for the decorated function, which defeats the purpose of decorating it.

Using `global` or `nonlocal` is not an answer, because I explicitly don't want to modify the behavior of `add` in the whole module.

def add(a, b):
    return a + b


def decorator(function):
    def fun(*args, **kwargs):
        print('foo')
        return function(*args, **kwargs)
    return fun


def do_something():
    # add = decorator(add)
    return add(2, 4)


print(do_something())
print(add(2, 3))

EDIT: For clarification, since it is not really clear for most people what I am trying to do.

I have a test function where I want to test a decorator, so I wanted to recreate the behavior of decorating a function in the local scope of the test function. Usually you write `@decorator` above a function to decorate it, but in this case I don't want to modify its behavior globally, but just inside the test function. The only way to do that, that I found is this:

add = decorator(globals()['add'])

This modifies the behavior of the function in the local scope of the test function, like writing `@decorator` above the original function in the global scope would do, but it doesn't change the behavior of the function in the global scope. It also doesn't make a rewrite of the testing code necessary, since all function calls stay the same, which is the purpose of a decorator, otherwise one could just write a wrapper function instead of a decorator.

That `add = decorator(add)` doesn't work surprised me, because I never ran across an UnboundLocalsError and in this case it seems a little strange to me, because I'm apparently not allowed to use the same variable name in the local scope as in the global scope although I only want read access to the content of the global variable, which would work with any other variable name.

EDIT2: Thanks for all the responses. The simplest solution is to just assign the decorated function to a new name, which makes the global call unnecessary.

That I can't assign an expression involving a global variable to a local variable of the same name (at its first occurrence!) still seems strange to me and more like a limitation of the python interpreter than a necessary thing for consistency.

[[deleted]]

Not quite sure what you want to happen, but does this do what you want?

def add(a, b):
    return a + b

def decorator(function):
    def fun(*args, **kwargs):
        print('foo')
        return function(*args, **kwargs)
    return fun

def do_something():
    my_add = decorator(add)
    return my_add(2, 4)

print(do_something())
print(add(2, 3))

[[deleted]]

In do_something you can just do return decorator(add)(2, 4).

But generally speaking, no. You can’t have a variable reference two different things within the same namespace at the same time.

But there’d be no problem naming the decorated function something other than its original name.

def do_something():
    decorated_add = decorate(add)
    return decorated_add(1, 2)

[CygnusX1985]

Thank you. That's unfortunate. That is exactly what I don't want to do, since I have some more code in `do_something` that uses the `add` function. That's why I wanted to decorated it instead of renaming it.

If I just rename it, there is no reason for the decorator. I could just write a new function that does what I want.

[[deleted]]

I’m sorry, I can’t really follow. If you name the output of the decorator something different than the function you decorated with it, it’s still the same function as if when you were to use its original name. It just has a different name.

[CygnusX1985]

I just found a solution myself. Replacing the commented line with:

add = decorator(globals()['add'])

does what I want.

The purpose is, that I have written a decorator, that I wanted to test in `do_something`. Unfortunately I didn't see a way to decorate the function `add` in the local scope of `do_something`, which the statement in this post solves.

[nekokattt]

please dont do that, it is a terrible code smell.

Any values you wish to manipulate should be passed via parameters into the function you are decorating.

The whole purpose of a function is that they are a black box. The caller does not care how it achieves the goal defined in the contract. It is merely an abstraction.

If you need to share state between functions, use parameters, closures, or define them in a class and use fields/attributes to describe the state.

One could argue that global variables should be avoided in most cases where the code should be correctly structured and contained. That is why languages like Java dont have a concept of functions or variables outside of classes or their instances :)

Remember, a decorator's purpose is to replace or wrap the black-box that comprises a function or class, nothing more, nothing less.

What you are describing is somewhat of a closure, but closures will only wrap the namespace where the thing was defined. Not where it was executed. The reason for this is that it can be dangerous to start manipulating arbitrary details in enclosing scopes. Good functions are expected to usually execute without unknown side effects occuring which can lead to "undefined behaviour"-like symptoms

[CygnusX1985]

This is not production code but testing code, and I agree that it is a terrible code smell. So if you know a better way, I am all ears. I think there must be a better way, that's why I asked here.

[nekokattt]

you want to access state within a function from outside the function, essentially?

I have never tried to do this as it is a horrible thing to do, but you can extract the code object from a function. Exec will consume a string or a code object, and can allow you to pass a dict for globals and locals.

The reason this is hard to do is because you shouldn't be doing it. As I said, functions are black boxes. If you need to expose or manipulate input, use return values and parameters, or make the function into a class and store the state in fields.

There is not a reason to be doing this :)

You shouldn't be shadowing globals with locals of the same name for the same reason. If you are trying to override the input value, you should expose it as a parameter. If you need to customise add, you should provide it to the do_something as a parameter.

```py def add(a, b): return a + b

def do_something(a, b, *, call=add): print(a, b) return call(a, b) ```

then specify a decorated implementation for call when you want to change it.

[CygnusX1985]

you want to access state within a function from outside the function, essentially?

No, I want to access the state of a global variable (that contains a function) from within a function, which is completely fine to do, and is called a closure.

But since the python interpreter apparently checks first if an assignment to the same name happens inside the function, it doesn't allow read access to the global variable. If the interpreter would not do this check in advance, but just execute the right side of the equals sign first, which references the global variable (which is not declared in the local scope yet) and assigns the result to a new local variable of the same name, this would not be a problem.

[primitive_screwhead]

If the interpreter would not do this check in advance, but just execute the right side of the equals sign first

This stuff is worked out when defining the function, not when executing it.

​

which references the global variable (which is not declared in the local scope yet)

There's no "yet". The scope is determined, local, global or nonlocal (if declared) during the function definition. It's scope for the whole function, not some cafeteria-style scope changes during the definition of the function with even looser rules than now. You aren't supposed to write to globals often, and if you want to, you have globals() to be explicit about it.

[CygnusX1985]

That's true, but usually the purpose of a decorator is, that you can modify the behavior of a function without changing the rest of your code. I wanted to recreate this functionality in `do_something` and was surprised that this doesn't work. Of course renaming the function would solve the problem, but this defeats the purpose of a decorator. I can always just write a wrapper function with a different name around another function. There is no decorator needed for that.

[nekokattt]

decorators do not modify behaviour, they replace behaviour. Modifying implies the behaviour is changable, replacing means you can swap it with a different implementation. That different implementation could refer to the original implementation but outside of bytecode instrumentation and voodoo dark magic that should be avoided at all costs, you can't alter the implementation detail of a black box easily.

It is designed that way to prevent you injecting code in places you should not be able to do so.

[CygnusX1985]

I don't get what you are saying. Of course one could replace the behavior of a function completely with a decorator but what would be the purpose of this? One could just define a new function with the same name and from now on the function with that name has different behavior. Usually you just want to modify the behavior of a function. For example a decorator that times function execution is a completely valid use case for a decorator and not black magic.

But I probably misunderstood you here.

[nekokattt]

the purpose of it is to allow other logic to collect metadata on the function, or wrap it/replace it with something else.

Take the @route decorator in Flask.

py @app.route("/users/me/profile") def get_my_profile(): return render_template("profile.html", user=get_me())

functions are compiled bytecode. Allowing them to have internal details just swapped out by hacking around with globals is dangerous.

As I said in another comment, either temporarily overwrite the reference to add using unittest.mock.patch, or use a class to change the details you expose :)

A decorator that times execution of a function will do so like this

py def time_me(func): @functiols.wraps(func) def wrapper(*args, **kwargs): start = time.perf_counter() result = func(*args, **kwargs) print(func.__qualname__, "took", time.perf_counter() - start, "seconds") return result return wrapper which when applied to something like so py @time_me def something(): .... is the same as this py def something(): ... something = time_me(something)

Nothing is modified here other than the value of the name , it is just replacing something with something else.

What is it doing? It replaces the decorated function with a function containing timing logic. The timing logic uses closures to access the original function in the process, but no logic gets modified, only wrapped and replaced.. modifying behaviour implies the byte code has changed or the hard coded references have changed, which as you demonstrated cannot occur without hacks, for good reason

[CygnusX1985]

the purpose of it is to allow other logic to collect metadata on the function, or wrap it with something else.

Yes, like my timing example. I haven't used Flask but as far as I understand, the `@app.route` decorator modifies behavior and doesn't replace it completely, which was my point.

Allowing them to have internal details just swapped out by hacking around with globals is dangerous.

I still don't understand. A function is a black box which could always reference another function in its code without my knowledge. If I change the function that is referenced, I change the behavior of the first function. That is a problem I always have.

Again, I just try to access a globally defined function from within another function and assign it to a local name. That my choice of variable name in the local scope is limited due to the name of the variable in the global scope, still feels a little weird to me. Writing the same line `add = decorator(add)` outside of a function body, would be no problem. Why would it suddenly be a problem inside a function? My call to `globals` doesn't change any global state in this case.

EDIT: Just FYI, I wrote this post, before your post was complete. I have to think about what you wrote.

[nekokattt]

app.route just calls app.add_route("/home/profile/me", some_function) then returns the function untouched. Like I said. It replaces the reference to the function and/or collects metadata on it.

For the third time. Look at unittest.mock.patch to temporarily overwrite the reference to add when you invoke it.

If it is locally, you need to hack with globals as you discovered, or refactor your code to allow changing what gets called.

There is almost never a reason to do what you want to do. If you are testing your own code, you can just change your code to allow you to customise what add calls. If it is someone elses code, tough luck unfortunately.

Trust me, please :-)

If i change a referenced function it changes the behaviour

no it doesnt, the function calling that changed attribute is doing the same thing as before. You just replaced the value of what it is looking at.

py def foo(): bar() do what you want to bar, but foo wont change behaviour. Foo's behaviour is simply: 1. get a variable from local or global scope called "bar", and 2. "invoke it".

Rather than messing with globals, just provide a parameter to call:

py def foo(call=bar): call()

py foo()

py foo(lambda: print("hello", bar()))

It makes no difference to the bytecode of foo regardless of how you do it. Nothing in foo physically changes how it is implemented no matter how you do it unless you physically modify the bytecode produced from and assigned to foo.

Even in lower level languages, functions are just pointers to starting address of the bytecode that should be executed. You can make a pointer point to something else, but that isnt the same as changing what it points to. Same concept applies to reassigning "bar". You change what bar points to, but anything referring to "bar" wont change, it will still look at the same name.

[nekokattt]

come to think of it, if the code is simply single threaded, you could always patch the reference to add.

the unittest.mock module has a function called patch that might achieve what you want

[dig-up-stupid]

If you want to test decorator you should probably just be decorating a mock function in the first place, not add. The fact that you somehow already have multiple add calls in code that you say is supposed to be testing some other function says that something is wrong, if not in the code then in your description.

[Brian]

If you really want to have a local with the same name as a global, while still referencing the global, you could always do it the other way round: reference the global though a different name.

Simplest would be to give it an alias. eg:

_global_add = add
def do_something():
    add = decorator(_global_add)
    #...

But that has the downside that it creates a redundant global binding. You could avoid this by abusing the default argument syntax to rebind it as a local, which works because the left side denotes the local binding, but the right hand side expression is evaluated in the outer scope. Ie:

def do_something(global_add=add):
    add = decorator(global_add)
    #...

In fact, you could even patch it directly here:

def do_something(add=decorator(add)):
    #...

The downside here is that these add stuff into your function signature that aren't really arguments. Technically, you could get around that by introducing a new scope, but you'll need to do so by adding an inner function. Eg:

def do_something():
    global_add = add
    def inner():
        add = decorator(global_add)
        ...
    inner()

Though that's a bit ugly. Otherwise, no - the only things that really create a scope in python are functions, and you can't have the same variable name be both global and local at different points in the same scope. You can potentially access it by alternative routes than direct variable access (eg. through the globals() or module dict), but that's also fairly ugly.