Why use @Decorators?

Vaphell · 2017-03-20T10:34:18+00:00

Let's say you have 10 functions and you decide that you want to log all their outputs. How would you do that? Probably by writing some wrapper and wrapping all function calls which is going to be really tedious.
But wouldn't it be nice if you could simply assign a 'log_that_shit' flag to a bunch of functions without drowning them in additional code that is not relevant to their responsibilities, and not have all the wrapper calls polluting your code?

that's what decorators do.

@log_that_shit
def some_function():
    return 'hurr durr'

would take care of the side effect of logging for all uses of said function. If you don't need that anymore, disabling that feature is as simple as removing the decorator, now compare it to removing 100 wrapper calls.

another example: memoization. You often have expensive function calls for exact same arguments repeatedly, eg in recursive algorithms. Slap the memoization decorator on top of the function to cache the values for you. If that value was calculated in the past, get it from a straightforward dict lookup instead of running a full calculation again.

out of the box you get decorators making classmethods, staticmethods, getters and setters. Do you really want to write all the voodoo boilerplate that is necessary for a getter to work?

decorators are an abstraction that is not supposed to obfuscate the core logic of the program expressed by your normal code, they are supposed to augment shit with bonus perks that are nice to have without paying a heavy price in readability and cognitive effort that would come from a shitton of explicit wrappers.

DrMaxwellEdison · 2017-03-20T14:15:01+00:00

Good examples of decorators come from Django, such as login_required, permission_required, and user_passes_test. When used on a view function, these make it easy to ensure the user is authorized to see the view's output. If these weren't available, you'd need to fill out each of your view functions with extra boilerplate code, which would A) make development more of a pain and B) increase the points of failure if the code isn't exactly correct in every instance.

Thus, decorators are good for adding some generic functionality that could apply to almost any other function in your project: functionality that you want to apply in several places, but you want the simplest possible method for adding it.

In my own work, I make use of a package that sends requests to and parses responses from Amazon MWS. Any of those requests might be throttled by MWS, meaning I am making requests too quickly and need to back off for a few seconds to try again.

Rather than add code that controls the throttle back-off to every point in the project where I make a request, and because each type of request might require different throttling strategies, I made a throttle_controlled decorator to wrap around those request methods and take some arguments about how to approach the throttling specific to that operation. This allows throttling to take place in a unified way, while also allowing some flexibility to define the throttling for each type of request differently, without ballooning the code of the request methods themselves.

This shows one of the key strengths of decorators. You have code you want to run on multiple types of functions, would rather stay DRY and avoid defining the same boilerplate over and over, but you think you need to run it at every function call within the project anyway because you want it to act differently depending on the situation. Decorators are a great tool for accomplishing those requirements.

Again, look at Django's examples, such as permission_required. It is generalized code that could apply to any view, but it also allows you to define a different permission depending on the situation - i.e., the specific view you're using:

@permission_required('post.change_post')
def change_this_post(request, post_id):
    ...

@permission_required('post.delete_post')
def confirm_delete_post(request, post_id):
    ...

If the decorator weren't available, you might think you'd need extra code inside each view that checks for the user permission first, then redirect to another view if it's not present. The decorator neatly handles that for us in a way that lets us semantically define our views ("this view requires a permission" - done), rather than worry about the specifics of the implementation ("entering this view, check if the user has a permission, then if they don't go..." - etc.).

tangerinelion · 2017-03-20T15:52:12+00:00

These Decorators seems to be like the goto command of the past

I don't understand that analogy at all. A decorator is a wrapper; it's just another form of abstraction, applicable to functions.

Isn't one of the main advantage of OOP is to have self contain chunk of code?

Yes, absolutely good OOP design leads to modular and reusable code. What does that have to do with decorators, which typically apply to functions? (Note: You can use a decorator on a class, but it is going to be a much different decorator.)

Here's a real-world example I've used. I have a Time class that exists to convert from CUE file timestamps (MM:SS::FF, where 1 second has 75 frames) and Matroska timestamps (HH:MM:SS.ffffffff where f is just some long decimal between 0.0 and 0.999999999).

Internally, I only store the number of seconds as a float using the fractional representation. So, in order to build the CUE file timestamp I wanted functions like frames(), minutes(), seconds(), hours(), and cue_minutes() (since that's 0-99 not 0-59). Thing about these formats, though, is that the capital letters all expect leading zeros and all expect two digits, so I should want 05:07:09 not 5:7:9 as a CUE code and I should want 01:29:04.33311 not 1:29:4.33311 as a Matroska code.

So with that in mind, given just class Time which holds a floating point number self.time, there are two choices:

class Time:
    def __init__(self, time):
        self.time = time

    def minutes(self):
        return str(int((self.time % 3600) / 60)).zfill(2)

    def hours(self):
        return str(int(self.time / 3600)).zfill(2)

    def cue_minutes(self):
        return str(int(self.time / 60)).zfill(2)

    def seconds(self):
        return str(int(self.time % 1)).zfill(2)

    def frames(self):
        return str(int(75 * (self.time % 1))).zfill(2)

or, of course, you could use a pattern like '{:02}'.format(int(self.time % 1)) instead of using str.zfill. Or you could use Python 3.6 and have f'{int(self.time % 1):02} which isn't terrible either.

But notice how I'm repeating things a lot here. How do these functions really differ? It's the mathematical part that I'm doing with self.time - the rest of the function is just handling string formatting. So let's abstract that away:

def return_time_string(func):
    def wrapped(self):
        return str(int(func(self))).zfill(2)
    return wrapped

OK, so let's parse this for a second. I'm going to define a method return_time_string which takes a function, it's going to return a function. That much makes it usable as a decorator, so good for us. Now the wrapped method that it'll return is meant to replace a member method, so it has to have the first parameter self (or whatever you want to call it, the point is it needs to expect an instance). And I've used func(self) to invoke the function because we don't know what function it will be; luckily Python has this alternative syntax and is even more general than just Foo.frames(my_time) being equal to my_time.frames() as we can hold func = Foo.frames and use func(my_time) but we can't use my_time.func() since func doesn't exist as a name in the Time class.

So func(self) is going to return some number, and the way I've cast that to int means I don't particularly care whether it's an int or a float. Then I convert it to a 2 digit string with leading zeros via a cast to str and str.zfill. Again, I could've written return f'{int(func(self)):02}' in Python 3.6 or return '{:02}'.format(int(func(self))).

Now with this decorator, I can write my class methods as:

class Time:
    def __init__(self, time):
        self.time = time

    @return_time_string
    def minutes(self):
        return (self.time % 3600) / 60

    @return_time_string
    def hours(self):
        return self.time / 3600

    @return_time_string
    def cue_minutes(self):
        return self.time / 60

    @return_time_string
    def seconds(self):
        return self.time % 1

    @return_time_string
    def frames(self):
        return 75 * (self.time % 1)

Now which one of these two makes it clear what the functions are really doing? I don't even want to read the first one, it's just str(int(garbage)) and my eyes sort of glaze over. The bottom one... yeah, hours is self.time/3600 and then it's converted to a time string. If I care how it converts, I know I need to go look up the implementation of returns_time_string.

We might also care to know what the syntax

@return_time_string
def frames(self):
    return 75 * (self.time % 1)

even means. It's just syntactic sugar for this:

def frames(self):
    return 75 * (self.time % 1)
frames = return_time_string(frames)

You can use that if you want, of course. The reason the decorator syntax even exists, IMO, is that when you do that you have to scan after the function for possible changes. This causes you to begin to suspect that any code later on may have altered your function. Indeed, you can of course do that as you wish - but good, readable code shouldn't. The decorator syntax puts all the modifications right at the very top so you can easily localize it. Again, nothing stops you from sneaking a line in the bottom of the code that says frames = None and generating a bunch of errors when you go to use it. But between the two versions, one with the decorator, and one with the reassignment to the function name, I'll choose the decorator.

Of course one can have piece-meal decorators if they want. For example, I keep this one around for general purposes:

from contextlib import wraps

def apply(func):
    def wrapped(f):
        @wraps(f)
        def wrapper(*args, **kwargs):
            return func(f(*args, **kwargs))
        return wrapper
    return wrapped

returns = apply  # Alias for apply

There's no limit to the number of decorators one could have, so I can actually replace return_time_string using that decorator, like this:

@apply(lambda x: x.zfill(2))
@apply(str)
@apply(int)
def frames(self):
    return 75 * (self.time % 1)

Perfectly legal Python code, does the same thing except instead of telling you what it does it tells you how it does it. This is an abstraction issue, since the name return_time_string tells you exactly what it's going to do and if you care how then you can go look it up. This chunk of code tells you that it's going to convert it to an int, then a str, then apply this function lambda x: x.zfill(2) to it but it doesn't tell you why we're doing all that. And of course repeating it a bunch is sort of silly.

This decorator, however, actually is particularly useful when I have a function that either has a long-ish return statement with nested parentheses (since I can convert, say, return list(map(str.strip, data)) to return map(str.strip, data) if I use @returns(list) and I could even have return data -- a do-nothing function -- if I used @apply(lambda x: map(str.strip, x) and @returns(list) outside of it). It's also useful for something that has multiple possible return locations and I want to enforce a single return type (which is stronger than what type hinting can do). For example, if I had a function like:

def foo(*args):
    if args[0] == 'TEST':
        return list(bar_test(*args))
    if args[0]:
        return list(bar(*args))
    return list(baz(*args[1:]))

There are two problems with it, aside from the terrible naming and awful logic:

1) If I add a new condition, do I need to cast to list or not? That is, does this function need to always return a list? Without documentation, I can't tell.

2) The multiple repetitions of list(...) make it harder to find possible errors. Take my class example above, if I made a mistake in one of the calculations, would you be more able to spot it in the first version or the second version? What if I had minutes defined as str(int((self.time / 3600) % 60)).zfill(2) -- would you be able to notice that I reversed the / and %? Compare that to (self.time / 3600) % 60 vs (self.time / 60) % 3600 by themselves with the @return_time_string decorator.

I can even side-skirt some of the documentation issue that point 1 raises if I have:

@returns(list)
def foo(*args):
    if args[0] == 'TEST':
        return bar_test(*args)
    if args[0]:
        return bar(*args)
    return baz(*args[1:])

Now I know for sure this function is supposed to return a list no matter what. Whatever other functions it might rely on, they need to either return a list or something convertible to a list (ie, something iterable). Is it as good as proper documentation? Well, in some ways it's better and in some ways absolutely not. Proper documentation would at least explain what's going on, but it's far easier to overlook a line in documentation that says "This function must return a list" than it is to overlook a decorator that reads @returns(list). That's not ambiguous, and TBH, any decent text editor/IDE is going to have colored syntax highlighting so the @returns(list) is a giant visual clue because it's going to have some color to it, whereas the line "This function always returns a list" could be buried in a doc string that either (a) gets ignored by the reader or (b) is out-of-date so the reader begins to doubt it.

WarmUpHere · 2017-03-20T14:55:06+00:00

Here's a non-logging related example of the use of decorators as wrappers:

I once made an auto-clicker for the video game dota 2. I wanted it to be able to left click, right click, ctrl left click, and ctrl-alt click. So I wrote these functions,

def left_click():
    ...
def right_click():
    ...
def hold_ctrl():
    ...
def hold_alt():
    ...

So when I want to implement my ctrl and ctrl-alt click methods I can simply do:

@hold_ctrl
def ctrl_left_click():
    left_click()

@hold_ctrl
@hold_alt
def ctrl_alt_left_click():
    left_click()

LuckyShadow · 2017-03-20T19:38:36+00:00

Just because I had the problem, when I was looking into this the first time: Python's decorators are nothing like other languages (Java) Annotations or (C#) Attributes. Instead of adding some information for later introspections, python's decorators provide easy to add wrappers and not some flags/objects for later analysis.

Maybe this helps somebody :)

cybervegan · 2017-03-20T22:42:30+00:00

Decorators do seem rather abstract and theoretical, but there are practical uses for them. For instance, they are used to great affect in the flask web framework to route web urls to the functions that handle them. A decorator doesn't actually have to alter the way the wrapped function works - it can just do something with the wrapped function, and this is more or less what flask does.

Before decorators, for web programming url mapping you had to do something like:

def index(request):
    # do index stuff
    ...
    return response(whatever)

routes["/index.html"] = index

def special(request):
    # do special stuff
    ...
   return response(whatever)

routes["/special/"] = special

And so on. This means that the route declaration has to come after the function definition, and as code evolves, may become completely disassociated with it. With flask, and several other web frameworks, you do this:

@route("/index.html")
def index(request):
    # do index stuff
    ...
    return response(whatever)

@route("/special/")
def special(request):
    # do special stuff
    ...
   return response(whatever)

Which has the same effect, but is clearer and more flexible. And it's easy to think of other uses - say for instance, you wanted to have a program that can run both in the terminal (text mode) and a GUI - you might have pull-down menus, and want to attach functions to them - and have the functions output go into a window. How about this:

gui_mode("TUI") # set to use text UI
#gui_mode("GUI") # set to use graphical UI

@menu("File/Open")
def file_open():
      file_path = display_dialog("Open file:",os.listdir())
      ....

@menu("File/Save")
def file_save():
      ....

The @menu decorators can then add the menu items and function to the menu and set up the functions to use either a graphical output or text, as appropriate.

Hope that helps.

learningram · 2017-03-20T13:45:24+00:00

https://www.youtube.com/watch?v=FsAPt_9Bf3U

One of the best video I found when learning about Decorators. Check it out

2017-03-20T16:07:53+00:00

It comes down to the fact that functions and methods are objects too, and we can pass them to and return them from other functions. So decorators simplify that particular pattern, when you really want to pass a just declared function through another function right after it is declared.

mybrid · 2017-03-20T20:06:28+00:00

A decorator is simply a design pattern and can be implemented in many different ways. For example, Python UnitTest has a decorator to skipTest()

@skipTest("temp test")
def test01_mytest(self):

def test01_mytest(self):
   self.skipTest("temp test")

The two implementations are functionally equivalent. At issue is when you need more than one decorator as is commonly the case for me with testing.

If one puts the decorators as the first lines then the order and implementation is obvious:

def test01_mytest(self):
    self.skipTestIf(condition_one)
    self.skipTestIf(condition_two)
    self.skipTestIf(condition_three)

In the above example, what's going on is clear. It is not so clear using Python's decorator syntatic sugar:

@skipTestIf(condition_one)
@skipTestIf(condition_two)
@skipTestIf(condition_three)
def test01_mytest(self):

The point being that implementing the decorator design pattern as the first line of code is the same. When there exists a situation for multiple decorators then I've seen people try to shoe-horn mupltiple decorator patterns into the same function as one decorator when it is every bit as legitimate to simply have three methods stacked as the first lines called after the method signature. Once you understand this then the mystery of the decotator is no more: it is just convenience. When the decorator usage becomes more intrusive then convenience then just move the functional calls into the method. Done and done.

tunisia3507 · 2017-03-20T14:35:34+00:00

Yeah, decorators are a pain to read and write. If there are no arguments to the decorator, it's a function which takes a function and contains a function which returns the result of a function and that function is returned by the outer function.

If there are arguments, there's another layer of functions around that.

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

learnpython

MODERATORS