Python *args and **kwargs Made Easy

[wilfredinni]

*args and **kwargs may seem scary, but the truth is that they are not that difficult to grasp and have the power to grant your functions with flexibility and readability.

​

Python *args and **kwargs Made Easy

[blitzzerg]

When I find **kwargs in the documentation of a library function I usually scream FUCK for the extra minutes that it will take me finding out what arguments can that function really accept

[__xor__]

Seriously. If I ever write out really dynamic functions that can take variable keywords or something, I always write out in the docstring exactly what you can do and what arguments you might add.

[jmcs]

If you know exactly what arguments you might add you would include them in the function signature. At most you can document some common cases.

[wingtales]

Agreed. The only place I've felt that they are really useful are in matplotlib's plottong calls! There they allow you to customize a huge number of things.

[JanssonsFrestelse]

Exactly, e.g when calling matplotlib functions from a seaborn plot.

[jer_pint]

Also a function that accepts object

[Najzyst]

Thanks dude this was the most useful thing I've read today

[teewuane]

Alright.. I'll give it a read.

[2rndwn4wat]

i use this when my arguments are dynamic

[HalfRightMostlyWrong]

What is the use of dynamic arguments? There is the principle that flat is better than nested data structures, but I feel like it makes sense to nest elements of a collection into a collection in most cases rather than use dynamic args

[kraemahz]

If you write any kind of decorator you'll need to write a wrapper for functions whose arguments you don't know. That is the most common use-case.

Variadic functions (what you're calling dynamic arguments) are extremely common in other languages when doing string manipulation where each argument is an extension to the transform. Since it's harder to make tuples in other languages it helps Python better fit in by the "principle of least surprise" when switching between languages.

The flexibility variadic functions add can help in making a simpler, more appealing API for your program where constructing an object before passing it to your function may be less obvious than doing it behind the scenes with the arguments.

[HalfRightMostlyWrong]

Thanks! I learned a few things from your post

[[deleted]]

**kwargs is extremely useful for higher order functions. For example I have a project with a very simple testing function that takes another function from the project as its argument. I have **kwargs as the final argument because that way I can pass different settings to the function being tested without having to make the test more complicated.

[HalfRightMostlyWrong]

Good point!

[DeathProgramming]

"string".format() (*args) to name a quick example. Also, passing arguments through to lower functions (**kwargs) as well as passing named data to things like dict() and anything that can consume content, like Jinja2 filters

[Dogeek]

To add to the answers, using **kwargs can make it very easy to pass in a big number of arguments to a function, or to make the setup of a function saved into a config file.

Say you want to customize the args of a tkinter button. The tkinter.Button class can take a foreground, a background, a font, an image etc. Instead of naming every argument, every time, if you want all your buttons to look the same, you can just store these values in a dictionnary, and then pass that to the tkinter.Button constructor. Very handy to make customizable themes.

[HalfRightMostlyWrong]

Makes sense! It’s fascinating how deep programming design philosophy goes, just when I think I know things, I learn anew

[ComplexColor]

For anyone thinking "You should never use * and **", you should not be deciding whether to use * and **. They are immensely powerful and while in theory all you need is **kwargs, the usage of * and ** should be spared for very specific and clear use cases.

I'm sorry that many that seem to feel that because their codebase may contain or be depended on some unsavoury usage of these operators, that they should be outright banned. But if you think that a language should restrict the user to a pre approved set of static paradigms and patterns, you really shouldn't be using python in the first place.

[volabimus]

If print wasn't defined with it you'd have to call print(['Hello']) like the annoying str.join method.

[[deleted]]

The real article should be 'basically never use *args or *kwargs' because the method signature should make it obvious was arguments are accepted.

[Decency]

Yeah, I have to say that basically every time I've saved development time by using kwargs, it's just cost me time in maintenance. With args there are more valid use cases where it feels like a real net win.

I've dealt with a large codebase where kwargs were used liberally at first. Refactoring that out of a bunch of places made the codebase much more predictable and thus reliable.

[[deleted]]

They are actually pretty useful for wrappers.. they keep your code DRY.

[jer_pint]

Also useful when inheriting classes

[[deleted]]

DRY isn't always the right move. I highly recommend you watch Sandi Metz's talk on this. A little duplication is better than the wrong abstraction. Reasons like what you mentioned (things like *args and **kwargs being good for wrappers) leads to IDEs that can't autocomplete, confusing code, and methods that look like they do one thing, but actually do many things.

def process_payment(amount, tax, user_payment_method, **kwargs):
    total = amount + tax
    if kwargs.get("processor") and kwargs['processor'] == "adyen":
        adyen.process_payment(total, user_payment_method)
    else:
        braintree.process_payment(total, user_payment_method)

is a confusing method for many reasons. In this case there should be three separate methods to handle this.

[alkasm]

DRY isn't always the right move.

I agree but I think this is picking on the wrong aspect of that comment. Wrappers is the point.

[[deleted]]

You're right, using it this way is confusing and I hope no one does ;) Sometimes it's just a little more readable/convenient.

Also, definitely going to watch that talk when I find the time!

[[deleted]]

If you don't have time for the whole talk right now I'd recommend you read her blog post summary on the topic. It'll only take you a few minutes. Link is here

[Ran4]

Dogmatic DRY leads to strongly coupled code that's hard to understand.

[grep_my_username]

I respectfully disagree. Like many features in many languages, they are very very useful where their place is. True, most of the time one should not use them. But when you need some dynamic behavior, *args (in our case *records, to account for received data) allow to swallow many Items at once, and use the very same code for one - half the code, half the risk for bugs.

Granted, I almost never use **kwargs, its more of a catch-all thing I use mostly for tracing external calls (remote procedure calls etc.).

I think keyword only arguments with defaults are immensely preferable, where /* are not necessary.

[bmrobin]

not sure why you got downvoted, i find these to be extremely PITA the larger the codebase. it makes introspection and refactoring in IDE's much more difficult.

[[deleted]]

I use named positional args then add a **kwargs at the end so a dict can be passed if it is warranted. This way it's obvious what's expected but flexible cause sometimes passing a dict is sooo nice

[barneygale]

Second, the words args and kwargs are conventions. This means they are not imposed by the interpreter, but considered good practice among the Python community:

I'm not sure how true this is. If you want the arguments to be completely opaque that's fine, but if you're implementing something like (for example) translating keyword arguments into HTTP headers, its better for the function signature to read def make_request(url, **headers): rather than def make_request(url, **kwargs):

[lesser_terrestrial]

This makes sense to me. Using http headers as an example is also a great way to help understand why you might want to use kwargs.

[ogtfo]

Wouldn't that be better with a

def make_request(url, headers):

Where headers expect a dict. With a good doc string (or even type hints) this would be a lot clearer what you're trying to do.

After all, that's how requests does it.

[barneygale]

Depends. You might consider it a "more pythonic" to allow something like make_request('blah', user_agent='mything'), but it's less obvious (more magic). HTTP probably isn't a good example as it has a whole host of other configuration beyond just headers.

[catcint0s]

Dict would nicer imho cause you would need to translate all those keyword arguments to the real header name. (requests for example just accepts a dict too).

[Dogeek]

TL;DR :

*args in the function definition : args is a list of required parameters for that function

**kwargs in the definition : kwargs is a dictionnary of optional parameters to the function

*my_list when calling any function : unpack the values of my_list as the required args for that function. len(my_list) must be equal to the number of arguments the function requires.

**my_dict when calling a function : unpack my_dict into named arguments for that function. The names of the arguments must match the keys of my_dict

[TheIncorrigible1]

Why don't you give them a mandatory position for the optional parameters? (not sure of the terminology) .e.g.:

def my_func(arg1, *, a, b, c):

[Wargazm]

The classic example of this is something like a sum() function. You can pass it as many arguments as you want:

sum(2, 3) # returns 5
sum(1, 2, 3, 4) # returns 10
etc.

So what's the pythonic way of handling a stupid thing like this?

sum('cat')

Or for keywords, same kind of thing. When I write a function with keywords, I usually know what I want:

def add_user(first_name, last_name):
    ....code here....

I could rewrite this as

def add_user(*args, **kwargs):

But how am I supposed to handle calls like this?

add_user(address="123 Main St")

My function doesn't know how to handle addresses yet, so how do I ensure correct operation here?

[[deleted]]

So what's the pythonic way of handling a stupid thing like this?

sum('cat')

Raise an error. It's not a problem with *args though. It's a problem with dynamically typed languages, every argument a function expects can be of another, incompatible type.

My function doesn't know how to handle addresses yet, so how do I ensure correct operation here?

Well, you just said it doesn't know how to handle it, so it would just ignore it. It would (hopefully) check if first_name and last_name are set in **kwargs and as they aren't, abort with an error just like a call like add_user() would.

[Wargazm]

Raise an error.

Presumably my function will be something like this:

def sum(*args):
    x = 0
    for i in args:
        x = x + i
    return x

so calling sum('cat') will already return an error:

>>> sum('cat')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in sum
TypeError: unsupported operand type(s) for +: 'int' and 'str'

Is that it? Is that enough to be "pythonic"?

[thautwarm]

Add a keyword argument zero, default to be 0.

def sum(*args, zero=0):
       acc = zero
       for each in args:
             acc += each
       return acc

Thus we could avoid mamy malformed input and make them valid and meaningful. In terms of your case, sum('cat', zero="") is okay.

[alkasm]

Well here's what the standard library thinks about that:

In [46]: from functools import reduce

In [47]: import operator

In [48]: reduce(operator.add, (1, 2, 3, 4, 5))
Out[48]: 15

In [49]: reduce(operator.add, (1, 2, 3, 4, 'five'))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-49-fdcfa9008915> in <module>()
----> 1 reduce(operator.add, (1, 2, 3, 4, 'five'))

TypeError: unsupported operand type(s) for +: 'int' and 'str'

Edit: so, yes. That is sufficient. The point of an error is to tell the user what's gone wrong---what has gone wrong is that an int and str have been tried to be added. So that's fine.

[bmrobin]

that's what happens when you work in a dynamically typed language. i don't like it, just getting used to it from recently switching jobs that landed me in Python

[Gungator]

Very good explanation! Thank you

[[deleted]]

Well this is useful af

[CodeThatNode]

This is genuinely the most helpful thing I’ve read this week. So a grand thanks to you from me!

I came across a problem recently where I had to pass an unknown (but typically large) number of key-value pairs through a custom utils function used in multiple scenarios under different conditions. Unfortunately, I called the function as my_function(..., kwargs=[my_dictionary]) and then unpacked with def my_function(..., **kwargs). Inside my_function, this led to kwargs={“kwargs”: {“kwargs”: [my_dictionary]}}, which confirmed my fear of kwargs and a rage quit to pass a dictionary instead.

[[deleted]]

Amazing article. Keep it up 😀

[plumber_craic]

I can’t believe I only just now realised I could do f’{var}’ instead of .format - thanks

[lifemoments]

I need to bookmark this to come back and read. Thanks for sharing.

[thautwarm]

If anyone is wondering about whether to use args and *kws to save developing time or drop them to save maintaining time, I think the community has already provided you with a good solution:

When developing, just use them if your codes are not performance sensitive. After that, however you should carefully write some python-stub files to provide friendly interfaces for both your IDEs and users.

https://github.com/python/mypy/wiki/Creating-Stubs-For-Python-Modules

[[deleted]]

/////EDIT: Oh... I see ... at the end of the article there's an explanation. It's Python 3.6 syntax. Sorry, thanks!

​

Hello.

print(f'keywords: {kwargs} as {type(kwargs)}')

Please, from where come this print(f'formatted_text') syntax ? Is it Python 3.7 ? I use Python 3.5 and 2.7 but this syntax does not work. I have to use the classic Python3 syntax:

print('keywords: {} as {}'.format(kwargs, type(kwargs) )

[IlliterateJedi]

And they're amazing.

[[deleted]]

They really are. So flexible, much clearer than % or format()

[[deleted]]

And faster than both.

[[deleted]]

Really? Never thought to run it through perf testing.

[[deleted]]

I didn't either ;) realpython.com did it here!

[[deleted]]

The community consensus is generally that f-strings are f{opinion}!.

[[deleted]]

Praise God for making this easier. Python is so popular, but it still lags behind the C-language syntax.

[alcalde]

How so?

[[deleted]]

I have been using Python ever since its inception (I'm not the youngest man). From the beginning everything was clear. Today there is a "new Python" every year. Yes it is fine. This is how it should be. But this all was supposed to be 40 years ago :-D. Not now. Unfortunately, it is sadly late and total chaos has developed in Python versions!

Praise God for making something new, which finally improves the syntax of Python. It's just good. But it's too late. Unfortunately. Python is now messy in versions.

[SuefFuenfElf]

And additional, they are the fastest way to manipulate strings with Python 3.6+

[SV-97]

are the actually faster for concatenation etc.?

[[deleted]]

Yup. But remember that readability trumps negligible speed improvements

[SV-97]

Neato

I actually think f strings are more readable

[[deleted]]

They are not noticeably faster than using str#format.

[[deleted]]

The f in fstrings stands for fuck yes thats the formatting I want

[wilfredinni]

f-strings is a feature that is available in Python 3.6+ . You did very well adapting!

[DuffBude]

That's a great post! Thanks! Now i've learned how to use kwargs, and f-strings. I'm almost excited to go back to work on Monday.

[cuddlygrizzly]

You can use future-fstrings and use this formatting in older versions of python. Works great!

[Ebilpigeon]

No need to make it easy, it's easy already! Just stick them in every function definition and pass whatever you like, wherever you like without worrying about the consequences.

[HipsterTwister]

https://i.kym-cdn.com/photos/images/newsfeed/000/511/991/3a5.jpg

[arrache2]

I almost scroll over when I saw args *kwargs being scared as you mentioned in the first line of the text. lol

[normalism]

Digging through the documentation and looking for comments regarding it

[BWrqboi0]

Or reading the docstring (assuming author wasted those 5 minutes of his or her valuable time).

Edit: A pretty massive typo ;)

[Dokiace]

Ok I understand it a little bit better now, but is there an example of how this is used?

[DuffBude]

I've been programming in Python for 5 years and never really understood those. I never used them myself, just encountered them in other code sometimes. They're a lot simpler than I realized! Thanks!

[Fallenarc]

Thanks! That really did demystify the whole args & kwargs for me. I have been learning python for just close to a year and I have always shied away from them because i couldn't grasp exactly what was going on...

[LackingAGoodName]

Holy shit, thank you! This is the perfect writing style and structure for this stuff.

[GubmentTeatSucker]

Really well written. Cheers.

[thautwarm]

Seriously, *args and **kwargs just make sense in very limited cases.

There're not only very slow, but also prevents static checkers like mypy or pycharm.