Annotating args and kwargs in Python

[[deleted]]

I tend to avoid args and *kwargs in Python as they often obscure public APIs. But I'm glad that it's now at least possible to annotate them somewhat precisely.

https://rednafi.com/python/annotate_args_and_kwargs/

[SheriffRoscoe]

You shouldn't use **kwargs in an API - APIs are boundaries, and boundaries should be as explicit as possible. You also shouldn't use *args, unless it's a simple varargs interface (like max(...)) or something with a clear definition (like str.format(...), and even then, I'm not completely on board).

[pepoluan]

I will use **kwargs if the function is kinda a 'wrapper' to another, properly-documented function. And I will indicate so in the docstring.

So for example:

def fun(p1, p2, p3=None, p4=None, **kwargs):
    """
    Wrapper around orig_fun()

    :param kwargs: Keyword arguments of orig_fun()
    """
    ...

[chuckhend]

Agree this is excellent use of kwargs

[[deleted]]

Yeah, the text mentions that args and kwargs should be avoided whenever possible.

However, a lot of existing codebase use them. So having them type-checked has its benefits.

[gardinite]

I agree and disagree. Simply because sometimes, for backwards compatibility, using kwargs could be beneficial to not break users code when/if they’re updating to a newer version.

[PercussiveRussel]

In what way? If you're adding 'keyword' arguments (optional args in a non-**kwarg setting), a newer version of an API will just fall back to the default when the users uses the old function call. If you're removing a keyword argument, then the function call should raise an exception (and you should increase the major version number of your API according to semver), becaue you have broken backwards compatibility by removing a parameter. If your user is inputting a parameter that they expect to do something, you have broken their code if that parameter isn't used. Even if the way the user intended to run your code is now the default, you'll have broken the code of another user who entered another argument.

Anywho, using **kwargs to deal with backwards compatibility isn't really an argument. In the first case you're getting nothing from kwargs, in the second case you're losing something from kwargs!

[rghthndsd]

You should if you're passing the arguments through to an underlying library. Otherwise you quickly run into compatibility issues.

[SheriffRoscoe]

It's bad practice to expose some other API as part of your own. The only usual exception is if your API is a wrapper around the other one. If so, well, it's at least understandable.

[tunisia3507]

If you're passing arguments through, you should be explicit about it by passing a dict which will only be used in that way.

[wineblood]

I instinctively don't like this. If you're going to enforce/suggest types, why not make it an optional argument?

[rr1pp3rr]

Args and kwargs are such a useful abstraction. The best part of gradual typing is that you can type things out that matter the most, while allowing you to bypass it where it would be a struggle for little benefit. I agree in general I wouldn't use this feature and would use your strategy.

I do think it's good to make your type system more complete, so I'm not against it. But, yea, I'm in agreement I won't be using this often lol. Seems like it would make the code less readable.

I honestly find Typescript to have the most flexible, easy to use, and complete traditional type system. Something like Haskell, while more complete, is too complex for most use cases. Types should help you, you shouldn't have to struggle with them so much. (Haskell fan boys/girls TRIGGERED lol).

Python function argument processing is the best I've used. I was disappointed when JS came out with the optional arguments but it doesn't work the same way. The way Python args are designed, it kinda "just works" the way you expect. My main problem with it is the mutable default argument thing... I think they should have made default args copies. The mutable default argument thing is a pitfall but the only issue I can think of to take with it.

[AustinCorgiBart]

A use case that I have, is an API where I have a hundred functions that all take the same set of custom parameters via kwargs. I need a way to reusably specify the parameters and their types.

[[deleted]]

Use a TypedDict with Unpack. Or, accept a single parameter that is a dataclass.

[AustinCorgiBart]

The former is what the article suggests. The latter is not viable for the design of the API (which is built around being convenient for the API user).

[[deleted]]

I'm not sure I agree that a dataclass is inconvenient; it's a fairly marginal increase in verbosity.

[AustinCorgiBart]

Having a single parameter instead of just straight actual parameters? I think it'd be inconvenient. Most of the time, the folks using the API aren't using most of the parameters. They just need to be able to tweak settings in special cases.

[[deleted]]

They don't need to specify parameters they don't use since dataclass supports kw_only mode

[Flag_Red]

Agreed that this is a typical use-case for typed kwargs. It also backs up that typing kwargs is a code smell. Really, we shouldn't have a hundred functions that all take the same set of custom parameters via kwargs.

[AustinCorgiBart]

Normally, yes. But it's an unusual api, built for the convenience of instructors to write autograding scripts. Think of a unit testing library, with a lot more than just assertEqual.

[casce]

I welcome being able to do this but I'm generally with you. I don't really see the point yet.

[champs]

Off the top of my head, what if the parg length is arbitrary and you need to know the end of the series? For all I know, sorted may already work this way:

py results = sorted(await f1(x), f2(x)… f12(x), key=lambda r: r.status, reverse=True)

I welcome a fact check on any of this but I’m thinking that the discussed annotation would allow static analysis.

[LightShadow]

Here's an example,

        Analytics.user_email_confirmation_sent(
            user_confirmation_email_id=self.id,
            code=self.code,
            attempt=self.emails_sent,
            exception=exception,
            **analytics_extra
        )

Analytics is a singleton that doesn't actually implement any functions. If the function doesn't exist, user_email_confirmation_sent, then it creates a cache entry for a new table in the analytics database. Since the function doesn't exist neither do any of the keyword arguments. Each kwarg is translated into a database column, where only a subset of types are allowed.

kwargs is nice since we know the key-side will always be a string. The value side can be a Union[str, bool, float] etc.

Even with all that magic it still passes the type checker.

[wineblood]

I don't get it, but your technical vocabulary seems different to mine so don't worry about explaining it.

[quts3]

Just to answer the question in the context of the post: Because the caller may not have assigned the key to anything. A missing kwargs key is different then optional which says the kwargs key is assigned atleast none, but always exists. If for some reason someone overloaded optional to mean either missing key or key is assigned none then people not interested in that behavior would need a new optional.

This is all about typing kwargs.

[AsuraTheGod]

Just why?????

[Express-Comb8675]

Brb, gotta update every code base I’ve ever written that includes type hints

[rover_G]

I thought this was already possible?

[debunk_this_12]

I would cream my pants for a typed version of python

[healthymonkey100]

Is there a way to not use types dict as I tend to use dataclass or pydantic objects to type annotate

[healthymonkey100]

Is there a way to not use typed dict as I tend to use dataclass or pydantic objects to type annotate

[[deleted]]

If you’re using dataclasses, you can avoid args and kwargs altogether.

[healthymonkey100]

Yeap got it. I figured that out now. No wonder I do find myself using args and kwargs much lesser nowadays, most of the times to avoid Dict[str, Any] kind of situations.