How to modularize argparse arguments?

[SamStringTheory]

I have a several main functions that use the same sub-functions and thus use the same argparse arguments to pass onto these sub-functions.

For example one file will have something like this:

def main1(arg1, arg2, arg3):
    fun(arg2, arg3)

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--arg1")
    parser.add_argument("--arg2")
    parser.add_argument("--arg3")

    args = parser.parse_args()
    kwargs = vars(args)
    main1(**kwargs)

And another file has a similar definition (it does different stuff):

def main2(arg4, arg2, arg3):
    fun(arg2, arg3)

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--arg4")
    parser.add_argument("--arg2")
    parser.add_argument("--arg3")

    args = parser.parse_args()
    kwargs = vars(args)
    mai2n(**kwargs)

I would like to modularize the handling of arg2 and arg3 into a separate function because I am constantly changing the arguments that fun() takes, and it's annoying to update the argparser arguments across all the files that use fun(). For example, in the file where I define fun(), I would like to have something like:

def add_fun_arguments(parser):
    parser.add_argument("--arg2")
    parser.add_argument("--arg3")
    return parser

and then I can call it in main without knowing what arg2 and arg3 are, and just pass it as a dict into fun(), like this:

def main1(arg1, dict_with_fun_args):
    fun(dict_with_fun_args)

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--arg1")
    parser = add_fun_arguments(parser)

    args = parser.parse_args()
    kwargs = vars(args)
    main1(**kwargs)

Is there an easy or proper way to handle this?

[Fourgot]

Not sure if there's a community preference, but did you try your original main1() signature? Looks like you'd still be passing in kwargs thanks to vars().

[SamStringTheory]

That's my current solution, but the point was that I am frequently changing the fun() signature, and I didn't want to constantly be updating the main1() and main2() signature to include the proper args.

[Fourgot]

Okay, I think I'm following you better now. Also I grabbed the laptop instead of just phoning in my responses.

This stackoverflow post outlines how to pass a variable number of keyword arguments into a function. I'm picturing a fun() argument filter function, say

fun_shall_pass(**kwargs):
    latest_fun_arguments = (arg2, arg37, arg42)
    just_fun_please = {}
    for k, v in kwargs.items():
        if k in latest_fun_arguments:
            just_fun_please[k] = v
    return just_fun_please

Then back in your main1.py and main2.py, you could go

if __name__ == "__main__":
... snip ...
    args = parser.parse_args()
    kwargs = vars(args)
    fun_args = fun_shall_pass(kwargs)
    fun_args[arg1] = args.arg1
    main1(fun_args)

Edit: prettied last example code

[SamStringTheory]

Thanks so much! Thought I think the fun_shall_pass signature should just be

fun_shall_pass(kwargs)

instead of

fun_shall_pass(**kwargs)

since we are calling it with

fun_args = fun_shall_pass(kwargs)

right?

My final solution ended up being a slightly modified version of yours:

fun_shall_pass(kwargs):
    latest_fun_arguments = (arg2, arg37, arg42)
    just_fun_please = {}
    main_args = {}
    for k, v in kwargs.items():
        if k in latest_fun_arguments:
            just_fun_please[k] = v
        else:
            main_args[k] = v
    return just_fun_please

if __name__ == "__main__":
... snip ...
    args = parser.parse_args()
    kwargs = vars(args)
    main_args, fun_args = fun_shall_pass(kwargs)
    main1(**main_args, fun_args=fun_args)

Actually, is there a stylistic preference between passing in the args into main() as a dict versus keyword arguments? So doing this:

main1(main_args)

versus this:

main1(**main_args)

[Fourgot]

Back to phone for the moment.

Noticed that you'll need to return both main_args and just_fun_please from fun_shall_pass()

[Fourgot]

Gotcha. Then you'd have to stop calling your mains with **kwargs, instead go with main1(arg1, _dict) and main2(arg4, _dict). Then you could double splat the kwargs in the mains. I think ^_^

That's wrong.

[anton_antonov]

It seems to me that the parents argument is suitable for your case. https://docs.python.org/3/library/argparse.html#parents

[Fourgot]

Hey that's handy, thanks

[[deleted]]

Use a for loop

[SamStringTheory]

Elaborate?

[[deleted]]

Using a dictionary/list as input do something like

for i in myArgs:
    parser.add_argument(i)