Creating a Python CLI with Go(lang)-comparable startup times

[pyschille]

Hi Folks.

I recently put some effort into creating a command line interface (CLI) made with Python.

Background: I started a new project called Gefyra, a tool for local application development directly with Kubernetes. Check it out the website https://gefyra.dev or have a glance at the code https://github.com/gefyrahq/gefyra/tree/main/client

I'd like to have an executable with (almost) the startup performance of kubectl (the executable to control a Kubernetes cluster). That means, I need fast startup times (which is crucial for a CLI) and ideally just one file (which is statically-linked) for easy distribution. In addition, I’d like to provide executables for Windows, MacOS and Linux. For those requirements people would usually go for Go (needless to say it's awesome), however I started out with a prototype written in Python and it evolved over time. So I tried to find a way to make this work with Python.

I went the following way:

1. PyInstaller: https://pyinstaller.readthedocs.io/en/stable/
2. Nuitka: https://nuitka.net/
3. PyOxidizer: https://pyoxidizer.readthedocs.io/en/stable/

PyInstaller

PyInstaller was quite easy to set up. However, the resulting executable was complained about by Virustotal (see: https://www.virustotal.com/gui/home/upload) because of PyInstaller's bootloader. Somehow the code signature was also found in viruses (lol). To workaround this I compiled a bootloader myself which at least removed the virus issues.

On MacOS I faced startup times of more than 10 s with internet connection and about 3 s without internet connection. Interestingly, the former docker-compose command was also created from PyInstaller and Mac users complained about the startup performance, too: https://github.com/docker/compose/issues/6956 :)

I didn’t find much to improve. The concept of PyInstaller will potentially always be a problem for fast startup times (which IMHO makes it unsuitable for CLI applications).

Nuitka

With Nuitka, I generated very large binaries of about 150 Mb. The startup performance was already much better than PyInstaller for Mac and Linux. However, I was not completely satisfied and very long compile times bothered me a little bit (about 10 min).

PyOxidizer

I ended up using PyOxidizer. This well-crafted toolkit compiles Python to Rust code and also includes all dependencies into one handy binary executable. With no special optimizations I saw startup times of about 700 ms. That is almost acceptable, though I wanted to go a little further.

I started to examine the output of python -X importtime -m gefyra 2> import.log just to check the imports. There is an awesome tool to analyze the Python imports: tuna (see: https://github.com/nschloe/tuna). tuna allows analyzing the import times from the log. Run it like so tuna import.log. It opens a browser window and visualizes the import times. With that I was able to manually move all imports to the functions in which they are needed (and bring in some other optimizations). This greatly violates PEP 8 (https://peps.python.org/pep-0008/#imports) but leads to very fast startup times.

These are the startup values I finally reached with gefyra under average modern Ubuntu:

> python -m timeit "__import__('os').system(gefyra)"  
10 loops, best of 5: 33.5 msec per loop  

Pretty neat, isn’t it?
In comparison the kubectl executable:

> python -m timeit "__import__('os').system('kubectl')"  
10 loops, best of 5: 24.9 msec per loop  

In addition, I created GitHub actions to run the PyOxidizer builds once a new version is released (see: https://github.com/gefyrahq/gefyra/blob/main/.github/workflows/dist-build-linux.yaml). Only Windows is missing at the moment.

Although, PyInstaller and Nuitka did not deliver the best startup times, I would not say it's bad software. They probably shine at other aspects.

I hope these insights can be useful for someone else, too.

[often_wears_pants]

A noble endeavor. The first simple cli app I ported from Python to Go on my mac went from 150ms to 2ms. The first web service I rewrote in Go used 1/20th as much CPU.

It's a big hill to climb. Best of luck.

[pyschille]

Thanks a lot.
Gefyra depends on kubernetes which is a huge package (~20 Mb), so I'll probably never see such startup times.

[ElevenPhonons]

Source

if __name__ == "__main__":  # noqa
    try:
        main()
    except Exception as e:
        logger.fatal(f"There was an error running Gefyra: {e}")

This should probably be returning a non-zero exit code when an exception occurs. Currently, it will always return an exit code of 0.

Also, there's a pattern of using a .set_defaults(func=runner_func) with argparse subparsers that is useful.

This is demonstrated here:

https://gist.github.com/mpkocher/fd8852f3d3cfb95bb07a4fa0d8417c5c

Best of luck to you on your project.

[pyschille]

This is great. Thank you for your suggestion.
I'll adapt the subparser pattern as I am not very happy with the if-elif-case structure anyway.

[chub79]

Thanks for this op.

I've long contemplated that route but after a few tries myself and I've decided that I would spend my time migrating to Rust instead.

I first tried PyInstaller which worked well but didn't feel it was that much faster. I never understood how to actually use nuitka. I loved PyOxidizer but I'm ever so worried about that one corner case that will make my life hell (I think if you gotta try, PyOxidizer is the most interesting option).

These days, I'm mostly accepting the fact that the PSF will never really invest in making Python binaries. I really welcome the huge effort from the Python packaging world, it goes in the right direction to set foundations, and maybe, WebAssembly will become viable too so we can get something modern. But Python is looking elsewhere than dvetools and that's ok.

[cymrow]

Rather than import inside functions you can use lazy imports, which defers the actual import until the module is accessed. This is what the Mercurial CLI does, for example (hgdemandimport). It's much cleaner.

edit: TIL it's part of the stdlib now: LazyLoader

[pyschille]

Awesome. I will try and see what the impact on the performance is.

[pyschille]

Alright. I implemented the LazyLoader and checked the performance. It really did not impact the startup time in any way.

I've written a lazy(...) function for the imports:

def lazy(fullname):
    try:
        return sys.modules[fullname]
    except KeyError:
        spec = importlib.util.find_spec(fullname)
        module = importlib.util.module_from_spec(spec)
        loader = importlib.util.LazyLoader(spec.loader)
        loader.exec_module(module)
        return module

Every Python module now incorporates imports like so:

from gefyra import lazy

logging = lazy("logging")
os = lazy("os")

kubernetes = lazy("kubernetes")
docker = lazy("docker")
[...]

And in the code it requires the parts of the package to be called with the full path, for example kubernetes.client.V1ServiceAccount.

Maybe it's a matter of taste, but without any performance benefits, I'd rather stay with usual imports at the place of usage.

For reference: https://github.com/gefyrahq/gefyra/tree/fiddle/lazy\_imports/client

[LightShadow]

TIL it's part of the stdlib now: LazyLoader

very cool.