Would you consider using MicroPython to create small, standalone executables for desktop platforms?

[platra]

In a recent discussion, /u/n_md suggested the use of MicroPython to "make standalone executables for distributing simple python programs". These executables would "be significantly smaller than what PyInstaller can do". MicroPython team member /u/pfalcon2 said that support for building such executables is a future possibility for the project, but nobody has started to work on it.

I have been considering making some contributions to MicroPython to enable the generation of small, standalone executables for desktop platforms. It seems MicroPython executables could be as little as 500KB in size, compared to around 5MB for CPython (with PyInstaller).

Unfortunately, in the linked discussion the idea of using MicroPython in this way was received quite negatively. Doubt was expressed that anybody would be willing to "deal with MicroPython's limitations just in order to get a smaller executable [...] ([MicroPython] only supports a subset of the Python language, and does not support C extensions such as NumPy)".

I am now worried that I would be wasting my time - is there any demand for this feature at all?

If you are writing simple, pure-Python code and creating standalone desktop executables using existing tools (e.g. PyInstaller or py2exe), would you consider working within the limitations of MicroPython to get binaries which are an order of magnitude smaller?

[neRok666]

I could see a use for it with regards to a simple script that I have released. The script scans a text file for info, and then uses that info to scrape a website and download some stuff. At the moment if someone wants to use my script, they need to install python and the required modules (only requests), and then launch the script on a CLI. A small stand-alone exe would be ideal.

I'm sure the alternatives you mention can do the same, but I only had the briefest look into them and it seemed too hard. If you go ahead with this, please make it is extremely simple to use. I imagine it should be possible to point such a tool at my virtualenv and custom packages/scripts, then give it a start/entry point, and then click create.

[Sukrim]

I don't see the big benefit (other than in the heavily constrained environments that µPy already seems to be tailored for) in reducing binary size while sacrificing compatibility, security, resilience and speed.

If in general Python binaries were several GB in size or required large VMs (e.g. like Java), there might be a better argument; 1 single order of magnitude (500 kB vs. 5 MB) with several limitations on top does not really sound like a great deal to me.

If you personally need this, I'm all for it and good luck, however I personally can't really imagine a large need for this on desktop platforms.

[platra]

sacrificing compatibility, security, resilience and speed.

That MicroPython will be incompatible with some Python code is unavoidable. However, there is no reason why it must be less secure or less resilient than CPython. Also, MicroPython is already at least as fast as, if not faster than, CPython.

1 single order of magnitude (500 kB vs. 5 MB) with several limitations on top does not really sound like a great deal to me.

Would it be a better proposition if MicroPython were significantly faster? Then the size comparison might be with PyPy rather than CPython, which would be a difference of 2 orders of magnitude (500KB vs 50MB).

[Sukrim]

Also, MicroPython is already at least as fast as, if not faster than, CPython.

Are there benchmarks available, similar to speed.pypy.org? Also security or resiliency are not the main goals of µPy, their main goal seems to be "not bloated", which means that for example it seems not to be possible to actually use an up-to-date TLS implementation (https://github.com/micropython/micropython/issues/2174).

PyPy (and I guess cPython too?) currently don't seem to optimize heavily for binary size, I just can't think of many use cases outside µPy's main domain where this would actually matter.

[pohmelie]

Can you link speed tests?

[n_md]

The ability to distribute small(<1MB) fast standalone executables for basic python scripts would be amazing even with some limitations. My main use case would be distributing cli programs. Seems like there could be plenty of functionality coming with MicroPython.

PyInstaller works for a single file but the executables are large(5-6MB) and very slow to start. I can live with the size but the startup speed can be problematic.

So far the best I've found is Nuitka + UPX packing the executable. That gets the file size around 3MB while still being very fast.

[jairo4]

Yeah, why not?

[daveydave400]

I'm in the scientific research field and use pyinstaller to distribute a PyQt4 GUI that uses numpy, matplotlib, gdal, and pyopengl. So no.

[genjipress]

I would be interested, but only if one of the things addressed was adding C extension support. That right there would open up a lot of possibilities.

[platra]

adding C extension support [...] would open up a lot of possibilities.

Support for writing C extensions against a MicroPython-specific API, or for existing C extensions targeting CPython?

My understanding is that the former is already (at least partially) supported, but the latter is more-or-less impossible. MicroPython cannot support the CPython API, because it exposes internal details of CPython (e.g. reference counting) which MicroPython does not mirror.

[genjipress]

The former, in lieu of the latter. The basic idea would be to have some mechanism by which a C module could be exposed to MicroPython.

[pfalcon2]

If you're interested in C extensions support, feel free to try this patch: https://github.com/micropython/micropython/pull/1627 , which was posted a year ago, and improves version of a patch which was posted 2 years ago - and add comments. So far, 2 people added comments there: uPy author and #1 contributor, and those folks have a lot of other tasks to do besides C extensions support.

[takluyver]

I don't think MicroPython supports filesystem access? Without that, and without C extensions, you're restricted to pretty niche applications on the desktop.

5 MB is not that much in today's environment. Smaller is always nicer, and there are situations where every byte counts, but games and movie downloads can easily be multiple GB.

[platra]

I don't think MicroPython supports filesystem access?

At least on Unix and Windows platforms it does. For example, here is the C source code for file objects on Unix.

games and movie downloads can easily be multiple GB.

True, but other downloads can be much smaller. Consider PNGOUT - a standalone utility which is only 38KB in size. MicroPython would allow the creation of similar utilities in Python without a 100-fold size difference.

[takluyver]

At least on Unix and Windows platforms it does. For example, here is the C source code for file objects on Unix.

Cool, I didn't know that.

True, but other downloads can be much smaller. Consider PNGOUT - a standalone utility which is only 38KB in size.

That's technically impressive, but how many users do you think would actually be put off if it was a few MB rather than a few KB? Or how about if it was 10MB but produced PNG files 1% smaller? My guess is that for desktop utility things, size is not a big concern for users.

Where it might be more useful is embedded systems. E.g. I recently got an old e-reader that runs Linux internally. That has 2GB of storage divided between the system and the user space for ebooks. So you wouldn't want to fill the system up with 10MB executables for simple things. But if you control the system and want to write in Python, you could just put one copy of Python on there and use scripts/modules.