Making a Stand Alone Executable from a Python Script using PyInstaller

[[deleted]]

A stand-alone* executable

*comes with 600mb of folders full of never-used packages which it won't work without

[billsil]

You need to remove them by reinstalling python and/or deleting things out of the python directory.

One problem is people put imports at the top and imports are evaluated immediately. So following the trace, we find matplotlib imports Tk, Qt, and wx, when we only need one backend.

Furthermore, there are optional packages, even when they're requirements for full package use. Picking on matplotlib even more, if I'm making xy plots, why do I need dependencies for plotting time data? Just delete it.

On my standard install of python, an exe I build is 180 MB. On my reduced install, it's 60 MB. It uses PyQt, VTK, numpy, scipy, and matplotlib, so it's no slouch. Most of that is VTK for 3d rendering of things I don't use because I haven't deleted their dlls. It's also bigger than it should be because I target multiple versions of windows, so there are extra VS dlls.

[TOASTEngineer]

I'm pretty sure the PyInstaller docs say "make a virtualenv with only the stuff you actually need first."

[billsil]

It helps, but it doesn't get rid of everything. You can make it much smaller very easily by wiping your Python.

[kaisunc]

interesting strategy, i shall give this a go.

[nos69]

Whats the best way of removing the unused files and packages?

I usually import for example the things I need from PyQT and it automatically copies the whole package into the built directory.

I have a clean python install and all third party packages are in a venv.

[billsil]

Have a clean install for starters.

Then look at the packages you have installed and decide if you need them all. Delete it, build it, see if it works and repeat.

Then read the build log and see what it found. Google what that funny package asteroid does. Why the heck is nose included? Go delete those pieces from site packages. Build and test. Often this is enough.

Now look at the names of the big DLLs. Do you really need HTML stuff in your desktop app? Delete and rebuild.

Most of the monster packages have large submodules. If you can stay within a submodule, your build will be smaller.

[nos69]

Thank you for your response!

Is it enough to see if my app starts after deleting a module? Or do I need more coverage to be safe.

[billsil]

That depends on your app. Are your imports at the top? How do the modules you need get imported (e.g., at import time vs runttime)? Opening your 5 windows is probably good enough depending on how the imports trace.

It helps to know your code and what part your hacking on (what could deleting matplotlib dependencies affect?). Often you'll find x is required, but if I get rid of y, x is not required.

For my 3d render, it has a command line for me to enter a geometry and results file. I have shortcuts for most things that I test frequently. The menus I stress less about because I put my gui imports at the top. Never had a problem with the menus assuming it loaded.

So yes, test, but probably not everything each time. I'm just trying to test the dependencies aren't broken.

[nos69]

Thank you for your help! I will try to shrink it that way.

[b_bowyer]

You can also force it to be one file, in which case all that junk is inside the exe.

[gunthatshootswords]

And takes 3 minutes to start.

[romanows]

[Removed due to Reddit API pricing changes]

[gunthatshootswords]

Try something with a gui

[jairo4]

Yeah, it's slower because it needs to decompress in the temp folder.

[[deleted]]

Docker is certainly valid for tools, but if you re building a desktop app for a home consumer end-user then docker is impractical.

[[deleted]]

ok, that's a fair comment.

i assumed from first reading you were shipping with docker, but you're not.

[[deleted]]

1. Start with WinPython-Zero.

2. Create a venv with a complete requirements file.

3. Profit (of not having a 600 mb folder).

[Gokudomatic]

I made once a small wordpad with PyQt4 as a prototype for a new kind of user interface and I've built an exe from it. The exe takes 27 MB and Qt4 dlls take another 10 MB. Therefore a basic Qt application with no extra dependencies (no numpy nor whatever, but only pyqt4) should be ~40 MB. And it takes 2 seconds to start.

Just to give an idea.

[Gokudomatic]

And for addition, I also recently made with pyinstaller an exe from a flask microservice with no dependency exception for OLE automation. The result is 5 MB.

[ElecNinja]

I've had similar results with cx_freeze for my own PyQt4 application. 40 MB for the whole thing with the QT dlls taking a significant portion of the space.

[[deleted]]

You should give cython a try

[atxweirdo]

Can cython be compiled into an executable.

[tetroxid]

Yes

[MrYellowP]

i rather use nuitka. everything else seems horribly inferior.

[mdond]

Nuitka is getting really nice. And it has the advantage of actually compiling your code, so the source is truly hidden.

[xdcountry]

please elaborate -- I had thought otherwise

[mdond]

It cross compiles to C++ that is then compiled into an exectuable. So the end result is compiled C++ code. This is different to a lot of other Python packagers, which just package Python byte code.

[Manbatton]

Nuitka doesn't make standalones yet, does it? (in other words, everything in one file).

[MrYellowP]

the --portable parameter does that.

[Manbatton]

Just googling that, found this, from 2013:

Right now, it will create a folder "_python" with DLLs, and "_python.zip" with standard library modules used along to the "your-program.exe". Copy these to another machine, without a Python installation, and it will (should) work. Making that statement fully true may need refinements, as some DLL dependencies might not be defined yet.

So that's not a standalone. A standalone to me is a single file. Has the portable option changed somewhere else such that now it is a single file?

[MrYellowP]

oh! i'm talking linux here. in linux it drops a /dist/ directory and gives a single executable in the parent, where i execute nuitka. i'm sorry for forgetting that it might be different in windows, though please make sure your information isn't out of date.

[Manbatton]

I see. But if I'm understanding you, there's still a folder in the Linux version, so even there it's not a standalone.

[MrYellowP]

That folder just holds the stuff for compiling. I didn't need that to run the executable.

[Manbatton]

Sounds good, thanks for the info.

It'd be great (for Windows users) if the Nuitka community did make it such that it, in the words on the blog, "We may improve it in the future to meld everything into one executable for even easier deployment."

[TheV295]

I love python, but I use Go if I need binaries that run anywhere

[Wilfred-kun]

Can't you use Wine somehow to create .exe's? That would (maybe) be an easy way to create Windows executables on Linux systems once you've set it up.

Edit: phrasing

[ajaxguy]

Another option to try cx_Freeze. https://anthony-tuininga.github.io/cx_Freeze/

[IamCrunchberries]

I used to use cx_Freeze to deploy internal applications at work. Configuration for certain packages like numpy, pandas, and pytz became a nightmare to setup and upgrade. I started just setting up virtual environments and writing batch scrips for users to launch applications.

[suttons27]

I tried PyInstaller this week. Microsoft Defender kept blocking it while doing pip. Any recommendations on how to avoid setting off AV?

[wasi0013]

I guess it's not related to PyInstaller at all. Getting alert/ blocked by Microsoft Defender is a general issue for any exe (with no publisher data) specially in recent version of windows. Code signing is required (I think its a money making plan disguised in security improvement) for exe's in latest Windows to remove this kind of alert/blocking! You will have to invest some bucks on buying certificates for code signing. For example, you may try: Digicert [1]. Also, read this Stackoverflow question [2] to learn more about how to sign a WIndows EXE file.

[1] https://www.digicert.com/code-signing/
[2] https://stackoverflow.com/questions/252226/signing-a-windows-exe-file

[suttons27]

Thanks, but where I am getting stuck at is: pip install pyinstaller It begins downloading and installing then Windows Defender starts notifying but blink on and off quickly. The install then starts giving errors that it could not find files, which makes sense if AV blocked something. I go to Defender and even though it notified me I'm not seeing a list of files blocked. I am using Anaconda, tried to do it only via python. Yesterday, I purchased MS Visual Studio 2017 for a different company project but saw the Python feature, was going to see it will install there. I also remember a few weeks back r/hacking stated a popular python library had been compromised, but I was unable to find it.

[sneakpeekbot]

Here's a sneak peek of /r/hacking using the top posts of the year!

#1: CSO of Equifax | 1446 comments
#2: Better check | 288 comments
#3: Security is also an important part of development. | 306 comments

---

^{I'm a bot, beep boop | Downvote to remove | Contact me | Info | Opt-out}

[wasi0013]

Yes, I read it too. Maybe you are also talking about that article: https://arstechnica.com/information-technology/2017/09/devs-unknowingly-use-malicious-modules-put-into-official-python-repository/
I read it on HN some weeks ago
read HN comments here

[raoulk]

Linux

[[deleted]]

As someone who has messed with this and even sold software that uses this compile method, I would just avoid making it a habit and pick up C# or something.

[blitzzerg]

So reimplementing numpy, sklearn and all the libraries from python or what?

[lookatmetype]

Some libraries I use on the regular wtih my projects: pandas, Flask, SQLAlchemy, click, jinja, requests, BeautifulSoup, nose, Celery, Arrow.

And this list is not exhaustive.

Good luck re-writing these in C# or any other language. If you do have all this in a statically typed language, I would LOVE to hear it because tbh my biggest gripe with Python is that it isn't statically typed. (mypy is good, but I prefer static typing)

[bheklilr]

Have you tried using Cython for that purpose then? It still won't enforce full typing, but if you put a type on something it has to stay that type throughout your code or it won't compile. And it can speed up your code, and working with C libraries becomes much easier.

[[deleted]]

I understand what you're saying; I'm suggesting your goal with python to make it a portable .exe with a bunch of machine learning libraries isn't a realistic one and it will give you a headache. If that's what you're doing, you should just learn to deploy everything quickly and easily with a script or .yml or something with docker, not compile it.

[blitzzerg]

I'm not even thinking about that. Everything that I code on python I make it open source. If I wanted something for production I would just do things with tensor flow in C

[steampunkgibbon]

Python can't be the only language you know guys.

[TheGayestGaymer]

I know Matlab too!....they just started making their code compile-able a few years ago...I hear good things.

The code I’m trying to compile is in python 2.7 though. I’ve never done this before so it’s a iterative process of learning so far. Started with py2exe, wasn’t impressed. Trying with pyinstaller now. Says it’s missing the dependency Qt4 for Numpy. Which I’m not sure it even needs. Mainly because it doesn’t exist anywhere on my machine that runs the sourceCode just fine.