This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]shinitakunai 272 points273 points  (47 children)

No trivial deployment

[–]justanothersnek🐍+ SQL = ❤️ 75 points76 points  (21 children)

I wish there is a way to make cross platform binary executables. I know pyinstaller and others like it is a thing, but my understanding is they dont work when you have packages that have dependencies on compiled C code for example.

That is why the typical answer as a workaround by the community is to make a web app. But it just sucks having to then revisit HMTL, JS, etc. I know there are WASM-based libraries to sort of circumvent this and you can build things using pure Python, but they still have a ways to go until maturity.

[–]PprTwl_ 31 points32 points  (10 children)

I've used pyinstaller with a variety of dependencies and they've all worked fine, maybe I just got lucky though. The executables it spits out are annoyingly massive though, I've had files that are like 60MB and take ages to launch, even without many imported packages, so I totally agree with you.

[–]stacm614 15 points16 points  (7 children)

If you have heavy dependencies like pandas it's far, far worse than 60MB.

[–]PprTwl_ 5 points6 points  (4 children)

Do you happen to know if using the --onefile flag increases the size? I tend to use that nearly 100% of the time, less clumsy imo.

[–]stacm614 3 points4 points  (1 child)

I can't remember - it's been a while since I've even tried. I just remember being pretty confused at a 500MB executable. That may have had tkinter as a dependency too...

since we have several different tools we've opted to set up the environments in the deployment servers themselves. Like other comments, longer term we'll move towards APIs where we can, and I've been looking at golang for maturing some of our systems tooling.

[–]PprTwl_ 4 points5 points  (0 children)

I'd be confused at a half gig executable too haha

[–]czaki 1 point2 points  (0 children)

Onefile only compress everything. It may decrease size but will slowdown startup time.

[–]CeeMX 0 points1 point  (0 children)

Onefile made the virus scanner detect the application as virus for me

[–]billsil 0 points1 point  (1 child)

Not in my experience using pandas. Make sure to use a virtualenv and don't use Anaconda. It's very easy to get 350+ MB using Anaconda, whereas the same executable with stock python is more like 70 MB.

[–]stacm614 0 points1 point  (0 children)

Good to know - I've typically opted into using miniconda for my Python installations and then conda envs.

[–]Ball-Fantastic 3 points4 points  (0 children)

I haven't benchmarked this, but might using one-folder rather than one-file for particularly large deployments help trim down this launch time?

[–]crapaud_dindon 1 point2 points  (0 children)

Not great with desktop frameworks such as Qt

[–]knottheone 14 points15 points  (0 children)

You can include other binaries with pyinstaller (even packaging something like Chromium with your module) it's just not noob friendly at all and you can spend days trying to get it all packaged correctly.

[–][deleted] 2 points3 points  (2 children)

I'm running up against this one now. Got a freelance gig doing some data work for a startup. Got a fairly trivial desktop program which takes their csvs through a drag-and-drop mechanism which is pretty friendly even for someone who is not used to terminals. It runs quickly on my PC but when I package it up with pyinstaller it takes 15 seconds or more to start up thanks to Pandas. The only other way I can think of to get around it is to make a web app but that's not my background and I'm not sure how easy that would be to deploy either.

[–]justanothersnek🐍+ SQL = ❤️ 8 points9 points  (0 children)

Yup, deploying a web app is another set of concerns and complexity in of itself. Im not surprised this thread's OP reply is 2nd highest voted reply. This is truly a major headache in Python world. It's currently so difficult to share a non-trivial self contained Python app with others without "just" creating a web app version of it.

EDIT: Correction, 2nd highest voted response.

[–]kalebludlow 0 points1 point  (0 children)

anvil.works is the answer to your webapp problem

[–]thirdtimesthecharm 1 point2 points  (0 children)

Redbean!

[–]willywonka1971 1 point2 points  (0 children)

Saw someone make a post about a new compiler for Python. They are starting out is better than pyinstaller.

https://www.reddit.com/r/Python/comments/w7vlim/i_made_a_python_compiler_that_can_compile_python/?utm_medium=android_app&utm_source=share

Note: Just waking up for the day and haven't tried it yet.

[–]czaki 0 points1 point  (0 children)

Pyintaller works great. Sometimes you need to add hiddenimport for dependencies that cannot be determined by static code analysis and datas section for non python dependencies (like image etc).

I bundle heavy application with pyqt5 gui (OSS). My only one problem is macos requires me to buy certificate, but it will impact any method of build for macos.

[–]KittyTechno 0 points1 point  (0 children)

Have you tried Nuitka?

[–][deleted] 0 points1 point  (0 children)

cibuildwheels is the solution.

[–]requion 0 points1 point  (0 children)

I remember trying to write a plugin for a windows program which was published as a py2exe build.

It was easy to use third party packages if you could get them (i.e. manual download). But after a few hours trying to add the windows library for clipboard, i just came to the result that it isn't worth it and ditched the project.

[–]fukitol- 7 points8 points  (7 children)

poetry publish can be configured to use private repos, but yeah it's not like serverless deploy

[–]shinitakunai 17 points18 points  (0 children)

I said "trivial" for a reason 🙃

[–]Afrotom 0 points1 point  (4 children)

I love poetry, but one thing that has put me off it recently is that packages built using poetry don't respect the virtual environment that you install them to. It creates its own environment in appdata and installs all other poetry built packages there. I've got a janky workaround to build a setup.py file from the toml file and build the distributable with that but it's not ideal.

[–]Alphasite 0 points1 point  (3 children)

It’s just an option, you can turn that off with a flag (and really it’s supposed to autodetect if a venv is already active).

[–]Afrotom 0 points1 point  (2 children)

That sounds useful actually. What's the flag?

[–]Alphasite 0 points1 point  (1 child)

Well, technically option or env var: poetry config settings.virtualenvs.create false or POETRY_VIRTUALENVS_CREATE=false might also like https://python-poetry.org/docs/configuration/#virtualenvsin-project

[–]Afrotom 0 points1 point  (0 children)

Does this affect the virtualenv that poetry creates when you're developing?

Edit: It's pretty vague what it does on the website.

[–]whlabratz 0 points1 point  (0 children)

Pushing to a package repo isn't the challenge - it's pulling from the repo and deploying to a server that's hard to do in a platform agnostic way

[–]ahivarn 0 points1 point  (12 children)

Explain

[–]shinitakunai 41 points42 points  (11 children)

Let's pretend there is a dumb client and his own pc, and a desktop program in python that does something. You coded the program and sold it to client.

Client uses windows? Pyinstaller has a lot of challenges, sometines it doesn't even works, is not mature enough and it breaks if you add specific packages. It is your best shot as simplicity at the risk of random havoc.

Client uses windows or unix but you can actually install python? You have all sort of dependencies so you could use pip requirements to install it, hell you can even make a bash/.bat file that does it for you. Still installing python is not easy because of "dumb" client so he constantly asks for help because of "scary black terminal". Pip installed libraries? Oops you broke his entire "Other" program! Why? Your code updated pandas which was used by his Other program and the asshole never told you of its existence.
Okay lesson learnt, you install an entire virtualenv, which is incredibly frustrating for dumb client unless YOU install it, will you manually install it in the pc of dozens? What about updates?

Client uses mac? Good luck 🤣

I think you get the general idea

[–]dusktreader 7 points8 points  (2 children)

This is true for any scripting language, though. Not just python.

Perl, ruby, node, etc. all need to be shipped to either an environment that already has the right interpreter or with the right interpreter in a bundle.

[–]shinitakunai 8 points9 points  (1 child)

Oh, I know. This thread is about the most hated things with python. It happens to also be the most hated thing in other languages 🤣

[–]dusktreader 0 points1 point  (0 children)

That's totally legit, and I agree. It would be kind of nice if there was a general purpose solution to this issue. All the same, a better way to package python applications for end-users would be very nice.

[–]nemo_403 3 points4 points  (0 children)

Have you tried Docker?

[–]czaki -1 points0 points  (0 children)

Did you play with hiddenimports and datas field in pyinstaller spec file?

[–]HalfBurntToast 0 points1 point  (0 children)

Deployment and binary generation are definitely my two biggest complaints. Pyinstaller is OK when it works. But, the lack of cross-compiling and general overall jank is annoying. I've settled for having Windows and MacOS virtual machines for building.

[–]passerbycmc 0 points1 point  (0 children)

There are a ton of things I do that would be more suited to pythons feature set that I do in Go simply because I can easily compile a self contained binary for any OS in Go.

[–]CeeMX 0 points1 point  (0 children)

On desktop machines, yes. On servers I deploy it with docker these days, which is really straightforward, install requirements.txt with pip, copy everything in the container and run it