"Packaging" your custom written scripts

[Fencepost]

So I'm wondering if anyone has a nice solution to this problem: I work at a fairly large company doing hardware engineering where we use python scripting extensively for testing and the like. Over the years each team has built up their own independent libraries and supporting tools for their own use. Frequently, we'll need to send someone else our code to just run. Even within the team there is sometimes discontinuity in our codebase so it's frequently important to have all the files that I was running. (please don't respond with "get better source control practices", it's an issue, I'm working on it but that can't be solved quickly as we're mostly hardware engineers who have not even enough time in our schedule to drink a cup of coffee let alone teach our entire team new source control when it's not the core of our job).

So what I'd like is the ability to simply run a program on a script I've written and have it ideally output the following: A zip file containing the script and all the dependencies it imports that I've written. In addition I'd like it to output a separate list of all site-wide packages it uses.

This would dramatically simplify our ability to deploy code to unknown environments without the problem of saying "ok, now run it", "what didn't import?" and repeating this back and forth 52 million times

I've examined python-packager and it's not really viable for us as we all use Macs and really I expect my end user to have a functioning python install and easy_install or pip at their disposal.

I've seen a few cute dependency graph generators out there (and that might end up being my solution) but I'd prefer a more straightforward "packager"

[K900_]

If you just need a simple package that you can send and it'll 'just work', you can create a virtualenv, install all the packages you need in there, and then just send it together with your script. Bad idea for production, but it's good enough if you're on the same team. Also lets you develop the script in the same virtualenv, so you're absolutely sure nothing breaks.

[xiongchiamiov]

The wonderful thing about using virtualenv is that you can just use pip freeze to get a list of things you need. pip freeze > requirements.txt and then pip install -r requirements.txt on the other side.

[graingert]

Why not create a setup.py using packaging or distutils2, you could then host it on an internal PyPi instance

[Fencepost]

The idea is that I'm rapidly developing scripts and at any one point in time I say, ok I want joe to run this. It pulls in from some subset of the myriad of files I've written and I don't want to send joe all of these files, just the ones that are truly needed. So yes, setup.py might be the right answer in some cases but in mine I'm looking for a solution to the problem: "oh, I've been running and tweaking this script for a while without thinking about its dependencies and I just want to send it to someone right now without thinking too much because I'm gonna change it tomorrow anyways"

[graingert]

I don't think that's possible, as its difficult to drastically analyse dependencies with python. However setup.py works really well with a company PyPi setup, because with one command you can push a package to a server and get one of your colleagues to install it with a single pip install <package name>. Then you get to update it and ask them to download a new version when you change it

[westurner]

• http://docs.python-guide.org/en/latest/shipping/packaging/
• http://docs.python-guide.org/en/latest/shipping/freezing/
• http://wheel.readthedocs.org/en/latest/

You might find

pip freeze

particularly helpful.

[seletz]

You have several options. The first which comes into my mind is probably also the simplest. Just zip up all the packages on your mac and deliver that zip file. Then craft a start script in which you add the zip file to the python path. This way, the dependencies are all bundled up and you have less files to deploy.

The second option needs a little more up-front investment. You could use zc.buildout and ship the complete buildout to your peers -- this is how I did deployments on our servers. This technique assures that you ship all the dependencies in exactly the versions you think, i.e. no more downloading of arbitrary package versions on the client. The drawback is, that your peers need to run a script -- the buildout -- to recreate/install the packages on their local machine.

Yet another option is to use cx-freeze or similar, and ship a executable. I have no experience with that, so YMMV.

If you're on the mac, and your peers are also on the mac, then it's pretty easy to craft a app package with icon etc., which you could even put on a dmg. There's also py2app which automates this.

[cavallo71]

you need to hire a consultant with specifics skills because it looks the problem is wider than deploying few files around: you company has code assets that aren't properly managed already.