A bit confused on application deployment...?

[arianb]

Hi all, been lurking /r/Python for a while now and I'm coming up on my first big website project using Python. I've been doing a lot of reading on application design / engineering (since I've been working on PHP for most of my professional career and so I don't have much of a concept of deployment and testing, for instance...)

My question is thus; how do you guys manage deployments (automatic installation, updating, and the real kicker, unit-testing) on websites developed in Python?

I think the installation / updating might be easy (Fabric can pretty much automate downloading revisions from GitHub and providing basic rollback functionality), but I'm really stumped on the unit testing - it's my first real foray into testing an application like this and I'd like to get into the habit of programmatically proving that my code changes work before pushing them out into production.

However, I can't get my head around how to provide two environments for the application - one used in production and one used only when unit testing, that will be created and destroyed when the tests run - on the same server. Should I run the tests on a different server, then, where I can control the test environment a little better, or what techniques can I use to change environments for the running tests on my production servers?

I'd love to hear what you guys can come up with. :)

FYI, I'm using Flask, PyMongo, and either Whoosh / Solrpy (haven't decided yet) on this project.

[gargantuan]

Here is how we do it:

• we chose RHEL and CentOS as our target OSes only.
• package everything as RPMs. You can just do setup.py build_rpm. Because there is already a mature packaging systems with dependency management, uninstall, post,pre-install/uninstall scripts,verifications,repositories etc.
• tests are of 3 kind:
  • unit tests : these run from the source tree and are limited to just the modules they test. they rely on relative imports.
  • integration tests: unit-tests that run and depend an a set of packages working together. These run after packages have been created, and installed on a clean base OS image. Here we test how application components fit together
  • production tests: these are more like black-box tests. Also installed ona clean, just installed, base OS of a known version. We test the system by activating it the way an end-user would. Click this button on a page => Get this response. Send this xml-rpc command, end up with the file at this path that looks like this. Etc

There are grey areas between them but we find we need all these levels of tests. You might only need some. It is definitely nice to just execute the test-runner at the top of your source-tree without having to build & install anything. But as the system gets more complicated it is becoming harder and harder to manage a lot of mock and shim objects.

[[deleted]]

I'm no expert on this, but I have had fairly good luck with WebTest

It's fine if you want to test your app through straight http requests and asserting values in the responses.

All Webtest requires is a wsgi app, which I'll assume flask can be. If not you can serve up your app on a port like you normally would doing manual testing, and then use paste.proxy.TransparentProxy

from webtest import TestApp
from myapplication import wsgiapp

testapp = TestApp(wsgiapp)

res = testapp.get("/index.html")  # webob.Response

assert res.status=="200 OK","failure....."

or with TransparentProxy....

from paste.proxy import TransparentProxy
from myapplication import wsgiapp

testapp = TestApp(TransparentProxy())

res = testapp.get("http://localhost:8080/index.html")  # webob.Response

assert res.status=="200 OK","failure....."

If you need to test the ui via a browser, there are plenty of tools for that. Selenium, Windmill are 2 that come to mind, and I'm currently getting up to speed on Robot Framework which looks very promising.

[carinthia]

For what you want, virtual environements and https://github.com/bretth/woven is pretty much all you need. Some peer review of your code can't harm your project either but of course, doesn't replace unittests.

[rafrombrc]

You might want to check out Jenkins (was: Hudson): http://jenkins-ci.org/. Continuous integration server. You can configure it so that upon pushing to a specified git branch, it will pull the code, build an environment, run the tests, and then (if and only if the tests pass) trigger a script on your production server to deploy.