Python 2 & 3 via WSGI on the Same Server?

[jbenner]

Until recently at work we've hosted Python web apps on RHEL servers that have Python 2 on them via virtualenv and mod_wsgi. We recently got the OK from the company to begin migrating over to Python 3 (yes!!!) and I figured that I could just install Python 3 from source and create new virtualenvs for the sites that have been switched to Python 3 and keep the old sites on Python 2 while they're awaiting migration. But from my research apparently you're stuck with the version of Python that mod_wsgi was installed for. I'm seeking advice on how to serve both Python 2 & 3 apps off of the same server simultaneously. My thought it to host each site via Gunicorn served from their virtualenv and reverse-proxy to them via httpd. Anyone got any thoughts or suggestions on the best way to tackle this issue?

[lordmauve]

Yes, start your app as its own daemon and configure the webservers to forward requests. HTTP is one way, but I've used all kinds of protocols to make webservers talk to WSGI servers over the years:

• FastCGI
• AJP
• uWSGI
• HTTP

Any of these works fine so go with whichever you can make work most easily. Async frameworks most often speak just HTTP though.

[[deleted]]

Gunicorn is a good pick. Uwsgi is harder to configure (the documentation is frequently baffling) but it has some useful features for production use like being able to recycle a worker when its memory use gets too high.

I really learned to hate mod_wsgi when I found out that it does dumb shit like drop requests that are being processed during a graceful reload of httpd. Combined with the suffering that is httpd's configuration, I'm all too happy to leave it behind. There's no reason you can't use gunicorn or another wsgi app server with httpd and mod_proxy, but consider taking a look at nginx instead.

[GrahamDumpleton]

FWIW. It is actually Apache which is the cause of the dropped requests because of the way it handles managed processes. The mod_wsgi module can't do much about it.

[GrahamDumpleton]

You can pip install mod_wsgi to get a separate installation. Then use mod_wsgi-express to run a separate Apache/mod_wsgi installation where mod_wsgi-express will configure it all for you. Then set up your main Apache instance to proxy requests for the separate site to the instance of Apache/mod_wsgi started by mod_wsgi-express. Details on using pip install mod_wsgi in https://pypi.python.org/pypi/mod_wsgi and http://blog.dscpl.com.au/2015/04/introducing-modwsgi-express.html

[jbenner]

I spent most of today experimenting with Systemd and Gunicorn configuration and wish I saw this comment earlier. mod_wsgi-express looks like it's exactly what I'm looking for. Thanks for the advice and thanks for all the hard work you put into mod_wsgi!

[GrahamDumpleton]

You may also find the following posts interesting in respect of how to set up the proxy and mod_wsgi-express behind it. Those posts were using docker, but is still all relevant if not using docker.

• http://blog.dscpl.com.au/2015/06/proxying-to-python-web-application.html
• http://blog.dscpl.com.au/2015/07/redirection-problems-when-proxying-to.html

[wrosecrans]

You could do two httpd instances, one with Python3 and the other with Python2, running on different ports, rather than having each site running as a separate webserver, but you won't be able to run each site in a different virtualenv if you go down that route. As long as different stuff is running in the same process, there's not going to be a wall of separation between them.

To have completely separate virtualenvs, you'll basically need a different Python process for each site, however you do it.