On Staging with Python and Setuptools

tl;dr One environment per application revision (think package). Packages are immutable. Use symbolic links to switch between packages. Download from Github.

You’ve laboured and toiled over your code and after endless hours of polishing your app finally is the gem you intended to build. But how to bring it into production? Is there a good practices? Actually yes, there is:

Test in a clone of the production environment. The point of testing is to flush out, under controlled conditions, any problem that the system will have in production. If you test in a different environment, every difference results in a risk that what happens under test won’t happen in production.

Martin Fowler, Continuous Integration

To wrap up my understanding of staging based on Fowler’s recommendation:

  1. Decrease uncertainty
  2. Lever your application onto the target platform and verify its correctness
  3. Provide an atomic and non-intrusive (one-step) deployment work-flow
  4. Ensure a role-back option

In my opinion it is therefore necessary to uphold a set of invariants:

If you try to actually live up to this practice there appears to be little practical support, though. Take Setuptools for an example of a popular distribution scheme. I would love to read the success story of someone who has put Setuptools’ notion of “staging areas” and deployment into a non-intrusive work-flow without the premises I stated above.


This setup is actually in use by Nerdcorner and also in a not-too-different manner by my former employer since 2009.

A deployment to either stage (ie. staging) would require the following steps:

  1. Build source distribution on development machine (or any other convenient platform)
  2. Upload the distribution into the package repository
  3. Build environment and install application into companion: deploy_py.sh projectA-0.4.tar.gz
  4. Create symbolic link to the stage’s document root: rm staging && ln -s projectA-0.4 staging

That’s it. See my Github for the code: https://github.com/lusitania/keep-it-simple


Every environment is shared by exactly one application revision. The reason for this is twofold:

Non-intrusive deployment. Honestly, putting your app to work is a side show. It’s a process you do once in a while, which never changes and simply has to work. You are getting paid for a running app, not for the process of making it so. That is why

Atomicity. Critical steps must not fail. Any failure and verification must have happened before (package errors, failed unit test, …). In the example above: The system’s behaviour changes only once the vhost was re-linked. Removing and setting a symbolic link is the most atomic shell operation I can think of. This also doesn’t break Setuptools’ records in the companion environment since the application was installed in its true package location and not in one of the document roots. You are obviously not allowed to rename the folder in the package repository.

Role-back. Things can still go wrong. Don’t hamper yourself by overwriting previous states, keep your good stuff for a while.