Disallow spidering on any site in Test Mode

[tmtmtmtm]

[chrismytton]

https://pypi.python.org/pypi/django-robots could be useful for providing a dynamic robots.txt for subdomains.