INSTALL

=====================================
Datatracker Installation Instructions
=====================================


General Instructions for Deployment of a New Release
====================================================


 1. In order to fetch a new release of the django datatracker code, simply
    check out the appropriate tag from svn::

      svn co http://svn.tools.ietf.org/svn/tools/ietfdb/tags/$releasenumber

 2. Don't forget to copy $releasenumber/ietf/settings_local.py from the
    old release to the new one; otherwise things won't work!
    ::

      cp $oldreleasenumber/ietf/settings_local.py $releasenumber/ietf/

 3. Change into the directory of the new release and run migrations::

      cd $releasenumber
      PYTHONPATH=$PWD ietf/manage.py migrate
      cd ..

 4. Then re-point the 'web' symlink::

      rm ./web; ln -s $releasenumber web

 5. Restart apache::

      sudo /etc/init.d/apache2 restart

 6. It's now also a good idea to go to the datatracker front page:

      http://datatracker.ietf.org/

    to check that it's alive and kicking, and displaying the new release
    number at the bottom of the left-side menubar :-)

 7. If things **aren't** cool, revert the symlink step, re-pointing the
    symlink to the release that was running before the new release, and restart
    apache again to roll back to that.


Installing from Scratch
=======================

In addition to the new release deployment instructions above, the
settings_local.py file has to be set up properly, and Apache has to be
configured.  Since the IETF datatracker is only intended to be deployed from
scratch once, these instructions don't cover this scenario in any more detail.
The general Django depoloyment instructions are relevant, however.


Patching a Production Release
=============================

Sometimes it can prove necessary to patch an existing release.  The following
process should be used:

 1. Code and test the patch on a development system copy of the production
    release which has no other code changes (or on trunk, with no uncommitted
    code changes, if it's sufficiently close).

 2. Produce a patch file, named with date and subject::

      ~/src/ietfdb/working $ svn diff > 2013-03-25-ballot-calculation.patch

 3. Move the patch file to the production server, and place it in 
    '/a/www/ietf-datatracker/patches/'

 4. Make a recursive copy of the production code to a new directory, named
    with a patch number.  Assuming the production code is in 4.43/, and we
    have web -> 4.43/::

      /a/www/ietf-datatracker $ rsync -a web/ 4.43.p1/

    (you could use 'cp -rp' instead, but rsync seems to do this faster).

 5. Apply the patch::

      /a/www/ietf-datatracker $ cd 4.43.p1/      
      /a/www/ietf-datatracker/4.43.p1 $ patch -p0 \
			< ../patches/2013-03-25-ballot-calculation.patch

    This should not produce any messages about failing to apply any chunks;
    if it does, you probably should go back to 1. and figure
    out why.

 6. Edit ``.../ietf/__init__.py`` in the new patched release to indicate the patch
    version in the ``__version__`` string and current date and time in the ``__date__``
    string.

 7. Change the 'web' symlink, reload etc. as described in
    `General Instructions for Deployment of a New Release`_ .


Installing a Secretariat Release
================================

Secretariat releases are based on a regular tagged release, but contain
updated code in the ietf/secr/ tree.  They are named with a trailing
'.secr<N>' version number.  If the tagged release is for instance 4.42, the
first secretariat release based on that would have release number 4.42.secr1,
and the second secretariat release based on it would be numbered 4.42.secr2.

Secretariat releases are deployed in the same manner as regular releases, with
the addition that patches made to the regular release needs to be picked up
and applied.  Assuming we'releasing 4.42.p1.secr2, then after we've cd'd into
the release diretory to run migrations, we'd also run a patch step to pick
up the current patches to the current production release, which is pointed
to by ../web::

  /a/www/ietf-datatracker/4.42.p1.secr2 $ svn diff ../web/ | patch -p2

Assuming that runs without error messages, the deployment is continued as
usual, with re-pointing the symlink and restarting apache.

Additional Version-Specific Instructions
========================================

Version 4.50
------------

Before you run step 3 (migration) of the general instructions, please run some specific
initial migrations with the a --fake argument:

      cd $releasenumber
      PYTHONPATH=$PWD ietf/manage.py migrate group 0001 --fake
      cd ..


Version 4.42
------------

 - In order to serve the secretariat tools static pages (such as image, css and js
   files) the exceptions to which urls need to be handled by the python-handler
   must be updated.

   In the datatracker test server, the following configuration has been used to
   make apache handle the static files, and Django the dynamic pages.  The new
   part is the <LocationMatch/> which mentions /secr/.  Adapt as needed for ietfa::

       ----------------
       # Logging and document root settings omitted

	   <Location "/">
	       PythonPath "['/srv/www/ietfdb'] + sys.path"
	       SetHandler python-program
	       PythonHandler django.core.handlers.modpython
	       SetEnv DJANGO_SETTINGS_MODULE ietf.settings 
	       PythonDebug On
	   </Location>

	   <LocationMatch "^/(robots.txt|favicon.ico|images|css|js|media|error)(/|$)"> 
	       SetHandler None
	   </LocationMatch> 

	   # New for secretariat tools:
	   <LocationMatch "^/secr/(img|css|js|error)"> 
	       SetHandler None
	   </LocationMatch> 

	   <Location "/accounts/login/">
	       AuthType Digest
	       AuthName "IETF"
	       AuthUserFile /var/local/loginmgr/digest
	       AuthGroupFile /var/local/loginmgr/groups
	       AuthDigestDomain http://tools.ietf.org/
	       Require valid-user
	   </Location>

       # Caching and compression settings omitted
       ----------------

Version 4.40
------------

 - (DONE) Add ianasync user with an auth role in the "iana" group and an
   rfceditorsync user with an auth role in the "rfceditor" group (don't
   think Group(acronym="rfceditor") exists yet); IANA and RFC Editor need
   to know the passwords for the poke mechanism


 - (DONE) Make sure mailing list for iab-stream@iab.org is up (since we're now
   emailing that)

 - (DONE) Set rfc_must_published_later_than date in bin/iana-protocols-updates to today

 - (DONE) Run the 3 new doc South migrations

 - (DONE) New polling scripts, to be run every hour::

     web/ietf/bin/iana-changes-updates
     web/ietf/bin/iana-protocols-updates
     web/ietf/bin/rfc-editor-index-updates (replaces mirror_rfc_index)
     web/ietf/bin/rfc-editor-queue-updates (replaces mirror_rfc_queue)

 - (DONE) Import old events from IANA::

     bin/iana-changes-updates --from="2005-01-01 00:00:00" --to="2013-01-31 00:00:00" --no-email

 - (DONE) Pipe IANA review emails to the datatracker.  There used to be an action to pipe
   such mails to henrik@levkowetz.com, for testing this feature, but I haven't seen
   any in a little while, so I don't know if this has broken.  Anyway, the iana review
   emails should be piped into::

     /www/ietf-datatracker/web/ietf/bin/iana-review-email


 - (DONE) Tell IANA we're doing this for real now.


Version 4.34
------------

The migration step you do as a part of the release sequence is going to take
quite some time -- probably minutes.  It will generate some output while it's
working, too.  As long as it doesn't halt and say that something failed or
gave an error, this is as expected, and when it terminates, you should be OK
to continue.

Version 4.21
------------

This release will you to run migrations before moving the link to the new
version and doing the apache reload.  I know you have a routine for the steps
needed to deploy a new release by now, but thought I'd mention it, anyway.

If there is any problem at all doing the migrations, then you'll need to
do a fake initial migration, as follows::

  web $ PYTHONPATH=PWD ietf/manage.py migrate --fake meeting 0001

and then to the regular migration again.

Version 4.20
------------

Some one-time actions that need to be taken are as follows::

    Assuming that the release has been checked out in /a/www/ietf-datatracker/4.20:

      cd /a/www/ietf-datatracker/4.20

      PYTHONPATH=$PWD ietf/manage.py migrate --fake doc 0001
      PYTHONPATH=$PWD ietf/manage.py migrate --fake name 0001

      PYTHONPATH=$PWD ietf/manage.py dbshell <<< "delete from django_content_type where app_label='doc'
		and model='groupballotpositiondocevent';"

      PYTHONPATH=$PWD ietf/manage.py migrate doc || { \
	PYTHONPATH=$PWD ietf/manage.py dbshell <<< 'CREATE TABLE `doc_groupballotpositiondocevent`
		(`block_comment` longtext NOT NULL, `comment` longtext NOT NULL,
		`ad_id` integer NOT NULL, `comment_time` datetime NULL,
		`block_comment_time` datetime NULL, `pos_id` varchar(8) NOT NULL DEFAULT "norecord",
		`docevent_ptr_id` integer NOT NULL PRIMARY KEY);'
	PYTHONPATH=$PWD ietf/manage.py dbshell <<< 'DROP TABLE `doc_ballottype` CASCADE;'
	PYTHONPATH=$PWD ietf/manage.py dbshell <<< 'DROP TABLE `doc_ballottype_positions` CASCADE;'
	PYTHONPATH=$PWD ietf/manage.py dbshell <<< 'DROP TABLE `doc_ballotdocevent` CASCADE;'
	PYTHONPATH=$PWD ietf/manage.py dbshell <<< 'ALTER TABLE `doc_ballotpositiondocevent`
		DROP COLUMN `ballot_id` CASCADE;'
      }

      PYTHONPATH=$PWD ietf/manage.py migrate doc
      PYTHONPATH=$PWD ietf/manage.py migrate name
      PYTHONPATH=$PWD python ietf/wgcharter/migrate.py | tee -a ~/charter-migration.log