PostgreSQL Installer build system 
=================================

This is the PostgreSQL Installer build system. This document attempts to 
describe how the system is architected, how to set it up and how to extend 
it. It is a work in progress and will no doubt require further refinement
over time. There there is one goal however:

Build all PostgresSQL & add-on package installers for all supported platforms
with a single command.

Note that this system is not intended to replace the existing installer system 
used on Windows (pgInstaller) - it is intended to mirror it's basic functionality
however.

Modular system design
---------------------

The modular system is designed to be as flexible as possible and allow package
authors as much freedom as possible in the way they design their installers. 
There are some basic rules about how we design add-on packages however - it 
remains up  to the individual author to determine whether or not breaking any 
rules will break their package. They had better not break the system though!

* Registration:

A central registry file is used in which packages should register themselves. This
data will be used by StackBuilder to locate installed packages. The registry 
file is /etc/postgres-reg.ini, and should be considered analagous in function to
the sections of the Windows registry used for the same purposes on that platform.

StackBuilder requires specific entries for the PostgreSQL server, as well as an
entry indicating the installed version of each unique package. An example file
is show below.

; This section is for a server, and is analagous to the PostgreSQL key under
; HKEY_CURRENT_USER\Software on Windows
[PostgreSQL\8.3]
Version=8.3.3
InstallationDirectory=/opt/PostgreSQL/8.3
DataDirectory=/opt/PostgreSQL/8.3/data
Port=5432
Superuser=postgres

[PostGIS_1_3_PG83]
Version=1.3.2

[pgAdmin3]
Version=1.8.4
InstallationDirectory=/opt/pgAdmin3

It is up to the uninstaller for each package to leave or clean the data during 
uninstallation. The version number for a package should *always* be
cleared, but other data may be retained. For example, the server package will
not remove the data directory, thus it is appropriate to leave the 
DataDirectory, Port and Superuser values intact.

* Installers:

Each package installer should be capable of being silently or interactively 
installing and uninstalling the package. When uninstalling, as much of the package
as possible should be removed, however it is not always possible (through
lack of reference counting between packages) or desirable to remove everything.

Build platform
--------------

The build platform is Mac OS X Tiger. We use OS X because it allows us to run
all other Intel based OS's on the same machine. All other supported platforms
are run in VMWare Fusion virtual machines, with the exception of Solaris Sparc
which obviously must be an external machine. The supported platforms can be set
as external machines if required by exposing nfs shares from mac.

A number of additional dependencies are required when setting up the system,
such as libxml2 and libxslt, as well as useful utilties such as wget. Setting
these up correct is essentially to ensuring future Universal binary builds will
work as expected on all supported OS X versions (10.4/Tiger and above).

The builds use the -isysroot compiler flag to ensure the use of the 10.4u SDK.
This causes the linker to prefix some library paths with the isysroot path, which
can cause linker errors with private libraries such as libgeos. To resolve this,
it is necessary to symlink the build root into the SDK, e.g:

sudo ln -s /Users /Developer/SDKs/MacOSX10.4u.sdk/Users

For additional discussion of this issue, please see:
http://lists.apple.com/archives/Darwin-dev/2006/Apr/msg00041.html

Utilities (bison and automake 1.9/1.10 is required, others such as wget may be
useful) should be installed using MacPorts:

- Download the installer from http://www.macports.org/ and install the package.

- Add /opt/local/bin:/opt/local/sbin to the *end* of the path. Add the following
  line to ~/.bash_profile for the buildfarm user:

export PATH=$PATH:/opt/local/bin:/opt/local/sbin

- Edit the MacPorts variants configuration to ensure we always get Universal 
  builds. Add the following line to /opt/local/etc/macports/variants.conf:

+universal

- Install bison and friends:

$ sudo port install bison
$ sudo port install wget
$ sudo port ossp-uuid

Dependency libraries must be built with a little more control to ensure they 
use the correct SDK to allow them to be used on Tiger and above. We manually
build and install these packages into /usr/local/

- Download the source for each library (eg. libxml2 & libxslt).

- Unpack the source into /usr/local/src

- Configure the source with a command such as:

CFLAGS="-isysroot /Developer/SDKs/MacOSX10.4u.sdk -mmacosx-version-min=10.4" ./configure --prefix=/usr/local/ --disable-dependency-tracking

- Build and install:

make all
sudo make install

Note that we must make sure all additional libraries link against these
libraries, and not the older, system copies. In the case of libxslt, we can
do this by configuring with --with-libxml-prefix=/usr/local

- Set up DocBook SGML

Configuring DocBook SGML on Mac OS X with MacPorts.

1) Install the following packages using MacPorts:

    opensp
    openjade
    docbook-xsl
    docbook2X

Note: opensp might give error for universal mode, if installation fails, try
turning off universal mode:

sudo port install opensp -universal

2) Download the docbook-dsssl stylesheets from 
http://sourceforge.net/project/showfiles.php?group_id=21935&package_id=16611 
and then unpack the archive in /opt/local/share/sgml

3) Add a symlink so PostgreSQL can find the stylesheets:

     sudo ln -s  /opt/local/share/sgml/docbook-dsssl-1.79 \
                 /usr/local/share/sgml/docbook-dsssl

3) Download DockBook 4.2 (http://www.docbook.org/sgml/4.2/docbook-4.2.zip) 
   and the ISO 8879 character entities (http://www.oasis-open.org/cover/ISOEnts.zip)

4) Unzip both archives into /opt/local/share/sgml/docbook-4.2

5) Ensure that all the unpacked files are world readable.

6) Run the following command in the docbook-4.2 directory:

    perl -pi -e 's/iso-(.*).gml/ISO\1/g' docbook.cat

7) Create the file /opt/local/share/sgml/catalog, with the following contents:

CATALOG "openjade/catalog"    
CATALOG "docbook-4.2/docbook.cat"
CATALOG "docbook-dsssl-1.79/catalog"  

8) Make sure that settings.sh is updated with docbook installation path.

Build VMs
---------

All VMs (and in fact, the host machine) are setup to use user accounts called
'pginstaller'. In order to access each, the VMs must be setup with fixed IP
addresses which are recorded with an appropriate hostname in /etc/hosts on the
host machine. Each hostname is specified in settings.sh. It may be necessary to 
manually configure VMWare Fusion to bridge the network adaptor instead of using
NAT.

The top level 'pginstaller' directory is shared with all the VMs using the VMware
shared folders feature. The path to this directory is specified in settings.h
for each VM. Note that VMware doesn't map UIDs/GIDs between the host and the VMs
so it may be necessary to mount the shared directory using the UID/GID of the
user in the VM, eg using the following in /etc/fstab:

.host:/  /mnt/hgfs  vmhgfs  defaults,ttl=5,uid=500,gid=500     0 0

SSH authentication between hosts is achieved using certificates. These can be
generated on the host machine using:

ssh-keygen -t rsa

Copy the resulting id_rsa.pub file to ~/.ssh/authorized_keys on each VM. 

* Linux/Linux-x64
- Install chrpath utility in order to change the rpath of the installed PostgreSQL binaries in the staging directory.
  Use the following command to install the chrpath:
  * yum install chrpath

* Windows

The Windows VM is the most tricky to setup:

- Install Windows XP Pro with service pack 2
- Install an appropriate anti-virus package
- Install Visual Studio 2005, and update to the latest service pack
- Create the 'buildfarm' user account, as a limited user.
- Install the a basic installation of Cygwin from http://www.cygwin.com/. 
  Include the OpenSSH package.
- Configure sshd with ssh-host-config, using the buildfarm user account as
  the service account. Make sure that openssh log file has correct ownership
  and permissions. It is absolotely necessary NOT to use default local account
  with sshd because compilation will fail when invoked via ssh.
- Make sure that port 22/TCP is open in the Windows Firewall configuration.
- Install the public ssh key in C:\Cygwin\home\buildfarm\.ssh
- Install zip.exe and unzip.exe into the System32 directory. These utilities
  can be found at ftp://ftp.tex.ac.uk/tex-archive/tools/zip/info-zip/WIN32/
- Create folder 'c:\pgBuild'
- Depending upon the modules to build, install various utilities in c:\pgBuild
- Install bison, flex, diffutils in c:\pgBuild (Available from http://gnuwin32.sourceforge.net)
- Prebuilt iconv, libxml2, libxslt, openssl and zlib from http://zlatkovic.com/pub/libxml,
  install them in c:\pgBuild
- gettext (Please consult developer for specific version for PostgreSQL)
- Mingw (gcc and g++). 
- MSys 1.0 
- Compile and install ZLib in Msys using --prefix=/mingw
- krb5 in c:\pgBuild
- vcredist in c:\pgBuild
- apache ANT in c:\pgBuild
- wxWidgets in c:\pgBuild
 
Build Machines as external machines
-----------------------------------
In order to set build machines as external machines, Create NFS share pointing to 
top level 'pginstaller' directory on Mac. For this purpose free tool 'NFS Manager' 
can be used. On linux side, update /etc/fstab to create nfs mount to this NFS share. 

Build scripts
-------------

* settings.sh

This script is derived from settings.sh.in which is stored in source control. It
is configured for the specific build machine, and allows us to specify what
platforms and modules we're building, and some global configuration options.

This script (_and_ the source version, settings.h.in) must be edited whenever
new platforms or packages are added.

* common.sh

This script contains common utility functions that may be used throughout the
build system.

* build.sh

This script is the main build script. To build everything, simply run the 
following command on the build host:

sh build.sh

For quick rebuilds, an option is provided to rebuild just the installers from 
the existing code in the staging directories:

sh build.sh -skipbuild

This script must be edited whenever a new module is added to call the appropriate
functions in the package build script.

Directories
-----------

* output/

This directory will contain all the completed installers.

* scripts/

This directory contains miscellaneous scripts that may be useful to multiple
modules or the overall build system.

* resources/

This directory contains installer resources that may be useful to multiple
modules or the overall build system.

* tarballs/

This directory contains all the tarballs we use for builds

* <everything else>/

Each additional directory contains a single package. These may be internally built
as required, though the interface should remain consistent - ie. a single build
script called build.sh, exposing functions called _prep_<packagename>, 
_build_< packagename > and _postprocess_< packagename >.

For a description fo the build system for a single package, see server/README.

Additional configuration in the VM's :
--------------------------------------

* Adding gd module to php in Windows
 
   * Prequisites:

     1) jpeg     (http://nchc.dl.sourceforge.net/sourceforge/gnuwin32/jpeg-6b-4.exe)
     2) libpng   (http://nchc.dl.sourceforge.net/sourceforge/gnuwin32/libpng-1.2.36-setup.exe)
     3) freetype (http://nchc.dl.sourceforge.net/sourceforge/gnuwin32/freetype-2.3.5-1-setup.exe)

     Install these in the pgBuild directory as jpeg, libpng and freetype respectively.

   * Modifications:

      Freetype:

       1) Modify the directory structure as:

          freetype --> include --> freetype2 --> freetype
          to
          freetype --> include --> freetype

          (leave the ft2build.h file in include directory as it is.)

       2) Copy the files:

          freetype/lib/freetype.lib to freetype/lib/freetype2.lib

      jpeg:

       1) Copy the files:

          jpeg/lib/jpeg.lib to jpeg/lib/libjpeg.lib


* Adding gd module to php in osx

   * Prequisites:
       1) Install jpeg libraries
       Download and extract jpeg from http://www.ijg.org/
       Compile and install: 
       >env CFLAGS="-isysroot /Developer/SDKs/MacOSX10.4u.sdk -mmacosx-version-min=10.4 -arch i386 -arch ppc" LDFLAGS="-arch i386 -arch ppc" ./configure --prefix=/usr/local --disable-dependency-tracking
       >make
       >sudo make install
      
      2) Install libpng
      Download and extract libpng from http://www.libpng.org/pub/png/pngcode.html
      Compile and install:
      >env CFLAGS="-isysroot /Developer/SDKs/MacOSX10.4u.sdk -mmacosx-version-min=10.4 -arch i386 -arch ppc" LDFLAGS="-arch i386 -arch ppc" ./configure --prefix=/usr/local --disable-dependency-tracking
      >make
      >sudo make install

      3) Install freetype
      Download and extract freetype from http://freetype.org/download.html
      >env CFLAGS="-isysroot /Developer/SDKs/MacOSX10.4u.sdk -mmacosx-version-min=10.4 -arch i386 -arch ppc" LDFLAGS="-arch i386 -arch ppc" ./configure --prefix=/usr/local --disable-dependency-tracking
      >make
      >sudo make install


* Adding gd module to php in linux

   * Prequisites:

       1) yum install freetype
       2) yum install libpng
         (libjpeg.so should also be present in /usr/lib and /usr/lib64 for linux and linux-x64 respectively)

Further info
------------

Contact dpage@pgadmin.org for further info.