diff --git a/README.md b/README.md new file mode 100644 index 0000000..4f1271f --- /dev/null +++ b/README.md @@ -0,0 +1,87 @@ +# COSIMA Spack Configuration + +This repository contains the spack configuration and the spack environments used +by COSIMA to deploy software on gadi. + +## Instalation instructions + +Clone this repository and its submodules to some appropriate location (e.g., +`/g/data/ik11/spack/0.20.1`): +```bash +$ git clone --recursive /g/data/ik11/spack/0.20.1 +``` +Next, create the python virtual environment: +```bash +$ cd /g/data/ik11/spack/0.20.1 +$ ./bootstrap_venv.sh +``` +Finally, to use this spack installation one just needs to activate the python +environment: +```bash +$ . /g/data/ik11/spack/0.20.1/venv/bin/activate +$ which spack +spack () +{ + : this is a shell function from: /g/data/ik11/spack/0.20.1/spack/share/spack/setup-env.sh; + : the real spack script is here: /g/data/ik11/spack/0.20.1/spack/bin/spack; + _spack_shell_wrapper "$@"; + return $? +} +``` + +## Installing software + +It is recommended that all software be installed using spack +environments. Currently two environments are provided (the names should be +self-explanatory): +1. `access-om3-devel` +2. `common_tools_and_libraries` + +Installation of a spack environment is usually quite straightforward, but +because this can be a CPU intensive operation and take quite some time, it is +best to do this in parallel and to use an interactive job. + +### Step-by-step instructions: + + 1. Activate spack environment + +First activate the spack environment +```bash +$ spack env activate +``` +where `` by the actual name of the environment. + + 2. Download the sources + +As the compute nodes do not have internet access, one needs to download all the +necessary sources from the login node. This is done using a spack mirror. +```bash +$ spack mirror create -d sources -a +``` +Here `sources` is the name of a mirror that has already been configured. + + 3. Submit interactive job + +This should not use more than a single node. Also, make sure to add `gdata/ik11` +and `scratch/ik11` to the storage options. + + 4. Install software + +Once the job has started, because it starts a completly new shell session, one +needs to activate again both the python and the spack environments: +```bash +$ . /g/data/ik11/spack/0.20.1/venv/bin/activate +$ spack env activate +``` +Then one can simply do +```bash +$ spack install +``` +In this case, although each individual build will use some level of parallelism, +spack will proceed through the installation of the packages sequentially. To +fully use parallelism one needs to tell spack to create a Makefile and use this +to install the software: +```bash +$ spack env depfile -o Makefile +$ make -j +```