-
Notifications
You must be signed in to change notification settings - Fork 89
Building Albany using Spack
Spack is a package manager for supercomputers, Linux, and macOS. It makes installing scientific software easy. With Spack, you can build a package with multiple versions, configurations, platforms, and compilers, and all of these builds can coexist on the same machine. The following links provide some documentation on Spack:
- Spack web page https://spack.io
- Spack documentation is at http://spack.readthedocs.io/en/latest/
- Spack Google Group https://groups.google.com/forum/#!forum/spack
- Specific examples of
~/.spack/linux/compilers.yamlfiles are highlighted below - Some slides by Xylar Asay-Davis (@xylar) about the Albany-supported spack build
A working Albany build is supported in the E3SM fork of the spack repo: https://github.com/E3SM-Project/spack.
git clone git@github.com:E3SM-Project/spack.git
cd spack
git checkout develop
. share/spack/setup-env.sh
This approach requires uploading the ssh-keys for your machine to github (under Settings). An alternative to the first line above is: https://github.com/E3SM-Project/spack.git if you prefer cloning using HTTPS instead of SSH, which does not require uploading ssh-keys.
Note that spack will try to download and install packages from the web. If your machine is behind a proxy, you will need to set proxies (export http_proxy=..., export https_proxy=...) prior to running . share/spack/setup-env.sh above.
spack --insecure install --dirty --keep-stage albany
The --keep-stage option will ensure that the Albany build is kept after the installation of Albany is complete. This is required if you plan to run the Albany regression tests post-build.
If you are interested in building Albany for the purpose of installing MPAS-Albany on top of Albany, please use the following command:
spack --insecure install --dirty --keep-stage albany+mpas
If you are interested in building the PyAlbany capability within an MPAS-Albany-enabled build, the following command should be used:
spack --insecure install --dirty --keep-stage albany+mpas+py
Note that Albany requires one to use one of the following compilers:
Clang 4.0.0 or higher
GCC 5.3.0 or higher
Intel 17.0.0 or higher
NVCC 9.2.88 or higher
Instructions are provided below for how to check what compiler you have, how to build a new compiler using spack, and how to tell spack with what compiler to build Albany.
spack location -i albany
To cd to that location automatically, one can run the following command:
spack cd albany
First, cd to the Albany build directory, which be default will be /tmp/$USER/spack-stage/spack-stage-albany-develop-*/spack-build* (this can also be reached by running spack cd albany. From here, you can execute the command ctest -V to run the tests (or run the tests manually).
If you have built Albany with the --keep-stage option, you can find the source code by default in the following directory: /tmp/$USER/spack-stage/spack-stage-albany-develop-*/spack-src. If you wish to make a change to the source code and recompile the code, you simply make the chance, then cd to the build directory (by default /tmp/$USER/spack-stage/spack-stage-albany-develop-*/spack-build*) and type make -j 16. No module loading / environment specification should be necessary.
-
Building some of the Albany dependencies might result in certificate errors. To ignore certificates, use the command
spack --insecure install albanyNote that ignoring ssh certificate issues can compromise your system's security. Note that
spack --insecure install albanyis equivalent tospack -k install albany. The--insecureoption is in general required on machines behind firewalls (with proxies). -
To build Albany using your current shell environment, instead of the Spack environment
spack install --dirty albany -
To build Albany using 16 processors with verbose build output
spack install -j 16 -v albany -
To run the Albany regression tests post-build, change into the build stage directory and run them with CTest
spack cd albany cd ../spack-build ctest
To run just the PyAlbany tests, a user can execute the following command:
ctest -R PyAlbany
- Uninstalling a spackage: spack unistall albany
The following is an example compilers.yaml file. It is typically found in the following location on Linux: ~/.spack/$arch/compilers.yaml.
compilers:
- compiler:
environment: {}
extra_rpaths: []
flags: {}
modules: []
operating_system: rhel6
paths:
cc: /sierra/sntools/SDK/compilers/clang/6.00-RHEL6/bin/clang
cxx: /sierra/sntools/SDK/compilers/clang/6.00-RHEL6/bin/clang++
f77: null
fc: null
spec: clang@6.0.0
target: x86_64
- compiler:
environment:
set:
LD_LIBRARY_PATH : /sierra/sntools/SDK/compilers/gcc/5.4.0-RHEL6/lib:/sierra/sntools/SDK/compilers/gcc/5.4.0-RHEL6/lib64
extra_rpaths: []
flags: {}
modules: []
operating_system: rhel6
paths:
cc: /sierra/sntools/SDK/compilers/gcc/5.4.0-RHEL6/bin/gcc
cxx: /sierra/sntools/SDK/compilers/gcc/5.4.0-RHEL6/bin/g++
f77: /sierra/sntools/SDK/compilers/gcc/5.4.0-RHEL6/bin/gfortran
fc: /sierra/sntools/SDK/compilers/gcc/5.4.0-RHEL6/bin/gfortran
spec: gcc@5.4.0
target: x86_64
To generate a default compilers.yaml file, please run spack compiler find.
To see what compilers you have on your machine, you can run spack compilers. Lets say this command returns the following:
==> Available compilers
-- gcc rhel7-x86_64 ---------------------------------------------
gcc@7.2.0 gcc@6.1.0 gcc@4.8.5 gcc@4.4.7
and we wish to build Albany for MPAS with gcc-7.2.0. This can be done as follows:
spack --insecure install --dirty --keep-stage albany%gcc@7.2.0+mpas
Lets say the compiler you wish to use is not available on your system, e.g., it is gcc@8.4.0. One first needs to build the compiler using spack:
spack --insecure install --dirty --keep-stage gcc@8.4.0
Once that is done, one needs to make a note of the location in which the compiler was installed, and update one's compilers.yaml file to have an entry for the new compiler. Next, one can run spack compilers again and verify that gcc@8.4.0 shows up. Finally, to build Albany with gcc-8.4.0, one runs the following command:
spack --insecure install --dirty --keep-stage albany%gcc@7.2.0+mpas
If you wish to use a different pre-built compiler for the spack build, you can add an entry to it in the ~/.spack/$arch/compilers.yaml file, following the example above.
By default, spack will stage its builds in the /tmp/$USER/spack-stage directory. To change this, edit the spack/etc/spack/default/config.yaml, line 62 to specify the new path, e.g.,
build_stage:
- /scratch/albany/nightlySpackBuild/spack-stage
In some cases, a user may not want spack to build every single TPL on which Albany depends; rather, one may want to have SPACK use pre-built TPLs. This can be accomplished by creating a ~/.spack/packages.yaml file that points to the pre-installed TPLs. This file will look as follows:
packages:
openmpi:
paths:
openmpi@1.10.7: /projects/sems/install/rhel7-x86_64/sems/v2/tpl/openmpi/1.10.7/gcc/10.1.0/base/7jgrwmo
buildable: False
cmake:
paths:
cmake@3.23.1: /projects/sems/install/rhel7-x86_64/sems/v2/utility/cmake/3.23.1/gcc/8.3.0/base/frlm6uo
buildable: False
boost:
paths:
boost@1.74.0: /projects/sems/install/rhel7-x86_64/sems/v2/tpl/boost/1.74.0/gcc/10.1.0/base/ua5ge5q
buildable: False