Skip to content

Latest commit

 

History

History
204 lines (139 loc) · 5.91 KB

FAQ.md

File metadata and controls

204 lines (139 loc) · 5.91 KB
Raw

FAQ for BlueBrain Spack Usage

Building Software

Q: How do I build and load a piece of software?

Expand answer

We'll install MG(1), a standard editor.

Make sure you're setup as per the instructions linked on the README, then install the editor via

  $ spack install mg

One can then run mg with

  $ spack load mg
  $ mg foo.txt

Q: Why do I have to rebuild the entire world?

Expand answer

If you are on the BlueBrain5, you shouldn't need to.

As described here, one can use the system packages available with an appropriate packages.yaml and upstreams.yaml in ${SPACK_ROOT}/etc/spack.

Q: Why is it so slow to interact with the Spack repository on GPFS

Expand answer

Make sure the spack repo is checked out in a subdirectory of $HOME. The spack repository is quite large, and when it is checked out under a /gpfs/bbp.cscs.ch/project/* directory, performance can be 10x slower than on the SSD provided storage of $HOME.

Q: Is there a binary cache?

Expand answer

We currently have a binary cache for central deployment only. As universally relocatable binaries are very fragile, we do not support binary caches for end-users.

Please make sure you have setup the correct configurations in:

  • ~/.spack/packages.yaml
  • ~/.spack/upstreams.yaml to avoid rebuilding packages that have already been build centrally.

Modules

Q: Why are the module files not being rebuilt?

Expand answer

The spack module tcl refresh command respects a blacklist that can be found via:

  $ spack config blame modules

Examples from our deployment workflow can be found in:

  • spack/deploy/configs/applications/modules.yaml
  • spack/deploy/configs/libraries/modules.yaml

Run spack --debug module tcl refresh and search for the module you expect to be built. Modify the whitelist to have the module built.

See also the general instructions.

Q: Why are my local modules broken after installing software manually?

Expand answer

When installing CMake-based software with

  $ spack setup package@version
  $ mkdir build
  $ cd build
  $ ../spconfig.py ..
  $ make
  $ make install

Spack will create a skeleton installation with bogus files to directly generate a module for the package to be installed. This may result in a "fake" library to be picked up when installing subsequent packages. Please use

  $ spack dev-build package@version

to install packages locally, and use spack setup only for local development/testing that other packages do not depend on.

Deployment

Q: How do I add/update my package/module versions?

Expand answer

We want to add a new version 2.0.0 to mypackage.

Make sure you're setup as per the instructions linked on the README, then change your package recipe to add or update the version specifying the corresponding tag or commit.

  $ spack edit mypackage
  …
  version('2.0.0', tag='v2.0.0')
  …

If the package is not yet deployed, then you can edit the environment depending on the type of package (applications.yaml, libraries.yaml…).

Assuming mypackage is a library:

  $ nvim deploy/environments/libraries.yaml

Under the specs section, make sure that the package is mentioned:

  - mypackage

Only specify a version if future updates to the package should be ignored.

After that you should edit the module file that will be at deploy/config/libraries/

  $ nvim deploy/config/libraries/modules.yaml

Under the whitelist section, ensure that your software is mentioned:

  - mypackage

Now you are ready to create a new branch and a PR with the changes. You can check the Jenkins build of your PR on Blue Ocean.

Pull Requests

Q: How do I test modules generated by a Pull Request?

Expand answer

If you followed the previous point you should be able to see if your PR was successfully built on Blue Ocean.

Then you can log into BB5 and run the following commands:

  $ module purge
  $ unset MODULEPATH
  $ source /gpfs/bbp.cscs.ch/apps/hpc/jenkins/pulls/xxx/config/modules.sh

Where xxx is the number of your PR.

At this point you should have the environment ready, so if your module was built correctly you should be able to load it.

  $ module load mypackage

Now you are ready to test mypackage.

Q: How do I debug my Pull Request?

Expand answer

To re-create the environment a Pull Request was built in, let's say #666, and debug failures, it is recommended to create a throw-away shell environment and execute the following commands. Note that the parameters in the first line correspond to the pull request and the stage you wish to debug (as labelled in Jenkins, but lowercase):

$ eval $(${SPACK_ROOT}/deploy/pull_env.sh pulls/666 applications)
$ spacktivate
$ spack install $(grep <my_failed_piece_of_software> ${HOME}/specs.txt)

Evaluating the first line will override local environment variables such as the current $HOME directory. After leaving the shell, this will leave a temporary directory behind, following the pattern spack_*. Please make sure to delete this directory when not needed any longer.