Skip to content

Commit cbb6802

Browse files
mboissonmboisson
authored
Merge pull request #33 from ComputeCanada/tecc-rsnt-doc
TECC RSNT doc
2 parents d2a9333 + 4150bf6 commit cbb6802

File tree

12 files changed

+2112
-4
lines changed

12 files changed

+2112
-4
lines changed

‎README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,16 @@
11
# Compute Canada software stack
2-
This meta repository is meant to host issues related to the Compute Canada software stack that is available through our public CVMFS.
2+
This meta repository is meant to host issues related to the Compute Canada software stack that is available through our public CVMFS.
33

44
* [Documentation on using our software stack on your computer](https://docs.computecanada.ca/wiki/Accessing_CVMFS)
5+
* [Documentation on managing the software stack](doc/INDEX.md) (mostly for CC staff)
56

67
## What should go in this repository
78
* Public issues regarding the use of Compute Canada's software stack on infrastructure not managed by Compute Canada. For example issues regarding the customization of the environment to other infrastructure
8-
* Major initiatives, such as deploying a new standard environment (StdEnv modules).
9+
* Major initiatives, such as deploying a new standard environment (StdEnv modules).
910
* Bugs in the standard environments
1011

1112
## What should not go in this repository
12-
* Requests for installing specific software (these are handled through our [Technical support](https://docs.computecanada.ca/wiki/Technical_support) team).
13+
* Requests for installing specific software (these are handled through our [Technical support](https://docs.computecanada.ca/wiki/Technical_support) team).
1314

1415
## Relevant repositories
1516
* https://github.com/ComputeCanada/easybuild-easyconfigs
@@ -19,4 +20,3 @@ This meta repository is meant to host issues related to the Compute Canada softw
1920
* https://github.com/ComputeCanada/nixpkgs
2021
* https://github.com/ComputeCanada/gentoo-overlay
2122
* https://github.com/ComputeCanada/wheels_builder
22-

‎doc/INDEX.md

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
# Software management
2+
3+
**Note:** Staff members should first read our internal [Software
4+
Installation](https://wiki.computecanada.ca/staff/Software_installation)
5+
documentation to learn how to request privileged access, set up their account,
6+
process software installation tickets, etc.
7+
8+
### Contents
9+
10+
- [Introduction](#introduction)
11+
- [Initial setup](#initial-setup)
12+
- [Choosing an installation method](#choosing-an-installation-method)
13+
- [Nix or EasyBuild?](#nix-or-easybuild)
14+
- [Gentoo or EasyBuild?](#gentoo-or-easybuild)
15+
- [Python packages](#python-packages)
16+
- [Installing software](#installing-software)
17+
- [With EasyBuild](#with-easybuild)
18+
- [With Nix](#with-nix)
19+
- [With Gentoo](#with-gentoo)
20+
- [Python packages](#python-packages)
21+
- [Deploying software](#deploying-software)
22+
- [Testing software](#testing-software)
23+
- [Examples](#examples)
24+
25+
## Introduction
26+
27+
The software available to Compute Canada users, including scientific and
28+
research software, is maintained in a single central software stack accessible
29+
from all Compute Canada clusters. This presents users with a consistent
30+
experience across all current and future Compute Canada sites.
31+
32+
A variety of tools are used to achieve this. [Nix](https://nixos.org/nix/) and
33+
[Gentoo](https://www.gentoo.org/) provide an underlying layer containing the
34+
basic tools and commands usually provided by Linux operating systems.
35+
[EasyBuild](https://github.com/hpcugent/easybuild) is used to build scientific
36+
and research software, parallel and numerical libraries, and other
37+
performance-critical software.
38+
[Lmod](https://www.tacc.utexas.edu/research-development/tacc-projects/lmod)
39+
makes software packages available as modules that users can selectively add to
40+
their environment. Custom [Python Wheels](http://pythonwheels.com/) provide
41+
Python packages optimised for the Compute Canada software stack. Finally,
42+
[CVMFS](https://cvmfs.readthedocs.io/en/stable/) distributes the software stack,
43+
making usable on all Compute Canada clusters and also publicly accessible.
44+
45+
## Initial setup
46+
47+
Before you can install software, you must [set up](setup.md) the Compute Canada
48+
software stack repositories.
49+
50+
## Choosing an installation method
51+
52+
### Nix or EasyBuild?
53+
54+
The first step to install new software on the setup is to determine whether it
55+
should be installed in Nix or with EasyBuild. Software in Nix is architecture-
56+
and platform-independent. In general, we expect that most software installed in
57+
Nix is software which would usually be provided by the OS. Nix also does not
58+
integrate with modules. This means that if multiple versions are needed to be
59+
swappable by users through modules, it has to be installed with EasyBuild.
60+
61+
The following guidelines should be followed, although exceptions may arise.
62+
63+
1. Is the software performance critical ?
64+
- Yes => [EasyBuild](easybuild.md)
65+
2. Do you expect that multiple versions will be needed, swappable through
66+
modules?
67+
- Yes => [EasyBuild](easybuild.md), or EasyBuild wrapping Nix
68+
- No => [Nix](nix.md)
69+
70+
For examples, you can refer to [this
71+
spreadsheet](https://docs.google.com/spreadsheets/d/1ySykcqUyVbJsNnDsWOdrxRZE0VseEQqNvW74Pf6iL00/edit#gid=0)
72+
(staff CC link) where a lot of software has been identified as belonging to
73+
either Nix or EasyBuild.
74+
75+
### Gentoo or EasyBuild?
76+
77+
The process for Gentoo is a little bit more strict than for Nix. As for Nix,
78+
software in Gentoo is architecture- and platform-independent and software which
79+
would usually be provided by the OS.
80+
81+
The following guidelines should be followed, although exceptions may arise.
82+
83+
1. Is the software performance critical ?
84+
- Yes => [EasyBuild](easybuild.md)
85+
2. Do you expect that multiple versions will be needed, swappable through
86+
modules?
87+
- Yes => [EasyBuild](easybuild.md)
88+
- No => [Gentoo](gentoo.md) => please contact the RSNT team first
89+
90+
The current list of installed (via `emerge`) Gentoo packages is tracked in the
91+
`world` file (`/cvmfs/soft.computecanada.ca/gentoo/2020/var/lib/portage/world`,
92+
which is also copied
93+
[here](https://github.com/ComputeCanada/gentoo-overlay/blob/master/etc/portage/world);
94+
the complete list including dependencies can be obtained using `equery list
95+
"*"`.
96+
97+
### Python packages
98+
99+
With very few exceptions, [installing Python packages](python.md) is done
100+
through wheels rather than modules.
101+
102+
## Installing software
103+
104+
### With EasyBuild
105+
106+
See: [Installing with EasyBuild](easybuild.md)
107+
108+
### With Nix
109+
110+
See: [Installing with Nix](nix.md)
111+
112+
### With Gentoo
113+
114+
See: [Installing with Gentoo](gentoo.md)
115+
116+
### Python packages
117+
118+
See: [Installing Python packages](python.md)
119+
120+
## Deploying software
121+
122+
Centrally-installed software is made available by [deploying it with
123+
CVMFS](cvmfs.md).
124+
125+
## Testing software
126+
127+
Several methods are available to [test software](testing.md) at various steps of
128+
the installation process before it is made available to users.
129+
130+
## Examples
131+
132+
- [ANSYS software installation procedure](examples/ansys.md)

‎doc/cvmfs.md

Lines changed: 236 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,236 @@
1+
# Deploying software with CVMFS
2+
3+
**Home:** [Software management](INDEX.md)
4+
5+
### Contents
6+
7+
- [Introduction](#introduction)
8+
- [Development vs production repositories](#development-vs-production-repositories)
9+
- [How to deploy software on CVMFS](#how-to-deploy-software-on-cvmfs)
10+
- [Example](#example)
11+
- [Advanced options to the `rsnt-sync` script](#advanced-options-to-the-rsnt-sync-script)
12+
- [Examples of `rsnt-sync` commands](#examples-of-rsnt-sync-commands)
13+
14+
## Introduction
15+
16+
This page covers only essential information about using CVMFS with our
17+
infrastructure and software stack. For complete CVMFS documentation, see the
18+
[official website](https://cvmfs.readthedocs.io/en/stable/).
19+
20+
There are three repositories to which we have access:
21+
22+
- **dev:** `/cvmfs/soft-dev.computecanada.ca`
23+
- **prod:** `/cvmfs/soft.computecanada.ca`
24+
- **restricted:** `/cvmfs/restricted.computecanada.ca`
25+
26+
You can use the dev version to push files and see that synchronization happens
27+
correctly. However, pushing installed software to the dev repository will not
28+
work transparently, because the installed software is not relocatable.
29+
30+
## Development vs production repositories
31+
32+
A new piece of software must always go first to the development repository
33+
before going to the production repository. Synchronization always happens:
34+
35+
- From the build node to the development repository
36+
- From the development repository to the production repository
37+
38+
The idea is that you first test your newly deployed software on the
39+
`cvmfs-client-dev.computecanada.ca` client before deploying it to the production
40+
repository. This is to ensure that you do not forget to synchronize some files,
41+
and to ensure that the piece of software actually works on a different
42+
environment than the build node.
43+
44+
## How to deploy software on CVMFS
45+
46+
**See also:** [Deploying restricted
47+
software](easybuild.md#deploying-posix-group-restricted-software-with-cvmfs)
48+
49+
Switch to the user `libuser`:
50+
51+
```
52+
sudo su - libuser
53+
```
54+
55+
As `libuser`, start a transaction:
56+
57+
```
58+
sudo /etc/rsnt/start_transaction <dev|prod|restricted>
59+
```
60+
61+
Then synchronize the files needed. You can sync manually, but to avoid most
62+
mistakes, a wrapper script is provided to make it easier. Typically, you will
63+
want to run one of the commands below, with the most common at the top:
64+
65+
```
66+
# Note: <software name> must be lowercase module name, regardless of what the recipe is named ...
67+
/etc/rsnt/rsnt-sync --what easybuild --software <software name> --version <software version>
68+
/etc/rsnt/rsnt-sync --what nix
69+
/etc/rsnt/rsnt-sync --what custom --path <path>
70+
/etc/rsnt/rsnt-sync --what config
71+
/etc/rsnt/rsnt-sync --what easybuild-recipes
72+
```
73+
74+
Note: the option `custom-python` exists as a backward compatibility to
75+
synchronize all python wheels.
76+
77+
For more advanced options, see the section below.
78+
79+
**If you make a mistake, don’t panic, simply run:**
80+
81+
```
82+
sudo /etc/rsnt/abort_transaction <dev|prod|restricted>
83+
exit
84+
```
85+
86+
This will revert the changes and you can start over.
87+
88+
Check that the changes rsynced match what you expect:
89+
90+
```
91+
ls /stratum0/cvmfs/soft.computecanada.ca/<sub path>
92+
```
93+
94+
or:
95+
96+
```
97+
ls /stratum0/cvmfs/soft-dev.computecanada.ca/<sub path>
98+
```
99+
100+
or:
101+
102+
```
103+
ls /stratum0/cvmfs/restricted.computecanada.ca/<sub path>
104+
```
105+
106+
Once you are confident that the changes in
107+
`/stratum0/cvmfs/{soft-dev,soft,restricted}.computecanada.ca/` represent what
108+
needed to be done, run:
109+
110+
```
111+
sudo /etc/rsnt/publish_transaction <dev|prod|restricted>
112+
exit
113+
```
114+
115+
Note that changes will propagate to the clusters within 30 minutes.
116+
117+
### Example
118+
119+
Here is a concrete example how to deploy deal.II 8.4.2:
120+
121+
```
122+
sudo su - libuser
123+
124+
sudo /etc/rsnt/start_transaction dev
125+
/etc/rsnt/rsnt-sync --what easybuild --software dealii --version 8.4.2
126+
sudo /etc/rsnt/publish_transaction dev
127+
128+
sudo /etc/rsnt/start_transaction prod
129+
/etc/rsnt/rsnt-sync --what easybuild --software dealii --version 8.4.2
130+
sudo /etc/rsnt/publish_transaction prod
131+
```
132+
133+
## Advanced options to the `rsnt-sync` script
134+
135+
The `rsnt-sync` script accepts many options. The most current information about
136+
them is obtained by running:
137+
138+
```
139+
/etc/rsnt/rsnt-sync --help
140+
```
141+
142+
As of Sept. 6th, 2017, the options are:
143+
144+
```
145+
/etc/rsnt/rsnt-sync --what <nix|config|easybuild|custom> \
146+
[--software <software name>] \
147+
[--version <software version>] \
148+
[--architecture <architecture>] \
149+
[--root_path <software root path>] \
150+
[--modules-only] [--software-only] [--no-size-only] \
151+
[--repo <dev|prod>] \
152+
[--path <source path>]
153+
```
154+
155+
Most of these options are useful only for synchronizing something installed via
156+
EasyBuild. The following are designed to be used with `--software <software
157+
name>`:
158+
159+
- `--version <software version>`: will synchronize only a specific version
160+
- `--architecture <architecture>`: will synchronize only for a specific
161+
architecture
162+
- `--root_path <software root path>`: will synchronize only software installed
163+
within a specific root path below the `easybuild/{software,modules}` folders.
164+
- `--module-only`: will skip synchronizing the `software` folder
165+
- `--software-only`: will skip synchronizing the `modules` folder
166+
167+
The option `--repo` allows you to specify a repository to synchronize to.
168+
Typically, this is automatically detected (provided that you have a transaction
169+
opened only on one repository).
170+
171+
The option `--path` allows you to synchronize a specific path only. The path
172+
specified is relative to the repository (i.e. relative to
173+
`/cvmfs/soft.computecanada.ca/easybuild`) and should NOT include a trailing
174+
slash.
175+
176+
## Examples of `rsnt-sync` commands
177+
178+
- To synchronize all versions of GROMACS, for all architectures:
179+
180+
```
181+
/etc/rsnt/rsnt-sync --what easybuild --software gromacs
182+
```
183+
184+
- To synchronize all builds of GROMACS 5.1.4 for all architectures:
185+
186+
```
187+
/etc/rsnt/rsnt-sync --what easybuild --software gromacs --version 5.1.4
188+
```
189+
190+
- To synchronize all builds of GROMACS 5.1.4 for architecture `avx`:
191+
192+
```
193+
/etc/rsnt/rsnt-sync --what easybuild --software gromacs --version 5.1.4 --architecture avx
194+
```
195+
196+
- To synchronize all builds of GROMACS 5.1.4 for architecture `avx2` under the
197+
`/cvmfs/soft.computecanada.ca/easybuild/{software,modules}/MPI/intel2016.4/openmpi2.0.2`
198+
path:
199+
200+
```
201+
/etc/rsnt/rsnt-sync --what easybuild --software gromacs --version 5.1.4 --architecture avx2 --root_path MPI/intel2016.4/openmpi2.0.2
202+
```
203+
204+
- To synchronize the path
205+
`/cvmfs/soft.computecanada.ca/easybuild/software/2017/avx2/MPI/intel2016.4/openmpi2.0/boost`,
206+
you would run:
207+
208+
```
209+
/etc/rsnt/rsnt-sync --what easybuild --path software/2017/avx2/MPI/intel2016.4/openmpi2.0/boost
210+
```
211+
212+
You would then need to synchronize the corresponding path in the `modules`
213+
subdirectory to synchronize the module as well as the software:
214+
215+
```
216+
/etc/rsnt/rsnt-sync --what easybuild --path modules/2017/avx2/MPI/intel2016.4/openmpi2.0/boost
217+
```
218+
219+
- To synchronize some configuration files (in the subdirectory
220+
`soft.computecanada.ca/config`), use:
221+
222+
```
223+
/etc/rsnt/rsnt-sync --what config
224+
```
225+
226+
- To synchronize any Python wheels, use:
227+
228+
```
229+
/etc/rsnt/rsnt-sync --what custom --path python/wheelhouse
230+
```
231+
232+
- To synchronize specific Python wheels, use:
233+
234+
```
235+
/etc/rsnt/rsnt-sync --what custom --path python/wheelhouse --software numpy --version 1.18
236+
```

0 commit comments

Comments
 (0)