Skip to content

Refactor/containers #118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 7, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/environment/conda.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@

<!--intro-start-->

Packages provided through the standard channels of [modules](modules.md) and [containers](../../containers/) are optimized for the ULHPC clusters to ensure their performance and stability. However, many packages where performance is not critical and are used by few users are not provided through the standard channels. These packages can still be installed locally by the users through an environment management system such as Conda.
Packages provided through the standard channels of [modules](modules.md) and [containers](../containers/) are optimized for the ULHPC clusters to ensure their performance and stability. However, many packages where performance is not critical and are used by few users are not provided through the standard channels. These packages can still be installed locally by the users through an environment management system such as Conda.

!!! warning "Contact the ULHPC before installing any software with Conda"
Prefer binaries provided through [modules](modules.md) or [containers](../../containers/). Conda installs generic binaries that may be suboptimal for the configuration of the ULHPC clusters. Furthermore, installing packages locally with Conda consumes quotas in your or your project's account in terms of [storage space and number of files](../../filesystems/quotas/#current-usage).
Prefer binaries provided through [modules](modules.md) or [containers](../containers/). Conda installs generic binaries that may be suboptimal for the configuration of the ULHPC clusters. Furthermore, installing packages locally with Conda consumes quotas in your or your project's account in terms of [storage space and number of files](/filesystems/quotas/#current-usage).

Contact the ULHPC High Level Support Team in the [service portal](https://service.uni.lu/sp?id=index) [Home > Research > HPC > Software environment > Request expertise] to discuss possible options before installing any software.

Expand Down Expand Up @@ -130,7 +130,7 @@ Micromamba supports almost all the subcommands of Conda. For more details see th

### Using environments in submission scripts

Since all computationally heavy operations must be performed in compute nodes, Conda environments are also used in jobs submitted to the [queuing system](../slurm/index.md). Returning to the R example, a submission script running a single core R job can use the `R-project_name` environment as follows:
Since all computationally heavy operations must be performed in compute nodes, Conda environments are also used in jobs submitted to the [queuing system](/slurm/). Returning to the R example, a submission script running a single core R job can use the `R-project_name` environment as follows:
```
#SBATCH --job-name R-test-job
#SBATCH --nodes 1
Expand All @@ -156,11 +156,11 @@ micromamba deactivate

_Useful scripting resources_

- [Formatting submission scripts for R (and other systems)](../slurm/launchers.md#serial-task-script-launcher)
- [Formatting submission scripts for R (and other systems)](/slurm/launchers.md#serial-task-script-launcher)

### Cleaning up package data

The Conda environment managers download and store a sizable amount of data to provided packages to the various environments. Even though the package data are shared between the various environments, they still consume space in your or your project's account. There are [limits in the storage space and number of files](../../filesystems/quotas/#current-usage) that are available to projects and users in the cluster. Since Conda packages are self managed, **you need to clean unused data yourself**.
The Conda environment managers download and store a sizable amount of data to provided packages to the various environments. Even though the package data are shared between the various environments, they still consume space in your or your project's account. There are [limits in the storage space and number of files](/filesystems/quotas/#current-usage) that are available to projects and users in the cluster. Since Conda packages are self managed, **you need to clean unused data yourself**.

There are two main sources of unused data, the compressed archives of the packages that Conda stores in its cache when downloading a package, and the data of removed packages. All unused data in Micromoamba can be removed with the command
```bash
Expand Down Expand Up @@ -204,7 +204,7 @@ Using an external packaging tool is possible because of the method that Conda us
- packages installed by the package tool are specific to an environment and cannot be shared as with Conda, since components are installed directly and not with links.

!!! important "Prefer Conda over external package managers"
Installing the same package in multiple environments with an external package tool consumes quotas in terms of [storage space and number of files](../../filesystems/quotas/#current-usage), so prefer Conda when possible. This is particularly important for the `inode` limit, since some packages install a large number of files, and the hard links used by Conda do not consume inodes or [disk space](https://stackoverflow.com/questions/55566419/why-are-packages-installed-rather-than-just-linked-to-a-specific-environment).
Installing the same package in multiple environments with an external package tool consumes quotas in terms of [storage space and number of files](/filesystems/quotas/#current-usage), so prefer Conda when possible. This is particularly important for the `inode` limit, since some packages install a large number of files, and the hard links used by Conda do not consume inodes or [disk space](https://stackoverflow.com/questions/55566419/why-are-packages-installed-rather-than-just-linked-to-a-specific-environment).

#### Pip

Expand Down Expand Up @@ -309,7 +309,7 @@ ${HOME}/micromamba/envs/julia/share
if the default location for the Micromamba environment directory is used.

??? info "Advanced management of package data"
Julia packages will consume [storage and number of files quota](../../filesystems/quotas/#current-usage). Pkg uses automatic garbage collection to cleanup packages that are no longer is use. In general you don't need to manage then package data, simply remove the package and its data will be deleted automatically after some time. However, when you exceed your quota you need to delete files immediately.
Julia packages will consume [storage and number of files quota](/filesystems/quotas/#current-usage). Pkg uses automatic garbage collection to cleanup packages that are no longer is use. In general you don't need to manage then package data, simply remove the package and its data will be deleted automatically after some time. However, when you exceed your quota you need to delete files immediately.

The _immediate removal_ of the data of uninstalled packages can be forced with the command:
```julia
Expand Down
16 changes: 8 additions & 8 deletions docs/containers/index.md → docs/environment/containers.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Many applications and libraries can also be used through container systems. The

![](https://apptainer.org/docs/user/latest/_static/logo.png){: style="width:200px;float: right;"}

The UL HPC supports [Apptainer containers](https://apptainer.org/docs/user/main/introduction.html). Apptainer is an open source container platform designed to be simple, fast, and secure. Apptainer is optimized for Enterprise Performance Computing (EPC){[^98] and High Performance Computing (HPC) workloads, allowing users to run containers in a trusted way.
The UL HPC supports [Apptainer containers](https://apptainer.org/docs/user/main/introduction.html). Apptainer is an open source container platform designed to be simple, fast, and secure. Apptainer is optimized for Enterprise Performance Computing (EPC)[^98] and High Performance Computing (HPC) workloads, allowing users to run containers in a trusted way.

[^98]: Typical examples of Enterprise Performance Computing workloads are deep learning inference and other machine learning workloads.

Expand Down Expand Up @@ -119,7 +119,8 @@ apptainer run --nv cuda_samples.sif

The latest command should print:
!!! note "Output"
<pre>CUDA Device Query (Runtime API) version (CUDART static linking)
```
CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 1 CUDA Capable device(s)

Expand Down Expand Up @@ -157,12 +158,11 @@ The latest command should print:
Supports MultiDevice Co-op Kernel Launch: Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 30 / 0
Compute Mode:
&lt; Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) &gt;
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 10.2, CUDA Runtime Version = 10.1, NumDevs = 1
Result = PASS
</pre>

```

### MPI and Apptainer containers

Expand Down Expand Up @@ -219,10 +219,10 @@ Once the container image is ready, you can use it for example inside the followi
module load tools/Apptainer
srun apptainer run mpi_osu.sif
```
The content of the output file:
The content of the output:

!!! note "Output"
```
!!! note "Job output"
```
# OSU MPI Bandwidth Test v5.3.2
# Size Bandwidth (MB/s)
1 0.35
Expand Down
10 changes: 5 additions & 5 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,16 +182,16 @@ of ULHPC.
The ULHPC Team supplies a large variety of HPC utilities, scientific applications and programming libraries to its user community.
The user software environment is generated using [Easybuild](https://easybuild.readthedocs.io) (EB) and is made available as environment modules through [LMod](https://lmod.readthedocs.io/). <!-- We have developped a specific tool named [RESIF](software/resif.md) -->

* [ULHPC Modules Environment](environment/modules.md)
* [ULHPC Supported Software List](./software/index.md).
- Available [modules](./environment/modules.md) are reachable **from the compute nodes only** via `module avail`
* [ULHPC Easybuild Configuration](environment/easybuild.md)
* [ULHPC Modules Environment](/environment/modules)
* [ULHPC Supported Software List](/software/).
- Available [modules](/environment/modules) are reachable **from the compute nodes only** via `module avail`
* [ULHPC Easybuild Configuration](/environment/easybuild)

<!-- * [RESIF Architecture-at-a-glance](software/resif.md) -->
<!-- * [Software Sets, compilers and toolchains at ULHPC](software/swsets.md) -->
<!-- * [Compiling/building software](development/software/index.md) -->
<!-- - [Building missing software with Easybuild](development/software/easybuild.md) -->
* [Running Containers](containers/index.md)
* [Running Containers](/environment/containers)
<!-- * [Contributing to the ULHPC User Software](development/software/contributing.md) -->

!!! question "Software building support"
Expand Down
2 changes: 1 addition & 1 deletion docs/software/physics/wrf.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ In our systems we provide repackaged containers developed by the Central Researc

## Available versions in the UL HPC systems

In the UL HPC system we support [Singularity containers](/containers). The University of Manchester containers have been repackaged as Singularity containers for use in our systems. The Singularity containers are:
In the UL HPC system we support [Apptainer containers](/environment/containers). The University of Manchester containers have been repackaged as Singularity containers for use in our systems. The Singularity containers are:

- WRF-WPS version 4.3.3: `/work/projects/singularity/ulhpc/wrf-wps-4.3.3.sif`
- WRF-Chem version 4.3.3: `/work/projects/singularity/ulhpc/wrf-chem-4.3.3.sif`
Expand Down
13 changes: 5 additions & 8 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,10 +62,11 @@ nav:
# - Multi-Factor Authentication: 'connect/mfa.md'
##############
- Environment:
- Overview: 'environment/index.md'
- Modules: 'environment/modules.md'
- Easybuild: 'environment/easybuild.md'
- Conda: 'environment/conda.md'
- Overview: 'environment/index.md'
- Modules: 'environment/modules.md'
- Easybuild: 'environment/easybuild.md'
- Containers: 'environment/containers.md'
- Conda: 'environment/conda.md'
###########
- Policies:
- Acceptable Use Policy (AUP): 'policies/aup.md'
Expand Down Expand Up @@ -141,7 +142,6 @@ nav:
# - Contributing: 'software/contributing.md'
# - BigData Analytics: 'software/bigdata/index.md'
# - Bioinformatics/Life Science: 'software/bio/index.md'
- Containers: 'containers/index.md'
- Computer Aided Eng. (CAE):
- FEniCS : 'software/cae/fenics.md'
- ANSYS : 'software/cae/ansys.md'
Expand Down Expand Up @@ -241,9 +241,6 @@ nav:
- Scalasca: 'development/performance-debugging-tools/scalasca.md'
- Valgrind: 'development/performance-debugging-tools/valgrind.md'
- APS: 'development/performance-debugging-tools/aps.md'
- Containers:
- About: 'containers/index.md'
# - Singularity: 'containers/singularity.md'
###############
# - Applications:
# - Overview: 'software/index.md'
Expand Down