Singularity Definition File That May Save Others a Lot of Time

[radimspetlik]

I suggest including the singularity.def file below. It produces a working Singularity/Apptainer image that can be used to run your method seamlessly.

Bootstrap:docker
From:nvidia/cuda:11.8.0-devel-ubuntu22.04

%environment
    # Make conda available everywhere
    export PATH=/opt/conda/envs/sugar/bin:/opt/conda/bin:/usr/local/cuda/bin:$PATH
    export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
    export SUGAR_HOME=/opt/sugar
    export PYTHONDONTWRITEBYTECODE=1
    export PIP_NO_CACHE_DIR=1
    # Helpful for some torch/cuda builds
    export FORCE_CUDA=1

    export TORCH_CUDA_ARCH_LIST="7.0;8.0+PTX"
    export CUDA_HOME=/usr/local/cuda

    # Only auto-activate when running under bash (interactive shells)
    if [ -n "${BASH_VERSION:-}" ]; then
        . /opt/conda/etc/profile.d/conda.sh
        conda activate sugar >/dev/null 2>&1 || true
    fi

%files
    . /opt/sugar

%post
    export DEBIAN_FRONTEND=noninteractive
    apt-get update
    apt-get install -y --no-install-recommends \
        git build-essential cmake ninja-build pkg-config \
        libgl1-mesa-dev libegl1 libxext6 libx11-6 libxi6 libxrender1 libsm6 libxrandr2 libxxf86vm1 \
        ca-certificates curl wget && \
        rm -rf /var/lib/apt/lists/*

    # Install Anaconda
    curl -O https://repo.anaconda.com/archive/Anaconda3-2025.06-0-Linux-x86_64.sh
    bash ./Anaconda3-2025.06-0-Linux-x86_64.sh -b -p /opt/conda
    export PATH=/opt/conda/bin:$PATH
    export CONDA_PLUGINS_AUTO_ACCEPT_TOS=yes
    conda init --all
    conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main
    conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r

    # Keep conda current; ensure pip exists
    conda update -y -n base -c defaults conda
    conda install -y -n base pip
    conda clean -afy

    # Make CUDA & arch list visible to the build that happens next
    export CUDA_HOME=/usr/local/cuda
    export TORCH_CUDA_ARCH_LIST="7.0;8.0+PTX"
    echo "Using TORCH_CUDA_ARCH_LIST=${TORCH_CUDA_ARCH_LIST}"

    # Enter project directory
    cd /opt/sugar

    # If repo uses submodules, bring them in (no-op if none)
    if [ -d .git ] && [ -f .gitmodules ]; then
        git submodule update --init --recursive
    fi

    python install.py

    # Make sure the env is activated automatically for container shells/exec
    echo '. /opt/conda/etc/profile.d/conda.sh' >> /environment
    echo 'conda activate sugar >/dev/null 2>&1 || true' >> /environment

    # Prepare common bind mount points
    mkdir -p /scratch /apps

    conda clean -afy || true

%runscript
    #!/bin/bash
    set -euo pipefail

    cd /opt/sugar
    # Default behavior: run whatever the user passes, inside the env
    exec "$@"