Defining Clusters

Clusters are defined in the clusters.yaml file, located in the definitions directory (--def-dir). A cluster is defined as follows

cluster_name:
    key1: val1

where cluster_name is later referenced to in the RunDef.

Configuration keys

Keys in boldface are required.

Key	Description	Possible values
type	Cluster system	local, slurm
cores	Number of cores requested per job	int
memory	Memory requested per job	2GB, 1024MB, etc.
num_slots	Number of allocable units (currently: number of runs per job)	int
max_jobs	Maximum number of jobs queued (default: math.inf)	int

JobQueues are defined via dask-jobqueue, and further configuration may be necessary:

• Configuration for slurm
  • Usually, you want to define queue and project.
  • The keys processes, n_workers and python should not be used.