Skip to content

Commit

Permalink
docs: add further details about IaC, prereqs, intro
Browse files Browse the repository at this point in the history
  • Loading branch information
ejseqera committed Dec 14, 2023
1 parent e80e80f commit 2cd63c3
Showing 1 changed file with 53 additions and 20 deletions.
73 changes: 53 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,34 +2,56 @@

## Infrastructure as code

### Using Seqera Platform CLI
Infrastructure as Code (IaC) provides the ability to provision and manage infrastructure through configuration files. Having IaC enables automation in set up, cconsistency and standardization in infrastructure, and documentation of how resources are configured.

This guide describes how to create resources on Seqera Platform through IaC using two methods: (1) Seqera Platform CLI, and (2) seqerakit.

### 1. Using Seqera Platform CLI

Most [Seqera Platform](https://seqera.io/platform/) entities such as Pipelines, Compute Environments and Datasets can be exported in JSON format via the [Seqera Platform CLI](https://github.com/seqeralabs/tower-cli#nextflow-tower-cli). This is very useful for creating infrastructure as code to store the exact configuration options used to create these entities, and to share and track changes over time.

This repository contains JSON representation of the Seqera Platform entitites generated by the Platform CLI that were used to create the `seqeralabs/showcase` Workspace:

- [Compute Environments](compute-envs)
- [Pipelines](pipelines)
- [Datasets](datasets)
- [Compute Environments](compute-envs) for AWS, Azure, Google Cloud, and SLURM
- [Pipelines](pipelines) from a select set of [nf-core pipelines](https://nf-co.re/pipelines)
- [Datasets](datasets) for running the pipelines with minimal test data

#### Scripts
#### Running the Bash scripts

The bash scripts required to programatically import and export these entities from Seqera Platform via the CLI have also been provided in the [`scripts`](scripts) directory.

See Seqera Platform CLI [usage docs](https://github.com/seqeralabs/tower-cli/blob/master/USAGE.md#usage-examples) for more examples.
To create the Compute Environments:

#### Prerequisites
```bash
bash scripts/bash/tw_computeenvs_import.sh
```

To add the Datasets:

```bash
bash scripts/bash/tw_datasets_add.sh
```

To add the Pipelines:

```bash
bash scripts/bash/tw_pipelines_import.sh
```

See Seqera Platform CLI [usage docs](https://github.com/seqeralabs/tower-cli/blob/master/USAGE.md#usage-examples) for more examples of interacting with the Platform.

1. [Seqera Platform CLI](https://github.com/seqeralabs/tower-cli#1-installation)
2. [`jq`](https://stedolan.github.io/jq/)
#### Software Prerequisites

1. [Seqera Platform CLI](https://github.com/seqeralabs/tower-cli#1-installation) installed
2. [`jq`](https://stedolan.github.io/jq/) installed

#### Assumptions

- Seqera Platform CLI has been installed and `tw info` has been configured
- `TOWER_API_ENDPOINT` and `TOWER_ACCESS_TOKEN` have been injected into the executing environment
- Credentials have already been set-up in Seqera Platform
- [Seqera Platform CLI](https://github.com/seqeralabs/tower-cli#1-installation) installed and `tw info` has been configured
- `TOWER_API_ENDPOINT` and `TOWER_ACCESS_TOKEN` have been injected into the executing environment (see the [Seqera Platform CLI documentation](https://github.com/seqeralabs/tower-cli/tree/master?tab=readme-ov-file#2-configuration) on configuring these)
- Credentials for AWS, Azure, and Google have already been set-up in Seqera Platform

### Using `seqerakit`
### 2. Using `seqerakit`

The [Seqera Platform](https://https://seqera.io/platform/) entities represented above in JSON can also be represented as YAML configuration files. These configuration YAMLs can be used by `seqerakit`, a Python package wrapping the Seqera Platform CLI, to define and automate how entities should be created.

Expand All @@ -43,29 +65,40 @@ This repository also contains YAML configuration files defining creation of Seqe

To use `seqerakit`, you will need to have:

- The [Seqera Platform CLI](https://github.com/seqeralabs/tower-cli#1-installation) installed
- `TOWER_API_ENDPOINT` and `TOWER_ACCESS_TOKEN` have been injected into the executing environment
- The package installed via `pip install seqerakit` or `conda install seqerakit`.
- The [Seqera Platform CLI](https://github.com/seqeralabs/tower-cli#1-installation) installed and `tw info` has been configured as described above
- `TOWER_API_ENDPOINT` and `TOWER_ACCESS_TOKEN` environment variables set
Credentials for AWS, Azure, and Google have already been set-up in Seqera Platform
- The package installed via:

```bash
conda install seqerakit
```

Or:

```bash
pip install seqerakit
```

Please refer to the package's [installation guide](docs/installation.md) for more details on the required dependencies.

#### Running `seqerakit`

Similar to executing the scripts above to import the entities defined in the JSONs files to Seqera Platform, the [YAML files](#using-seqerakit) provided in this repository can be used create `seqeralabs/showcase` entities on the Platform. For example:
Similar to executing the scripts above to import and create Pipelines, Compute Environments, and Datasets that are defined in JSON onto Seqera Platform, the [YAML files](#using-seqerakit) provided in this repository can also be used create `seqeralabs/showcase` entities on the Platform. For example:

To add the compute environments:
To create the Compute Environments:

```bash
seqerakit seqerakit/compute-envs/*
```

To add the datasets:
To add the Datasets:

```bash
seqerakit seqerakit/datasets/*
```

To add the pipelines:
To add the Pipelines:

```bash
seqerakit seqerakit/pipelines/*
Expand Down

0 comments on commit 2cd63c3

Please sign in to comment.