Separate operator-managed fields into .status

Author: HappyTetrahedron

docs/modules/SDDs/pages/0032-compile-pipeline.adoc renamed to docs/modules/SDDs/pages/0032-compile-pipeline-gitlab.adoc

@@ -14,6 +14,8 @@ include::partial$meta-info-table.adoc[]
 This describes how we want to implement GitLab CI/CD for Project Syn using a Commodore compile pipeline, which enables automatic compilation of cluster catalogs whenever the corresponding tenant repository is modified.
 
 Furthermore, it explains how want to extend the functionality of Lieutenant to enable it to automatically configure the compile pipeline on tenant repositories.
+
+While we only aim to support GitLab at the moment, the architecture should be sufficiently generic so it can be used for other Git hosts in the future.
 ====
 
 == Motivation
@@ -105,20 +107,25 @@ spec:
 
 === Tenant CRD
 
-We add a new field to the `Tenant` CRD, under the `.spec` key, called `compilePipeline`.
+We add new fields to the `Tenant` CRD: `spec.compilePipeline` and `status.compilePipeline`
 
-The `compilePipeline` field contains configuration pertaining to the automatic setup of the compile pipeline on the tenant repository.
+The `spec.compilePipeline` field contains configuration pertaining to the automatic setup of the compile pipeline on the tenant repository.
 It is optional.
-Absence of the field disables automatic setup and management of the compile pipeline.
 
-The `compilePipeline` field contains a dict with the following fields:
+The `spec.compilePipeline` field contains a dict with the following fields:
 
-* `clusters`: List of cluster IDs of clusters for which the compile pipeline should be executed.
-  This field is managed by the operator.
+* `enabled`: Boolean field which enables or disables automatic setup of compile pipelines for this tenant (regardless of whether it is enabled on the tenant's clusters).
 * `pipelineFiles`: Dictionary containing file paths as keys, and file contents as values.
   These files will be added to the tenant's `gitRepoTemplate.templateFiles` by the Lieutenant operator.
   This field is optional; if absent, no new template files are added to the `gitRepoTemplate`.
 
+`spec.compilePipeline` is optional. Its absence disables automatic setup of compile pipelines for the tenant, as does setting `spec.compilePipeline.enabled` to `false`.
+
+The `status.compilePipeline` field contains a dict with one field:
+
+* `clusters`: List of cluster IDs of clusters for which the compile pipeline should be executed.
+  This field is managed by the operator.
+
 [source,yaml]
 ----
 apiVersion: syn.tools/v1alpha1
@@ -127,14 +134,16 @@ metadata:
   name: t-my-tenant
 spec:
   compilePipeline:
-    clusters:
-      - c-my-cluster
     pipelineFiles:
       .gitlab-ci.yml: |
         include:
           - project: syn/commodore-compile-pipeline
             ref: master
             file: /.gitlab/commodore-common.yml
+status:
+  compilePipeline:
+    clusters:
+      - c-my-cluster
 ----
 
 === Operator
@@ -151,6 +160,8 @@ The operator will reconcile *GitRepos* as follows:
   The operator also runs a scheduled job which refreshes these tokens when they are close to expiring, or when they no longer exist on the repository host.
 * The content of `.spec.ciVariables` is written to the repository's configuration on the Git host.
   In the case of GitLab, it is written as CI/CD variables.
+  If the content of `.spec.ciVariables` changes, the corresponding configuration on the Git host should be updated.
+  A scheduled job in the ooperator regularly checks for drift between `.spec.ciVariables` and the configuration on the Git host, and updates the latter if necessary.
 
 NOTE: If the GitRepo is of type `unmanaged`, none of these steps will be executed.
 
@@ -159,13 +170,13 @@ The operator will reconcile *Clusters* as follows:
 * When `.spec.enableCompilePipeline` is set to `true`, the tenant's `spec.compilePipeline.clusters` is updated to contain the cluster ID.
 * Similarly, when the field is set to `false` or missing, the tenant's `spec.compilePipeline.clusters` is updated to not contain the cluster ID.
 
-The operator will reconcile *Tenants* as follows:
+The operator will reconcile *Tenants* as follows, if and only if `spec.compilePipeline.enabled` is set to `true`:
 
-* When `.spec.compilePipeline` exists and isn't empty, the following entries are added to the tenant repository GitRepo's `.spec.ciVariables`:
+* The following entries are added to the tenant repository GitRepo's `.spec.ciVariables`:
 ** `COMMODORE_API_URL`, containing the URL at which the Lieutenant API can be accessed.
 ** `COMMODORE_API_TOKEN`, containing a reference to the secret which contains the tenant's access token for the Lieutenant API.
-** `CLUSTERS`, containing a space-separated list of cluster IDs taken directly from `.spec.compilePipeline.clusters`.
-* For each entry in `.spec.compilePipeline.clusters`, another entry is added to the tenant repository GitRepo's `spec.ciVariabes`.
+** `CLUSTERS`, containing a space-separated list of cluster IDs taken directly from `.status.compilePipeline.clusters`.
+* For each entry in `.status.compilePipeline.clusters`, another entry is added to the tenant repository GitRepo's `spec.ciVariabes`.
   The key is `ACCESS_TOKEN_CLUSTERNAME`, where `CLUSTERNAME` is the ID of a specific cluster, with `-` replaced by `_`.
   The value is a reference to the secret containing the access token to access that cluster's catalog repository, taken from the secret specified in the catalog GitRepo configuration under `.spec.accessTokenSecretName`.
 * For each entry in `.spec.compilePipeline.pipelineFiles`, a new corresponding entry is added to the tenant's `.spec.gitRepoTemplate.templateFiles`.
@@ -178,12 +189,12 @@ The implementation takes care to gracefully skip the CI steps if not on GitLab.
 We leave the implementation extensible enough to add further Git management tools and CI systems without needing breaking changes.
 
 Existing compile pipeline configuration::
-If a setup already includes a bunch of tenant repositories with manually configured CI/CD, some care has to be taken to ensure the new implementation can "adopt" this configuration.
+If a setup already includes a bunch of tenant repositories with manually configured CI/CD, in the general case, the new implementation can "adopt" this configuration.
 +
-In particular, these repositories would already have a working `.gitlab-ci.yml` that probably can be left as-is.
+In particular, these repositories would already have a working `.gitlab-ci.yml` that probably could be left as-is, but can also be replaced by a lieutenant-managed one.
 +
 Any existing manually created Project Access Tokens will be superseded by new auto-generated ones.
-This will lead to a bunch of now-unused tokens needing to be cleaned up.
+This will lead to a bunch of now-unused tokens needing to be cleaned up, but should otherwise work without requiring extra effort.
 
 External Catalog Repositories::
 There may be cases where the catalog repositories are not hosted on the same repository host as the tenant repository, in which case API access for the purpose of creating Project Access Tokens is unavailable.

docs/modules/SDDs/pages/index.adoc

@@ -26,4 +26,4 @@ The all are using the xref:sdd-template.adoc[SDD Template].
 * xref:0028-reusable-config-packages.adoc[0028 - Reusable Commodore Component Configuration Packages]
 * xref:0030-argocd-multitenancy.adoc[0030 - Project Syn ArgoCD Multi-Tenant Support]
 * xref:0031-component-version-tracking.adoc[0031 - Central Component Version tracking]
-* xref:0032-compile-pipeline.adoc[0032 - Commodore Compile Pipeline for GitLab]
+* xref:0032-compile-pipeline-gitlab.adoc[0032 - Commodore Compile Pipeline for GitLab]