diff --git a/tools/anthosvmware-ansible-module/.gitignore b/tools/anthosvmware-ansible-module/.gitignore new file mode 100644 index 0000000000..39275d1763 --- /dev/null +++ b/tools/anthosvmware-ansible-module/.gitignore @@ -0,0 +1,28 @@ +# logs +*.log + +# Directories +.config/ +.terraform/ +.vagrant/ +inventory/vmlab +inventory/my-site + +# SSH Keys +*.pem + +# Backup files +*.bak + +# Ignore Mac .DS_Store files +.DS_Store + +# Ignore vscode files +.vscode/ + +header +headerj2 +footer +search-and-replace.sh +add-header.sh +roles/TEST_CASES.md diff --git a/tools/anthosvmware-ansible-module/OWNERS.md b/tools/anthosvmware-ansible-module/OWNERS.md new file mode 100644 index 0000000000..d99af08cf7 --- /dev/null +++ b/tools/anthosvmware-ansible-module/OWNERS.md @@ -0,0 +1,11 @@ +# Owners + +- Ravi Nalluri [ravi-nal] + +- Cat Chu [cathchu] + +- Alain Baxter [alain-baxter] + +- Marek Anderson [manderson-it] + +- Karim Elatov [elatovg] diff --git a/tools/anthosvmware-ansible-module/README.md b/tools/anthosvmware-ansible-module/README.md new file mode 100644 index 0000000000..582925386b --- /dev/null +++ b/tools/anthosvmware-ansible-module/README.md @@ -0,0 +1,941 @@ +# Ansible Automation for Anthos on VMware + +Automated install of Anthos clusters with Ansible, including +templating YAML files with Ansible for Anthos on VMware. + +## About this repository + +This git repository uses GitHub Actions (see `.github/workflows/`). + +- [https://github.com/marketplace/actions/ansible-lint](https://github.com/marketplace/actions/ansible-lint) +- [https://github.com/actionshub/markdownlint](https://github.com/actionshub/markdownlint) + +### Markdown linting + +- [https://github.com/actionshub/markdownlint](https://github.com/actionshub/markdownlint) +- [Markdown linting rules](https://github.com/markdownlint/markdownlint/blob/master/docs/RULES.md) + +### Ansible linting + +- [https://github.com/ansible/ansible-lint-action](https://github.com/ansible/ansible-lint-action) + +## Assumptions + +Consider these assumptions when you wonder how certain tasks are implemented or missing. + +- Ansible is run from the command-line +- Ansible `>= 2.9` +- Greenfield deployment: No Anthos resources exist when applying the automation. +- Anthos preview features are not considered in this automation but can be added easily (`vim roles/*cluster/templates/{admin,user}-cluster.yaml.j2`) +- If `secretsEncryption` is enabled for the admin cluster, it will be enabled for user clusters; + however, the keyversions are tracked in separate Ansible vars for rotation purposes +- One admin workstation per admin cluster + +## Prerequisites + +- Ansible +- Authenticate `gcloud` on jumphost with `gcloud auth login` so that Ansible can run the `gsutil` command on the jumphost +- vSphere: Create VM-Folder for Anthos VMs +- vSphere: Create folder on vSAN for Anthos Admin Cluster (if using vSAN). + Consider using value of Ansible variable `{{ ac_name }}` as the vSAN folder name to be consistent. + +Last tested with the following versions. + +```shell +ansible --version +ansible [core 2.11.4] +[...] + python version = 3.9.7 (default, Oct 13 2021, 06:45:31) [Clang 13.0.0 (clang-1300.0.29.3)] + jinja version = 3.0.1 + libyaml = True +``` + +### Prerequisites for Google Service Accounts + +Google Service Accounts ("GSA") are required for the automated cluster and Anthos component lifecycle operations performed by Ansible: +- Create GCP service accounts (SA) as per documentation [Creating Service Accounts](https://cloud.google.com/anthos/clusters/docs/on-prem/how-to/service-accounts) +- Download each GCP SA JSON key file to your machine/jumphost + +The below sections highlight the GSAs used in this automation code and their necessary permissions. + +#### Component Access GSA (Required) + +The Component Access service account downloads cluster artifacts from Google Container Registry (ex: images). It requires the below permissions in the target Google Cloud project: + +* `serviceusage.serviceUsageViewer` +* `iam.roleViewer` +* `iam.serviceAccountViewer` + +Set the JSON file key path with `component_access_gcpsa_path` within the target cluster inventories. + +#### Connect Register GSA (Required) + +The Connect Register service account is responsible for registering your admin and user clusters to a fleet in a GCP project. It requires the below permission in the target Google Cloud project: + +* `gkehub.admin` + +Set the JSON file key path with `connect_register_gcpsa_path` within the target cluster inventories. + +#### Logging and Monitoring GSA (Optional) + +The Logging and Monitoring service account can be used to export logs and metrics from your clusters to Google Cloud Logging and Monitoring. It requires the below permissions in the target Google Cloud project: + +* `stackdriver.resourceMetadata.writer` +* `opsconfigmonitoring.resourceMetadata.writer` +* `logging.logWriter` +* `monitoring.metricWriter` +* `monitoring.dashboardEditor` + +Set the JSON file key path with `logging_monitoring_gcpsa_path` within the target cluster inventories. + +#### Audit Logging GSA (Optional) + +The Audit Logging service account can be used to export Kubernetes audit logs from your clusters to Google Cloud Audit logs. It does not require any permissions to be granted to it. + +Set the JSON file key path with `audit_logging_gcpsa_path` within the target cluster inventories. + +#### Anthos Service Mesh Config GSA (Optional) + +The Anthos Service Mesh Config service account is used when installing ASM onto user clusters. It requires the below permissions in the target Google Cloud project: + +* `gkehub.editor` +* `meshconfig.admin` +* `iam.roleViewer` +* `serviceusage.serviceUsageViewer` + +> **Note 1:** This list is a subset of permissions outlined in the [public docs for installing ASM](https://cloud.google.com/service-mesh/docs/installation-permissions), which lists all of the permissions required from cluster preparation for ASM installation to the actual installation. Since much of the platform is already configured by the time of installation, this service account can operate with a more restrictive permissions. +> +>Also notably, this list omits `roles/privateca.admin`, which is a role required for using the [Certificate Authority Service](https://cloud.google.com/certificate-authority-service). The automation leverages IstioCA instead, so this role is not required. + +Set the JSON file key path with `asm_gcpsa_path` within the target cluster inventory. + +For more information about the permissions required for installing Anthos Service Mesh, please refer to the [documentation](https://cloud.google.com/service-mesh/docs/installation-permissions) + +#### Anthos Config Management GSA (Optional) + +The Anthos Config Management service account is used when installing ACM onto user clusters. It requires the below permissions in the target Google Cloud project: + +* `gkehub.editor` +* `serviceusage.serviceUsageViewer` +* `source.reader`, if using Google Cloud Source repositories + +Set the JSON file key path with `acm_gcpsa_path` within the target cluster inventory. + +## Playbook Execution + +This section outlines the execution of the two playbooks for the +installation and uninstallation of the admin workstation. +You must use verbosity level one (`-v`) or higher to see output from `gkeadm` and `gkectl`. + +### Export shell environment variables that are used by Ansible + +Should you not want to store sensitive values directly in the inventory (or any Ansible YAML file for that matter), playbooks can leverage environment variables. Below are the environment variables currently referenced in the automation. Some of them are required, but others are optional depending on your deployment set up. + +```shell +export VMWARE_HOST="" +export VMWARE_USER="username@vcenter.domain" +export VMWARE_PASSWORD="secret" + +# optional - private docker registry +export PRIV_REG_ADDRESS="" +export PRIV_REG_CA_FILE="ca.crt" # If using Ansible Tower, certificate file found on Tower +export PRIV_REG_USERNAME="username" +export PRIV_REG_PASSWORD="AnotherSecret" +export PRIV_REG_EMAIL="username@email.com" + +# optional - private generic artifact registry +export ARTIFACT_HOST="host" +export ARTIFACT_USERNAME="username" +export ARTIFACT_ENC_PASS="encryptedSecret" +export ARTIFACT_API_TOKEN="token" + +# optional - AIS configuration +export AIS_OIDC_CLIENTID="clientID" +export AIS_OIDC_CLIENTSECRET="clientSecret" +export AIS_GCS_BUCKET="uploadBucket" # If uploading the AIS login config file to a Cloud Storage bucket +``` + +These values can then be passed in during playbook runs (as seen below), or just referenced in the inventory as they are now. + +```shell +VMWARE_USER="user@vcenter.local" \ +VMWARE_PASSWORD="secret" \ +ansible-playbook ... -v -e glb_vc_username='$VMWARE_USER' \ + -e glb_vc_password='$VMWARE_PASSWORD' +``` + +For some of these sensitive values, consider using a external service for injecting them. + +### Admin Workstation - Playbook for Installation + +Run the playbook. + +Estimates for the admin workstation: + +- takes around 37 minutes when OVAs need to be downloaded from Google and uploaded to vCenter +- takes around 3 minutes 30 seconds to complete if OVAs are already uploaded + +```shell +# at least one -v is required to see output from gkeadm/gkectl +ansible-playbook -i inventory/site-a/admin.yml playbooks/adminws_install.yml -v + +# example summary +Thursday 26 August 2021 16:58:40 -0400 (0:00:00.029) 0:03:32.864 ******* +=============================================================================== +adminws : [adminws] Create admin workstation -- 199.44s +adminws : [adminws] Templating YAML files -- 6.89s +adminws : [adminws] Templating YAML files -- 6.24s +adminws : [adminws] Sanity Checks -- 0.10s +adminws : include_tasks -- 0.03s +adminws : include_tasks -- 0.03s +adminws : include_tasks -- 0.02s +adminws : [adminws] Note on optional values -- 0.02s +Playbook run took 0 days, 0 hours, 3 minutes, 32 seconds +``` + +### Admin Workstation - Playbook for Uninstallation + +Run the playbook (takes around 5 seconds to complete). + +```shell +# at least one -v is required to see output from gkeadm/gkectl +ansible-playbook -i inventory/site-a/admin.yml playbooks/adminws_uninstall.yml -v + +# example summary +Thursday 26 August 2021 19:41:31 -0400 (0:00:04.887) 0:00:05.051 ******* +=============================================================================== +adminws : [adminws] Delete admin workstation -- 4.89s +adminws : [adminws] Include tasks - uninstall -- 0.04s +adminws : [adminws] Include tasks - asserts -- 0.03s +adminws : [adminws] Include tasks - install -- 0.03s +Playbook run took 0 days, 0 hours, 0 minutes, 5 seconds +``` + +### Admin Workstation - Playbook for Upgrade + +Run the playbook (takes about 20 minutes to complete) after updating the `glb_anthos_version` variable defined in `inventory/site-a/all.yml` to the newer version. + +``` shell +# at least one -v is required to see output from gkeadmin/gkectl +ansible-playbook -i inventory/site-a/admin.yml playbooks/adminws_upgrade.yml -v + +# example +# TODO: add summary output +``` + +### Admin Cluster - Playbook for Installation + +Run the playbook (takes about 39 minutes to complete, potentially more with the use of a private registry). + +```shell +ansible-playbook -i inventory/site-a/admin.yml playbooks/admincluster_install.yml -v + +# example summary +Monday 04 October 2021 11:36:04 -0400 (0:28:03.499) 0:29:50.318 ******** +=============================================================================== +admincluster : [ac] Create cluster 1683.50s +admincluster : [ac] Templating YAML files -- 42.81s +admincluster : [ac] Templating YAML files -- 35.62s +admincluster : [ac] Templating YAML files -- 28.12s +admincluster : [ac] Sanity Checks - 0.10s +admincluster : [ac] Include tasks - install.yml -- 0.04s +admincluster : [ac] Include tasks - asserts.yml -- 0.04s +admincluster : [ac] Note on optional values -- 0.03s +Playbook run took 0 days, 0 hours, 29 minutes, 50 seconds +``` + +Optionally, you can skip some Ansible tasks, if desired. + +```shell +# --skip-tags is optional to skip specific Ansible tasks +ansible-playbook -i inventory/site-a/admin.yml playbooks/admincluster_install.yml \ + -v --skip-tags preflight,prepare +``` + +### Admin Cluster - Playbook for Uninstallation + +Run the playbook (takes less than 10 minutes to complete). + +```shell +ansible-playbook -i inventory/site-a/admin.yml playbooks/admincluster_uninstall.yml -v + +# example summary +Monday 02 May 2022 01:52:21 +0000 (0:00:00.016) 0:00:28.701 ************ +=============================================================================== +admincluster : [ac] Delete Monitoring System Pods -- 14.05s +admincluster : [ac] Unregister Admin Cluster from GKE Hub -- 4.96s +copy_gcp_sa : [copy-gcp-sa] Copy GCP SA JSON key files to admin workstation -- 2.94s +admincluster : [ac] Get Register Service Account Name -- 1.14s +admincluster : [ac] Delete VMs from vCenter -- 1.13s +admincluster : [ac] Switch to Register Service Account Name -- 1.04s +admincluster : [ac] Clean up old SSH host keys from incomplete runs -- 0.60s +admincluster : [ac] Get vSphere VM Names -- 0.47s +cleanup : Find GCP SA JSON files on admin workstation -- 0.46s +admincluster : [ac] Delete Logging System Pods -- 0.43s +cleanup : Clean up GCP SA JSON files on admin workstation -- 0.37s +cleanup : Find *credential*.yaml files on admin workstation -- 0.33s +copy_gcp_sa : [copy-gcp-sa] Create folder for GCP SA JSON key files on admin workstation -- 0.30s +cleanup : Clean up GCP SA JSON files on admin workstation -- 0.04s +cleanup : Clean up GCP SA JSON files on jumphost -- 0.04s +Include role for GCP SA JSON key files -- 0.04s +copy_gcp_sa : [copy-gcp-sa] Copy GCP SA JSON key files to jump host -- 0.03s +admincluster : [ac] Include tasks - uninstall.yml -- 0.03s +Include role to delete sensitive files -- 0.03s +cleanup : Clean up *credential*.yaml on admin workstation -- 0.03s +Playbook run took 0 days, 0 hours, 0 minutes, 28 seconds +``` + +### Admin Cluster - Playbook for Upgrade + +Run the playbook (takes about 30 minutes to complete) after updating the `glb_anthos_version` variable defined in `inventory/site-a/all.yml` to the newer version. + +> **Note:** Upgrades to the admin workstation and user clusters are a prerequisite to upgrading the admin cluster. Upgrades to the admin cluster will otherwise result in a failure. + +```shell +ansible-playbook -i inventory/site-a/admin.yml playbooks/admincluster_upgrade.yml -v + +# example summary +Friday 23 September 2022 17:10:56 -0400 (0:00:00.046) 1:37:25.400 ****** +=============================================================================== +admincluster : [ac] Upgrade admin cluster -- 5712.50s +admincluster : [ac] Upload new bundle to vSphere -- 120.96s +[copy_gcp_sa] Copy GCP SA JSON key files to admin workstation -- 2.96s +admincluster : [ac] Detect if Admin Cluster already exists -- 1.21s +admincluster : [ac] Clean up old SSH host keys from incomplete runs -- 1.20s +[copy_gcp_sa] Templating YAML files - vCenter credentials -- 1.00s +admincluster : [ac] Get target version from Admin Workstation -- 0.87s +[cleanup] Find GCP SA JSON files on admin workstation -- 0.49s +admincluster : [ac] Replace bundlePath in admin cluster config YAML -- 0.45s +[cleanup] Clean up GCP SA JSON file folder on admin workstation -- 0.40s +admincluster : [ac] Detect if Admin Cluster kubeconfig exists -- 0.38s +[cleanup] Clean up *credential*.yaml on admin workstation in admin cluster subfolder -- 0.30s +[cleanup] Find *credential*.yaml files on admin workstation in admin cluster subfolder -- 0.25s +[cleanup] Find *credential*.yaml files on admin workstation -- 0.24s +[copy_gcp_sa] Ensure credentials directory exists -- 0.23s +[copy_gcp_sa] Create folder for GCP SA JSON key files on admin workstation -- 0.22s +[cleanup] Clean up sensitive file on admin workstation -- 0.20s +admincluster : [ac] Sanity Checks -- 0.12s +admincluster : [ac] Include tasks - upgrade.yml -- 0.11s +[cleanup] Clean up sensitive files on jumphost -- 0.10s + +Playbook run took 0 days, 1 hours, 37 minutes, 25 seconds +``` + +### User Cluster - Playbook for Installation + +Run the playbook (takes about 40 minutes to complete). + +```shell +ansible-playbook -i inventory/site-a/usercluster01.yml playbooks/usercluster_install.yml -v + +# example summary +Monday 04 October 2021 17:17:38 -0400 (0:00:00.045) 0:28:36.093 ******** +=============================================================================== +usercluster : [uc] Create user cluster -- 1485.98s +usercluster : [uc] Create load balancer -- 138.95s +usercluster : [uc] Templating YAML files -- 34.05s +usercluster : [uc] Templating YAML files -- 28.60s +usercluster : [uc] Templating YAML files -- 28.22s +usercluster : [uc] Sanity Checks -- 0.07s +usercluster : [uc] Include tasks - uninstall.yml - 0.05s +usercluster : [uc] Include tasks - install.yml 0.04s +usercluster : [uc] Include tasks - asserts.yml 0.04s +usercluster : [uc] Note on optional values - 0.03s +Playbook run took 0 days, 0 hours, 28 minutes, 36 seconds +``` + +### User Cluster - Playbook for Uninstallation + +Run the playbook (takes less than 10 minutes to complete). + +```shell +ansible-playbook -i inventory/site-a/usercluster01.yml playbooks/usercluster_uninstall.yml -v + +# example summary +Monday 02 May 2022 00:19:09 +0000 (0:00:00.039) 0:08:03.354 ************ +=============================================================================== +usercluster : [uc] Delete user cluster -- 482.59s +usercluster : [uc] Clean up old SSH host keys from incomplete runs -- 0.62s +usercluster : [uc] Delete load balancer -- 0.04s +usercluster : [uc] Include tasks - uninstall.yml -- 0.03s +usercluster : [uc] Include tasks - install.yml -- 0.02s +usercluster : [uc] Include tasks - asserts.yml -- 0.02s +Playbook run took 0 days, 0 hours, 8 minutes, 3 seconds +``` + +### User Cluster - Playbook for Upgrade + +Run the playbook (takes about 30 minutes to complete) after updating the `glb_anthos_version` variable defined in `inventory/site-a/all.yml` to the newer version. + +> **Note:** Upgrade to the admin workstation is a prerequisite to upgrading user clusters. Upgrades to the user clusters will otherwise result in a failure. + +```shell +ansible-playbook -i inventory/site-a/usercluster01.yml playbooks/usercluster_upgrade.yml -v + +# example summary +Friday 23 September 2022 15:24:56 -0400 (0:00:00.186) 1:14:18.537 ****** +=============================================================================== +usercluster : [uc] Upgrade user cluster -- 3937.77s +usercluster : [uc] Upload new bundle to vSphere -- 505.80s +[copy_gcp_sa] Copy GCP SA JSON key files to admin workstation -- 2.89s +usercluster : [uc] Check if existing api server is up -- 1.42s +[cleanup] Find GCP SA JSON files on admin workstation -- 1.13s +usercluster : [uc] Templating YAML files -- 0.86s +[copy_gcp_sa] Templating YAML files - vCenter credentials -- 0.76s +usercluster : [uc] Get current user cluster version -- 0.74s +usercluster : [uc] Get gkectl version from Admin workstation -- 0.72s +usercluster : [uc] Clean up old SSH host keys from incomplete runs -- 0.68s +usercluster : [uc] Templating YAML files - IP block -- 0.50s +[cleanup] Clean up sensitive file on admin workstation -- 0.48s +usercluster : [uc] Check for existing cluster kubeconfig -- 0.36s +usercluster : [uc] Replace cluster version in its config YAML file -- 0.35s +[cleanup] Clean up GCP SA JSON file folder on admin workstation -- 0.31s +[cleanup] Find *credential*.yaml files on admin workstation -- 0.27s +[cleanup] Find *credential*.yaml files on admin workstation in user cluster subfolder -- 0.25s +[cleanup] Clean up *credential*.yaml on admin workstation in user cluster subfolder -- 0.25s +[cleanup] Clean up sensitive files on jumphost -- 0.24s +[copy_gcp_sa] Ensure credentials directory exists -- 0.22s + +Playbook run took 0 days, 1 hours, 14 minutes, 18 seconds +``` + +### ASM - Playbook for Installation + +Anthos Service Mesh (ASM) can be optionally installed onto a user cluster. + +Run the playbook (takes about 5 minutes to complete). + +```shell +ansible-playbook -i inventory/site-a/usercluster01.yml playbooks/asm_install.yml -v + +# example summary +Monday 26 September 2022 14:16:22 -0400 (0:00:00.021) 0:02:44.482 ****** +=============================================================================== +[asm] Install Anthos Service Mesh -- 116.65s +[asm] Create ASM offline package bundle -- 17.65s +[asm] Add mesh_id label to cluster -- 4.27s +[copy_gcp_sa] Copy GCP SA JSON key files to admin workstation -- 3.96s +[asm] Create asm-ingress namespace -- 2.92s +[asm] Switch to Connect Register Service Account Name -- 1.49s +asm : [ASM] Detect if ASM already installed and get current revision -- 1.46s +[asm] Switch to asm Service Account Name -- 1.16s +[asm] Create RBAC rule required for asm -- 1.09s +[asm] Copy istioctl to istioctl_dest_path -- 1.03s +[asm] Download asmcli -- 1.01s +[copy_gcp_sa] Templating YAML files - vCenter credentials -- 0.79s +[asm] Create the Ingress Gateway -- 0.77s +[asm] Create istio-system namespace -- 0.73s +[asm] Template asm-ingress namespace -- 0.71s +[asm] Delete RBAC rule created above -- 0.68s +[asm] Template the ASM control plane config file -- 0.65s +[cleanup] Find GCP SA JSON files on admin workstation -- 0.64s +[asm] Template RBAC rule required for asm -- 0.60s +[asm] Template Istio system namespace -- 0.60s +Playbook run took 0 days, 0 hours, 2 minutes, 44 seconds +``` + +### ASM - Playbook for Uninstallation + +Run the playbook (takes less than 5 minutes to complete). + +```shell +ansible-playbook -i inventory/site-a/usercluster01.yml playbooks/asm_uninstall.yml -v + +# example summary +Monday 26 September 2022 14:53:22 -0400 (0:00:00.023) 0:00:42.965 ****** ------- 0.38s +=============================================================================== +[asm] Create ASM offline package bundle -- 18.25s +[asm] Delete istio-system, asm-ingress and asm-system namespaces -- 13.94s +[asm] Remove the in-cluster control plane -- 4.89s +asm : [ASM] Detect if ASM already installed and get current revision -- 1.44s +[asm] Download asmcli -- 0.89s +[asm] Delete webhooks -- 0.48s +[cleanup] Find GCP SA JSON files on admin workstation -- 0.47s +[cleanup] Delete temp directory created for ASM files -- 0.47s +[asm] Create folder for ASM files on admin workstation -- 0.29s +[cleanup] Find *credential*.yaml files on admin workstation in user cluster subfolder -- 0.21s +[asm] Make asmcli executable -- 0.21s +[cleanup] Find *credential*.yaml files on admin workstation -- 0.19s +[cleanup] Clean up sensitive file on admin workstation -- 0.13s +asm : [mesh] Setup cluster config -- 0.08s +asm : [ASM] Sanity Checks -- 0.07s +[asm] Copy istioctl to istioctl_dest_path -- 0.05s +asm : [ASM] Include tasks - uninstall.yml -- 0.04s +asm : [mesh] Sanity Checks -- 0.04s +Include role to delete sensitive files -- 0.04s +[cleanup] Include tasks - asserts.yml -- 0.03s +Playbook run took 0 days, 0 hours, 0 minutes, 42 seconds +``` + +### ASM - Playbook for Upgrade + +Run the playbook (takes about 5 minutes to complete) after updating the `asm_version`, `asm_revision` and `asm_asmcli_version` variables in the target user cluster's inventory file. + +For more information, refer to the `role/asm/README.md`. + +```shell +ansible-playbook -i inventory/site-a/usercluster01.yml playbooks/asm_upgrade.yml -v + +# example summary +Monday 26 September 2022 14:37:58 -0400 (0:00:16.585) 0:02:43.074 ****** +=============================================================================== +[asm] Install Anthos Service Mesh -- 101.68s +[asm] Create ASM offline package bundle -- 17.00s +[asm] Wait until the ingress gateway pods are ready -- 16.59s +[copy_gcp_sa] Copy GCP SA JSON key files to admin workstation -- 4.18s +[asm] Add mesh_id label to cluster -- 3.23s +[asm] Restart the Pods to trigger re-injection -- 2.57s +[asm] Create asm-ingress namespace -- 2.27s +asm : [ASM] Detect if ASM already installed and get current revision -- 1.77s +[asm] Switch to Connect Register Service Account Name -- 1.34s +[asm] Switch to asm Service Account Name -- 1.08s +[asm] Copy istioctl to istioctl_dest_path -- 1.08s +[asm] Download asmcli -- 0.94s +[copy_gcp_sa] Templating YAML files - vCenter credentials -- 0.80s +[asm] Create RBAC rule required for asm -- 0.74s +[asm] Template asm-ingress namespace -- 0.73s +[asm] Template the ASM control plane config file -- 0.68s +[asm] Template RBAC rule required for asm -- 0.63s +[cleanup] Find GCP SA JSON files on admin workstation -- 0.47s +[copy_gcp_sa] Ensure credentials directory exists -- 0.42s +[asm] Delete old RBAC rule required for asm (if exists) -- 0.38s +Playbook run took 0 days, 0 hours, 2 minutes, 43 seconds +``` + +### ACM - Playbook for Installation + +Anthos Config Management (ACM) can be optionally installed onto a user cluster. + +Run the playbook (takes about 15 minutes to complete). + +```shell +ansible-playbook -i inventory/site-a/usercluster01.yml playbooks/acm_install.yml -v + +# example summary +Monday 19 September 2022 08:58:27 -0400 (0:00:00.020) 0:07:09.933 ****** +=============================================================================== +[acm] Wait for Config Sync to be synced -- 411.97s +[copy_gcp_sa] Copy GCP SA JSON key files to admin workstation -- 3.22s +[acm] Enable ACM -- 3.19s +[acm] Switch to ACM Service Account Name -- 1.38s +[acm] Get current ACM version -- 1.32s +[acm] Create ACM namespace -- 1.25s +[acm] Check ACM Feature Status -- 1.18s +[acm] Clean up old SSH host keys from incomplete runs -- 0.99s +[acm] Copy ACM namespace file -- 0.86s +[copy_gcp_sa] Templating YAML files - vCenter credentials -- 0.78s +[acm] Templating YAML files -- 0.56s +[cleanup] Find GCP SA JSON files on admin workstation -- 0.50s +[copy_gcp_sa] Create folder for GCP SA JSON key files on admin workstation -- 0.26s +[acm] Clean up root repository ssh key file -- 0.24s +[cleanup] Clean up *credential*.yaml on admin workstation in user cluster subfolder -- 0.21s +[cleanup] Clean up GCP SA JSON file folder on admin workstation -- 0.21s +[cleanup] Find *credential*.yaml files on admin workstation in user cluster subfolder -- 0.20s +[copy_gcp_sa] Ensure credentials directory exists -- 0.20s +[cleanup] Find *credential*.yaml files on admin workstation -- 0.19s +[cleanup] Clean up sensitive file on admin workstation -- 0.13s +Playbook run took 0 days, 0 hours, 5 minutes, 9 seconds +``` + +### ACM - Playbook for Uninstallation + +Run the playbook (takes less than 10 minutes to complete). + +```shell +ansible-playbook -i inventory/site-a/usercluster01.yml playbooks/acm_uninstall.yml -v + +# example summary +Tuesday 20 September 2022 09:40:18 -0400 (0:00:00.020) 0:01:05.454 ****** +=============================================================================== +[acm] Remove ACM namespaces -- 47.00s +[acm] Remove ACM operator configuration -- 3.79s +[copy_gcp_sa] Copy GCP SA JSON key files to admin workstation -- 3.52s +[acm] Disable ACM -- 3.21s +[acm] Switch to ACM Service Account Name -- 1.52s +[acm] Get current ACM version -- 1.40s +[copy_gcp_sa] Templating YAML files - vCenter credentials -- 0.81s +[acm] Remove ACM operator CRD -- 0.62s +[acm] Clean up old SSH host keys from incomplete runs -- 0.51s +[cleanup] Find GCP SA JSON files on admin workstation -- 0.48s +``` + +### ACM - Playbook for Upgrade + +Run the playbook (takes about 15 minutes to complete) after updating the `acm_version` variable defined in the relevant user cluster's inventory file. + +```shell +ansible-playbook -i inventory/site-a/usercluster01.yml playbooks/acm_upgrade.yml -v + +# example summary +Wednesday 21 September 2022 10:30:18 -0400 (0:00:00.028) 0:02:12.284 ****** +=============================================================================== +[acm] Wait for Config Sync -- 88.43s +[acm] Wait for upgrade to start -- 30.07s +[acm] Upgrade ACM -- 3.36s +[acm] Switch to ACM Service Account Name -- 1.49s +[acm] Get current ACM version -- 1.40s +[acm] Clean up old SSH host keys from incomplete runs -- 0.48s +[acm] Wait for Policy Controller -- 0.05s +[acm] Sanity Checks -- 0.04s +[acm] Compare current version with requested -- 0.04s +Playbook run took 0 days, 0 hours, 2 minutes, 12 seconds +``` + +## Ansible Tower Prerequisites + +Custom Credential Types to store GSA JSON key file content. + +### GSA - Component Access + +```yaml +# INPUT CONFIGURATION +fields: + - id: json + type: string + label: JSONcontent + secret: true +required: + - json + +# INJECTOR CONFIGURATION +env: + GCPSA_COMACC_FILE: '{{ tower.filename }}' +file: + template: '{{ json }}' +``` + +### GSA - Logging-Monitoring + +```yaml +# INPUT CONFIGURATION +fields: + - id: json + type: string + label: JSONcontent + secret: true +required: + - json + +# INJECTOR CONFIGURATION +env: + GCPSA_LOGMON_FILE: '{{ tower.filename }}' +file: + template: '{{ json }}' +``` + +### GSA - Connect Register + +```yaml +# INPUT CONFIGURATION +fields: + - id: json + type: string + label: JSONcontent + secret: true +required: + - json + +# INJECTOR CONFIGURATION +env: + GCPSA_CONREG_FILE: '{{ tower.filename }}' +file: + template: '{{ json }}' +``` + +### GSA - Audit Logging + +```yaml +# INPUT CONFIGURATION +fields: + - id: json + type: string + label: JSONcontent + secret: true +required: + - json + +# INJECTOR CONFIGURATION +env: + GCPSA_AUDLOG_FILE: '{{ tower.filename }}' +file: + template: '{{ json }}' +``` + +### Optional - privateRegistry for container images + +```yaml +# INPUT CONFIGURATION +fields: + - id: address + type: string + label: Address + - id: username + type: string + label: Username + - id: password + type: string + label: Password + secret: true + - id: email + type: string + label: Email + - id: ca_cert + type: string + label: CA_CERT + multiline: true +required: + - username + - password + +# INJECTOR CONFIGURATION +env: + PRIV_REG_CA_FILE: '{{ tower.filename.ca_file }}' + PRIV_REG_PASSWORD: '{{ password }}' + PRIV_REG_USERNAME: '{{ username }}' + PRIV_REG_ADDRESS: '{{ address }}' + PRIV_REG_EMAIL: '{{ email }}' +file: + template.ca_file: '{{ ca_cert }}' +``` + +### Optional - private repository for artifacts + +```yaml +# INPUT CONFIGURATION +fields: +- id: repo_base_url + type: string + label: "The base url for the repository, for artifactory this would be https://{hostname}/artifactory/{repository}" +- id: repo_username + type: string + label: "The username for authenticating to the repository" +- id: repo_enc_password + type: string + label: "The encrypted password for the user. Authentication can use this or the API token" + secret: true +- id: repo_api_token + type: string + label: "The API token for the user. Authentication can use this or the encrypted password" + secret: true +required: + - repo_url + - repo_username + +# INJECTOR CONFIGURATION +env: + ARTIFACT_BASE_URL: '{{ repo_base_url }}' + ARTIFACT_USERNAME: '{{ repo_username }}' + ARTIFACT_ENC_PASS: '{{ repo_enc_password }}' + ARTIFACT_API_TOKEN: '{{ repo_api_token }}' +``` + +### Optional - GCP SA - Enabling GKE Hub feature Anthos Config Management (ACM) + +```yaml +# INPUT CONFIGURATION +fields: + - id: gcpsaacm + type: string + label: JSONcontent + secret: true +required: + - gcpsaacm + +# INJECTOR CONFIGURATION +env: + GCPSA_ACM_FILE: '{{ tower.filename }}' +file: + template: '{{ gcpsaacm }}' +``` + +### Optional - GCP SA - Installing Anthos Service Mesh (ASM) + +```yaml +# INPUT CONFIGURATION +fields: + - id: gcpsaasm + type: string + label: JSONcontent + secret: true +required: + - gcpsaasm + +# INJECTOR CONFIGURATION +env: + GCPSA_ASMCFG_FILE: '{{ tower.filename }}' +file: + template: '{{ gcpsaasm }}' +``` + +### Optional - SSH Private Key - Authenticate ACM Config Sync to the main root repository + +```yaml +# INPUT CONFIGURATION +fields: + - id: acmsshprivkey + type: string + label: "SSH Private Key for ACM root repository" + secret: true + format: ssh_private_key + multiline: true +required: + - acmsshprivkey + +# INJECTOR CONFIGURATION +env: + GIT_ACMSSH_FILE: '{{ tower.filename }}' +file: + template: '{{ acmsshprivkey }}' +``` + +### Optional - AIS OIDC Identity Provider Secrets - Configure OIDC connection to Identity provider from AIS + +```yaml +# INPUT CONFIGURATION +fields: + - id: clientid + type: string + label: "Client ID for OIDC" + - id: clientsecret + type: string + label: "Client Secret for OIDC" + secret: true + - id: cadata + type: string + label: "Certificate chain to verify SSL for Identity Privider. Certificates must be in PEM format, concatenated and base64 encoded" +required: + - clientid + - clientsecret + +# INJECTOR CONFIGURATION +env: + AIS_OIDC_CLIENTID: '{{ clientid }}' + AIS_OIDC_CLIENTSECRET: '{{ clientsecret }}' + AIS_OIDC_CADATA: '{{ cadata }}' +``` + +### Optional - AIS LDAP Identity Provider Secrets - Configure LDAP connection to Identity provider from AIS + +```yaml +# INPUT CONFIGURATION +fields: + - id: username + type: string + label: "username for service account to connect to LDAP" + - id: password + type: string + label: "password for service account to connect to LDAP" + secret: true + - id: cadata + type: string + label: "Certificate chain to verify SSL for Identity Privider. Certificates must be in PEM format, concatenated and base64 encoded" +required: + - username + - password + +# INJECTOR CONFIGURATION +env: + AIS_LDAP_USERNAME: '{{ username }}' + AIS_LDAP_PASSWORD: '{{ password }}' + AIS_LDAP_CADATA: '{{ cadata }}' +``` + +### Optional - AIS OIDC login config file - Copy to Google Cloud Storage bucket + +This Google Service Account is optional. +It can be used to copy the Anthos user cluster login config to a Google Cloud Storage bucket. + +```yaml +# INPUT CONFIGURATION +fields: + - id: gcpsaais + type: string + label: JSONcontent + secret: true + - id: gcsbucket + type: string + label: "GCS bucket name" +required: + - gcpsaais + - gcsbucket + +# INJECTOR CONFIGURATION +env: + GCPSA_AIS_FILE: '{{ tower.filename }}' + AIS_GCS_BUCKET: '{{ gcsbucket }}' +file: + template: '{{ gcpsaais }}' +``` + +### Optional - ASM Ingress Secrets - Configure TLS secret for ingress gateway + +```yaml +# INPUT CONFIGURATION +fields: + - id: tls_cert + type: string + label: The certificate for TLS + multiline: true + - id: tls_key + type: string + label: The TLS key + multiline: true + secret: true +required: + - tls_cert + - tls_key + +# INJECTOR CONFIGURATION +env: + ASM_TLSCERT_FILE: '{{ tower.filename.tls_cert }}' + ASM_TLSKEY_FILE: '{{ tower.filename.tls_key }}' +file: + template.tls_cert: '{{ tls_cert }}' + template.tls_key: '{{ tls_key }}' +``` + +### Optional - ASM mTLS Secrets - Configure certs for mTLS connection between the workloads + +```yaml +# INPUT CONFIGURATION +fields: + - id: root_cert + type: string + label: The root certificate for mTLS + multiline: true + - id: cert_chain + type: string + label: The cert chain for mTLS + multiline: true + - id: ca_cert + type: string + label: The CA certificate for mTLS + multiline: true + - id: ca_key + type: string + label: The CA key for mTLS + multiline: true + secret: true +required: + - root_cert + - cert_chain + - ca_cert + - ca_key + +# INJECTOR CONFIGURATION +env: + ASM_ROOTCERT_FILE: '{{ tower.filename.root_cert }}' + ASM_CERTCHAIN_FILE: '{{ tower.filename.cert_chain }}' + ASM_CACERT_FILE: '{{ tower.filename.ca_cert }}' + ASM_CAKEY_FILE: '{{ tower.filename.ca_key }}' +file: + template.root_cert: '{{ root_cert }}' + template.cert_chain: '{{ cert_chain }}' + template.ca_cert: '{{ ca_cert }}' + template.ca_key: '{{ ca_key }}' +``` + +## **License** + +Copyright 2022 Google LLC. This software is provided as-is, without warranty or representation for any use or purpose. +Your use of it is subject to your agreement with Google. \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/ansible.cfg b/tools/anthosvmware-ansible-module/ansible.cfg new file mode 100644 index 0000000000..b559eb3969 --- /dev/null +++ b/tools/anthosvmware-ansible-module/ansible.cfg @@ -0,0 +1,33 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +[defaults] +host_key_checking = False +retry_files_enabled = False +command_warnings = False +deprecation_warnings = False +system_warnings = False +nocows = True + +inventory = ./inventory +roles_path = ./roles + +callback_whitelist = profile_tasks,timer,counter_enabled + +# to get easily readable output from gkeadm and gkectl +stdout_callback = yaml + +[ssh_connection] +pipelining = True +scp_if_ssh = True diff --git a/tools/anthosvmware-ansible-module/inventory/mesh-a/mesh.yml b/tools/anthosvmware-ansible-module/inventory/mesh-a/mesh.yml new file mode 100644 index 0000000000..2a5e0e2fa0 --- /dev/null +++ b/tools/anthosvmware-ansible-module/inventory/mesh-a/mesh.yml @@ -0,0 +1,38 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +all: + hosts: + member1: + ansible_host: 10.20.0.39 + uc_name: "uc1" + asm_network_id: "{{ uc_name }}" + member2: + ansible_host: 10.20.0.39 + uc_name: "uc2" + asm_network_id: "{{ uc_name }}" + vars: + yamldestpath: "/home/ubuntu" + ansible_user: "ubuntu" + sakeyfolder_base: "/home/ubuntu/sakeys" + mesh_project_id: "anthos-lab1" + mesh_unique_id: "mesh-lab1" + asm_version: "1.13" + asm_revision: "asm-1137-3" + asm_asmcli_version: "1.13.7-asm.3-config1" + asm_gcpsa: '{{ lookup("env", "GCPSA_ASMCFG_FILE") }}' + asm_gcpsa_path: "asm-meshconfig.json" + # Offline mode related + asm_offline_mode: false + asm_offline_bundle_file: "asm-1.13.7.tar.gz" \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/inventory/site-a/admin.yml b/tools/anthosvmware-ansible-module/inventory/site-a/admin.yml new file mode 100644 index 0000000000..2e5720ca35 --- /dev/null +++ b/tools/anthosvmware-ansible-module/inventory/site-a/admin.yml @@ -0,0 +1,27 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +all: + hosts: + children: + adminws: + hosts: + 10.20.0.3: + admincluster: + hosts: + 10.20.0.39: + upload: + hosts: + 10.20.0.39: +# jump host to reach the vSphere API \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/admincluster.yml b/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/admincluster.yml new file mode 100644 index 0000000000..228a1d0506 --- /dev/null +++ b/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/admincluster.yml @@ -0,0 +1,116 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# ansible_ssh_common_args is required if your connection is: workstation (Ansible) -> jumphost -> Admin Workstation +# ansible_ssh_common_args: '-o ProxyCommand="ssh -o StrictHostKeyChecking=no -W %h:%p -q anthosjumper@10.20.0.3"' + +# SSH private key on jump host to connect to Admin Workstation +# ansible_ssh_private_key_file="/home/anthosjumper/.ssh/user_id_ed25519" + +ansible_user: "ubuntu" +yamldestpath: "/home/ubuntu" + +# Admin Cluster (AC) role specific below +ac_name: "ac1" +component_access_gcpsa: '{{ lookup("env", "GCPSA_COMACC_FILE") }}' +component_access_gcpsa_path: "component-access.json" + +# vSphere/vCenter +ac_vc_fqdn: "{{ glb_vc_fqdn }}" +ac_vc_credfile: "credential.yaml" +ac_vc_credentry: "{{ glb_vc_credentry }}" +ac_vc_username: "{{ glb_vc_username }}" # set with extra vars or lookup from shell variable, see README +ac_vc_password: "{{ glb_vc_password }}" # set with extra vars or lookup from shell variable, see README +ac_vc_datacenter: "{{ glb_vc_datacenter }}" +ac_vc_datastore: "{{ glb_vc_datastore }}" +ac_vc_cluster: "{{ glb_vc_cluster }}" +ac_vc_folder: "{{ glb_vc_folder }}" # optional +ac_vc_respool: "{{ glb_vc_respool }}" # if default resourcePool use /Resources +ac_vc_cacertpath: "{{ glb_vc_cacertpath }}" +# for vSAN: ac_vc_datadisk must be created inside a folder when using vSAN. Folder must be created manually +ac_vc_datadisk: "{{ ac_name }}/{{ ac_name }}-admin-cluster.vmdk" + +# Networking +ac_nw_ipallocmode: "static" # dhcp or static +ac_nw_ipfile: "{{ ac_name }}-ip-block.yaml" +ac_nw_gw: "10.20.0.1" # gateway +ac_nw_nm: "255.255.255.0" # netmask 255.255.255.0 or similar +ac_nw_ntp: "{{ glb_ntp }}" # list from group_vars/all +ac_nw_dns: "{{ glb_dns }}" # list from group_vars/all +ac_nw_searchdomains: "{{ glb_dns_searchdomains }}" # list from group_vars/all +ac_nw_servicecidr: "10.96.232.0/24" # default +ac_nw_podcidr: "192.168.0.0/16" # default +ac_nw_vc_net: "VM Network" + +# values for the file content of network.ipMode.ipBlockFilePath +ac_ipblock_netmask: "255.255.255.0" +ac_ipblock_gateway: "10.20.0.1" +ac_ipblock_ips: ["10.20.0.40", "10.20.0.41", "10.20.0.42", "10.20.0.43", "10.20.0.44", "10.20.0.45"] + +# Load balancing +ac_lb_kind: MetalLB +ac_lb_vips_cp: "10.20.0.46" +ac_lb_vips_addons: "10.20.0.47" + +# masternode sizing +ac_masternode_cpus: 4 +ac_masternode_mem: 8192 +ac_antiaffinitygroups: false + +# GCP project IDs +ac_stackdriver_projectid: "gcp_project" +ac_stackdriver_clusterlocation: "us-central1" +ac_stackdriver_enablevpc: false +logging_monitoring_gcpsa: '{{ lookup("env", "GCPSA_LOGMON_FILE") }}' +logging_monitoring_gcpsa_path: "logging-monitoring.json" +ac_stackdriver_disablevsphereresourcemetrics: false +ac_gkeconnect_projectid: "gcp_project" +connect_register_gcpsa: '{{ lookup("env", "GCPSA_CONREG_FILE") }}' +connect_register_gcpsa_path: "connect-register.json" +ac_cloudauditlogging_projectid: "gcp_project" +ac_cloudauditlogging_clusterlocation: "us-central1" +audit_logging_gcpsa: '{{ lookup("env", "GCPSA_AUDLOG_FILE") }}' +audit_logging_gcpsa_path: "audit-logging.json" + +ac_autorepair: true + +# Kubernetes Secrets at-rest encryption +ac_secretsencryption_mode: "GeneratedKey" # optional +ac_secretsencryption_keyversion: 1 # optional + +ac_verbosity: 5 +ac_skipvalidations: "" +# example below to specify more than one skip validation option +# ac_skipvalidations: "--skip-validation-load-balancer --skip-validation-node-ips" + +# Optional - when using private artifact repository +private_download_base: "{{ glb_private_download_base }}" +private_download_auth_header: "{{ glb_private_download_auth_header }}" + +# Optional: AIS +# Uncomment out this section to specify AIS configuration +# An uncommented empty block will result in errors +# ais_install: true +# ais_authentication: +# - name: oidc-ad +# oidc: +# clientID: "" +# clientSecret: "" +# cloudConsoleRedirectURI: "" +# extraParams: prompt=consent,access_type=offline +# issuerURI: "" +# kubectlRedirectURI: "" +# scopes: "" +# userClaim: "" \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/adminws.yml b/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/adminws.yml new file mode 100644 index 0000000000..ea1afc52dc --- /dev/null +++ b/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/adminws.yml @@ -0,0 +1,62 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# private ssh key on the jump host to use for the admin workstation creation +sshprivkeypath: "/home/anthosjumper/.ssh/user_id_ed25519" + +# folder where the templated YAML files result in +yamldestpath: "/home/anthosjumper/anthos" # template out YAML files relative to playbooks/ subfolder + +# path to gkeadm binary on the jump host +gkeadm: "{{ yamldestpath }}/gkeadm" + +# SSH user used by Ansible to ssh connect to the jump host +ansible_user: "anthosjumper" + +# admin workstation specific below until end of file +adminws_name: "user-admws" +component_access_gcpsa: '{{ lookup("env", "GCPSA_COMACC_FILE") }}' +component_access_gcpsa_path: "component-access.json" +sakeyfolder_base: "/home/anthosjumper/sakeys" # different home directory on jumphost than adminWS +adminws_create_gsa: false + +# vSphere/vCenter +adminws_vc_fqdn: "{{ glb_vc_fqdn }}" +adminws_vc_validate_cert: true +adminws_vc_credfile: "credential.yaml" +adminws_vc_credentry: "{{ glb_vc_credentry }}" +adminws_vc_datacenter: "{{ glb_vc_datacenter }}" +adminws_vc_datastore: "{{ glb_vc_datastore }}" +adminws_vc_cluster: "{{ glb_vc_cluster }}" +adminws_vc_network: "{{ glb_vc_network }}" # VM Network +adminws_vc_folder: "{{ glb_vc_folder }}" # optional +adminws_vc_respool: "{{ glb_vc_respool }}" # if default resourcePool use /Resources +adminws_vc_cacertpath: "{{ yamldestpath }}/vcenter.pem" +adminws_datadiskname: "{{ adminws_name }}-data-disk/{{ adminws_name }}-data-disk.vmdk" + +# Networking +adminws_nw_ipallocmode: "static" # dhcp or static +adminws_nw_ip: "10.20.0.39" # IP address of VM +adminws_nw_gw: "10.20.0.1" # gateway +adminws_nw_nm: "255.255.255.0" # netmask 255.255.255.0 or similar +adminws_nw_dns: "{{ glb_dns }}" # list from group_vars/all +adminws_ntp: "{{ glb_ntp[0] }}" # only one can be set default ntp.ubuntu.com + +adminws_skipvalidations: "" +# adminws_skipvalidations: "--skip-validation" + +# Optional - when using private artifact repository +private_download_base: "{{ glb_private_download_base }}" +private_download_auth_header: "{{ glb_private_download_auth_header }}" diff --git a/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/all.yml b/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/all.yml new file mode 100644 index 0000000000..7712343a70 --- /dev/null +++ b/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/all.yml @@ -0,0 +1,56 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# vSphere parameters - shared by the admin and user clusters +glb_vc_fqdn: '{{ lookup("env", "VMWARE_HOST") }}' +glb_vc_username: '{{ lookup("env", "VMWARE_USER") }}' +glb_vc_password: '{{ lookup("env", "VMWARE_PASSWORD") }}' +glb_vc_credfile: "credential.yaml" +glb_vc_credentry: "vCenter" +glb_vc_datacenter: "dc.anthoslab.gke" +glb_vc_datastore: "vm-data" +glb_vc_cluster: "cluster.anthoslab.gke" +glb_vc_network: "VM Network" +glb_vc_folder: "user" # optional +glb_vc_respool: "cluster.anthoslab.gke/Resources" # if default resourcePool use /Resources +glb_vc_cacertpath: "/home/ubuntu/vcenter.pem" + +# networking +glb_dns: ["10.20.0.5"] # list +glb_dns_searchdomains: [""] # list +glb_proxyurl: "" # optional used by admin workstation, admin and user clusters +glb_noproxy: "" +glb_ntp: ["ntp.ubuntu.com"] # list + +# GCP +sakeyfolder_base: "/home/ubuntu/sakeys" +glb_anthos_version: "1.11.1-gke.53" # The Anthos version that will be used across the admin workstation, admin cluster, and user cluster installs +glb_major_version: "{{ glb_anthos_version | regex_search('[\\d]\\.[\\d]+') }}" + +# Optional - when using private container registry +glb_privatereg_url: '{{ lookup("env", "PRIV_REG_ADDRESS") }}' # fqdn and repo_name (ie, artifactory.domain.com/repo_name) +glb_privatereg_cacertfile: '{{ lookup("env", "PRIV_REG_CA_FILE") }}' # The CA certificate for this registry, will be copied to glb_privatereg_cacertpath +glb_privatereg_cacertpath: "ca.crt" # The absolute or relative path where CA certificate stored for this registry +glb_privatereg_cacerturl: "" # if you want to download the cert for url, specify here +glb_privatereg_username: '{{ lookup("env", "PRIV_REG_USERNAME") }}' +glb_privatereg_password: '{{ lookup("env", "PRIV_REG_PASSWORD") }}' # API token or password +glb_privatereg_email: '{{ lookup("env", "PRIV_REG_EMAIL") }}' + +# Optional - when using private artifact repository +glb_private_download_base: '{{ lookup("env", "ARTIFACT_HOST") }}' +glb_private_download_username: '{{ lookup("env", "ARTIFACT_USERNAME") }}' +glb_private_download_enc_pass: '{{ lookup("env", "ARTIFACT_ENC_PASS") }}' +glb_private_download_api_token: '{{ lookup("env", "ARTIFACT_API_TOKEN") }}' +glb_private_download_auth_header: "{{ 'Bearer ' + glb_private_download_api_token if glb_private_download_api_token|length > 2 else 'Basic ' + ([glb_private_download_username, glb_private_download_enc_pass] | join(':') | b64encode) }}" diff --git a/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/upload.yml b/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/upload.yml new file mode 100644 index 0000000000..5adfc6c57d --- /dev/null +++ b/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/upload.yml @@ -0,0 +1,54 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# ansible_ssh_common_args is required if your connection is: workstation (Ansible) -> jumphost -> Admin Workstation +# ansible_ssh_common_args: '-o ProxyCommand="ssh -W %h:%p -q anthosjumper@10.147.18.191"' + +# SSH private key on jump host to connect to Admin Workstation +# ansible_ssh_private_key_file="/home/anthosjumper/.ssh/user_id_ed25519" + +ansible_user: "ubuntu" + +component_access_gcpsa: '{{ lookup("env", "GCPSA_COMACC_FILE") }}' +component_access_gcpsa_path: "component-access.json" +upload_artifactory_fqdn: "YOUR_SERVER_FQDN_HERE" +upload_artifactory_repo_name: "YOUR_REPO_NAME_HERE" +upload_artifactory_username: '{{ lookup("env", "PRIV_REG_USERNAME") }}' +upload_artifactory_password: '{{ lookup("env", "PRIV_REG_PASSWORD") }}' + +govc_version: "0.29.0" +asm_asmcli_version: "1.13.7-asm.3-config1" +gcloud_version: "404.0.0" + +upload_artifactory_http_artifacts: + - file: "govc-{{ govc_version }}.tar.gz" + src: "https://github.com/vmware/govmomi/releases/download/v{{ govc_version }}/govc_Linux_x86_64.tar.gz" + dst: "govc/{{ govc_version }}/govc-{{ govc_version }}.tar.gz" + - file: "asmcli_{{ asm_asmcli_version }}" + src: "https://storage.googleapis.com/csm-artifacts/asm/asmcli_{{ asm_asmcli_version }}" + dst: "asm/{{ asm_asmcli_version }}/asmcli_{{ asm_asmcli_version }}" + - file: "google-cloud-cli-{{ gcloud_version }}-linux-x86_64.tar.gz" + src: "https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-{{ gcloud_version }}-linux-x86_64.tar.gz" + dst: "gcloud/{{ gcloud_version }}/google-cloud-cli-{{ gcloud_version }}-linux-x86_64.tar.gz" + +upload_artifactory_gs_artifacts: + - file: "gke-on-prem-admin-appliance-vsphere-{{ glb_anthos_version }}.ova" + src: "gs://gke-on-prem-release/admin-appliance/{{ glb_anthos_version }}/gke-on-prem-admin-appliance-vsphere-{{ glb_anthos_version }}.ova" + dst: "anthos/{{ glb_anthos_version }}/gke-on-prem-admin-appliance-vsphere-{{ glb_anthos_version }}.ova" + - file: "gkeadm-{{ glb_anthos_version }}" + src: "gs://gke-on-prem-release/gkeadm/{{ glb_anthos_version }}/linux/gkeadm" + dst: "anthos/{{ glb_anthos_version }}/gkeadm-{{ glb_anthos_version }}" + - file: "gkeadm-{{ glb_anthos_version }}.1.sig" + src: gs://gke-on-prem-release/gkeadm/{{ glb_anthos_version }}/linux/gkeadm.1.sig + dst: "anthos/{{ glb_anthos_version }}/gkeadm-{{ glb_anthos_version }}.1.sig" \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/usercluster01.yml b/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/usercluster01.yml new file mode 100644 index 0000000000..32c67472e1 --- /dev/null +++ b/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/usercluster01.yml @@ -0,0 +1,186 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# ansible_ssh_common_args is required if your connection is: Ansible -> jumphost -> Admin Workstation +# ansible_ssh_common_args: '-o ProxyCommand="ssh -W %h:%p -q anthosjumper@10.147.18.191"' + +# SSH private key on jump host to connect to Admin Workstation +# ansible_ssh_private_key_file="/home/anthosjumper/.ssh/user_id_ed25519" + +ansible_user: "ubuntu" +yamldestpath: "/home/ubuntu" + +# User Cluster (UC) role specific below +uc_name: "uc1" +component_access_gcpsa: '{{ lookup("env", "GCPSA_COMACC_FILE") }}' +component_access_gcpsa_path: "component-access.json" + +# vSphere/vCenter - inherited from admin cluster by default but can be overwritten +# to place user cluster in a different vSphere datacenter +# uc_vc_fqdn: "" +# uc_vc_credfile: "credential-{{ uc_name }}.yaml" +# uc_vc_credentry: "vCenter" +# uc_vc_username: "" +# uc_vc_password: "" +# uc_vc_datacenter: "" +# uc_vc_datastore: "" +# uc_vc_cluster: "" +# uc_vc_network: "" # VM Network +# uc_vc_folder: "" # optional +# uc_vc_respool: "" # if default resourcePool use /Resources +# uc_vc_cacertpath: "" + +# Networking +uc_nw_ipallocmode: "static" # dhcp or static +uc_nw_ipfile: "{{ uc_name }}-ip-block.yaml" +uc_nw_gw: "10.20.0.1" # gateway +uc_nw_nm: "255.255.255.0" # netmask 255.255.255.0 or similar +uc_nw_ntp: "{{ glb_ntp }}" # list from group_vars/all +uc_nw_dns: "{{ glb_dns }}" # list from group_vars/all +uc_nw_searchdomains: "{{ glb_dns_searchdomains }}" # list from group_vars/all +uc_nw_servicecidr: "10.96.0.0/20" +uc_nw_podcidr: "192.168.0.0/16" +uc_nw_vc_net: "VM Network" + +# values for the file content of network.ipMode.ipBlockFilePath +uc_ipblock_netmask: "255.255.255.0" +uc_ipblock_gateway: "10.20.0.1" +uc_ipblock_ips: ["10.20.0.52", "10.20.0.53", "10.20.0.54", "10.20.0.55"] + +# Load balancing +uc_lb_kind: MetalLB +uc_lb_vips_cp: "10.20.0.50" +uc_lb_vips_ingress: "10.20.0.245" +uc_lb_metallb_ips: "10.20.0.245-10.20.0.250" + +uc_dataplanev2: true + +# masternode sizing +uc_masternode_cpus: 4 +uc_masternode_mem: 8192 +uc_masternode_replicas: 1 +uc_antiaffinitygroups: false +# uc_masternode_datastore: "" # Set this to admin cluster datastore when you want to place user cluster in a differet vSphere cluster/datacenter than the admin cluster. + +# nodepools +uc_nodepools_name: "{{ uc_name }}-pool-1" +uc_nodepools_cpus: 4 +uc_nodepools_mem: 8192 +uc_nodepools_replicas: 3 +# uc_nodepools_osimagetype: cos +uc_nodepools_osimagetype: ubuntu_containerd +uc_nodepools: +- name: "{{ uc_name | regex_replace('^(\\w*-)(.*)([au]-.*)', '\\1\\3') }}-p01" + cpus: "{{ uc_nodepools_cpus }}" # default 4 + memoryMB: "{{ uc_nodepools_mem }}" # default 8192 + replicas: "{{ uc_nodepools_replicas }}" # default 3 + osImageType: "{{ uc_nodepools_osimagetype }}" + autoscaling: + minReplicas: 3 + maxReplicas: 4 + enableLoadBalancer: true # set to true only if uc_lb_kind equals MetalLB +- name: "{{ uc_name | regex_replace('^(\\w*-)(.*)([au]-.*)', '\\1\\3') }}-p02" + cpus: "{{ uc_nodepools_cpus }}" # default 4 + memoryMB: "{{ uc_nodepools_mem }}" # default 8192 + replicas: "{{ uc_nodepools_replicas }}" # default 3 + osImageType: "{{ uc_nodepools_osimagetype }}" + autoscaling: + minReplicas: 3 + maxReplicas: 4 + enableLoadBalancer: true # set to true only if uc_lb_kind equals MetalLB + +# GCP project IDs +uc_stackdriver_projectid: "anthos-lab-01" +uc_stackdriver_clusterlocation: "us-central1" +uc_stackdriver_enablevpc: false +logging_monitoring_gcpsa_path: "logging-monitoring.json" +uc_stackdriver_disablevsphereresourcemetrics: false +uc_gkeconnect_projectid: "anthos-lab-01" +connect_register_gcpsa_path: "connect-register.json" +uc_cloudauditlogging_projectid: "anthos-lab-01" +uc_cloudauditlogging_clusterlocation: "us-central1" +audit_logging_gcpsa_path: "audit-logging.json" + +uc_autorepair: true + +# Kubernetes Secrets at-rest encryption +uc_secretsencryption_mode: "GeneratedKey" +uc_secretsencryption_keyversion: 1 + +uc_verbosity: 8 +uc_skipvalidations: "" +# uc_skipvalidations: "--skip-validation-all" +# uc_skipvalidations: "--skip-validation-load-balancer" +# uc_skipvalidations: "--skip-validation-load-balancer --skip-validation-node-ips" + +# Optional - when using private artifact repository +private_download_base: "{{ glb_private_download_base }}" +private_download_auth_header: "{{ glb_private_download_auth_header }}" + +# Optional: ASM +asm_version: "1.13" +asm_revision: "asm-1137-3" +asm_asmcli_version: "1.13.7-asm.3-config1" +asm_network_id: "uc1-6789" +# ASM - GCP related +asm_gcp_project: "asm-test" # GCP project where user cluster is registered +asm_gcp_project_number: "123456789" +asm_gcpsa_path: "asm-meshconfig.json" +asm_user_email: "asm-sa@asm-test.iam.gserviceaccount.com" + +# Optional: ACM +acm_membership: "{{ uc_name }}" # user cluster name +acm_version: "1.12.1" # version in format #.#.# +# ACM - GCP related +acm_gcpproject: "acm-test" +acm_gcpsa_path: "acm-gcpsa.json" +# ACM - Config Sync +acm_configsync: + # Set to true to install and enable Config Sync + enabled: false + # FORMAT - unstructured, hierarchy + sourceFormat: unstructured + # REPO - git repository URL + syncRepo: "" + syncBranch: "" + # TYPE - none, ssh, cookiefile, token, gcpserviceaccount, gcenode + secretType: gcpserviceaccount + # If TYPE == gcpserviceaccount, then add its email below + gcpServiceAccountEmail: acm-sa@acm-test.iam.gserviceaccount.com + # DIRECTORY - folder in git repository for ACM + policyDir: "" + # PREVENT_DRIFT - true, false + preventDrift: true +acm_root_repo_sshkey: "" # ssh file key if using acm_configsync.secretType: ssh +# ACM - PolicyController +acm_policycontroller: + # Set to true to install and enable Policy Controller + enabled: false + +# Optional: AIS +# Uncomment out this section to specify AIS configuration +# An uncommented empty block will result in errors +# ais_install: true +# ais_authentication: +# - name: oidc-ad +# oidc: +# clientID: "" +# clientSecret: "" +# cloudConsoleRedirectURI: "" +# extraParams: prompt=consent,access_type=offline +# issuerURI: "" +# kubectlRedirectURI: "" +# scopes: "" +# userClaim: "" \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/usercluster02.yml b/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/usercluster02.yml new file mode 100644 index 0000000000..6970106c92 --- /dev/null +++ b/tools/anthosvmware-ansible-module/inventory/site-a/group_vars/usercluster02.yml @@ -0,0 +1,186 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# ansible_ssh_common_args is required if your connection is: Ansible -> jumphost -> Admin Workstation +# ansible_ssh_common_args: '-o ProxyCommand="ssh -W %h:%p -q anthosjumper@10.147.18.191"' + +# SSH private key on jump host to connect to Admin Workstation +# ansible_ssh_private_key_file="/home/anthosjumper/.ssh/user_id_ed25519" + +ansible_user: "ubuntu" +yamldestpath: "/home/ubuntu" + +# User Cluster (UC) role specific below +uc_name: "uc2" +component_access_gcpsa: '{{ lookup("env", "GCPSA_COMACC_FILE") }}' +component_access_gcpsa_path: "component-access.json" + +# vSphere/vCenter - inherited from admin cluster by default but can be overwritten +# to place user cluster in a different vSphere datacenter/cluster +# uc_vc_fqdn: "" +# uc_vc_credfile: "credential-{{ uc_name }}.yaml" +# uc_vc_credentry: "vCenter" +# uc_vc_username: "" +# uc_vc_password: "" +# uc_vc_datacenter: "" +# uc_vc_datastore: "" +# uc_vc_cluster: "" +# uc_vc_network: "" # VM Network +# uc_vc_folder: "" # optional +# uc_vc_respool: "" # if default resourcePool use /Resources +# uc_vc_cacertpath: "" + +# Networking +uc_nw_ipallocmode: "static" # dhcp or static +uc_nw_ipfile: "{{ uc_name }}-ip-block.yaml" +uc_nw_gw: "10.20.0.1" # gateway +uc_nw_nm: "255.255.255.0" # netmask 255.255.255.0 or similar +uc_nw_ntp: "{{ glb_ntp }}" # list from group_vars/all +uc_nw_dns: "{{ glb_dns }}" # list from group_vars/all +uc_nw_searchdomains: "{{ glb_dns_searchdomains }}" # list from group_vars/all +uc_nw_servicecidr: "10.96.0.0/20" +uc_nw_podcidr: "192.168.0.0/16" +uc_nw_vc_net: "VM Network" + +# values for the file content of network.ipMode.ipBlockFilePath +uc_ipblock_netmask: "255.255.255.0" +uc_ipblock_gateway: "10.20.0.1" +uc_ipblock_ips: ["10.20.0.52", "10.20.0.53", "10.20.0.54", "10.20.0.55"] + +# Load balancing +uc_lb_kind: MetalLB +uc_lb_vips_cp: "10.20.0.50" +uc_lb_vips_ingress: "10.20.0.245" +uc_lb_metallb_ips: "10.20.0.245-10.20.0.250" + +uc_dataplanev2: true + +# masternode sizing +uc_masternode_cpus: 4 +uc_masternode_mem: 8192 +uc_masternode_replicas: 1 +uc_antiaffinitygroups: false +# uc_masternode_datastore: "" # Set this to admin cluster datastore when you want to place user cluster in a differet vSphere cluster/datacenter than the admin cluster. + +# nodepools +uc_nodepools_name: "{{ uc_name }}-pool-1" +uc_nodepools_cpus: 4 +uc_nodepools_mem: 8192 +uc_nodepools_replicas: 3 +# uc_nodepools_osimagetype: cos +uc_nodepools_osimagetype: ubuntu_containerd +uc_nodepools: +- name: "{{ uc_name | regex_replace('^(\\w*-)(.*)([au]-.*)', '\\1\\3') }}-p01" + cpus: "{{ uc_nodepools_cpus }}" # default 4 + memoryMB: "{{ uc_nodepools_mem }}" # default 8192 + replicas: "{{ uc_nodepools_replicas }}" # default 3 + osImageType: "{{ uc_nodepools_osimagetype }}" + autoscaling: + minReplicas: 3 + maxReplicas: 4 + enableLoadBalancer: true # set to true only if uc_lb_kind equals MetalLB +- name: "{{ uc_name | regex_replace('^(\\w*-)(.*)([au]-.*)', '\\1\\3') }}-p02" + cpus: "{{ uc_nodepools_cpus }}" # default 4 + memoryMB: "{{ uc_nodepools_mem }}" # default 8192 + replicas: "{{ uc_nodepools_replicas }}" # default 3 + osImageType: "{{ uc_nodepools_osimagetype }}" + autoscaling: + minReplicas: 3 + maxReplicas: 4 + enableLoadBalancer: true # set to true only if uc_lb_kind equals MetalLB + +# GCP project IDs +uc_stackdriver_projectid: "anthos-lab-01" +uc_stackdriver_clusterlocation: "us-central1" +uc_stackdriver_enablevpc: false +logging_monitoring_gcpsa_path: "logging-monitoring.json" +uc_stackdriver_disablevsphereresourcemetrics: false +uc_gkeconnect_projectid: "anthos-lab-01" +connect_register_gcpsa_path: "connect-register.json" +uc_cloudauditlogging_projectid: "anthos-lab-01" +uc_cloudauditlogging_clusterlocation: "us-central1" +audit_logging_gcpsa_path: "audit-logging.json" + +uc_autorepair: true + +# Kubernetes Secrets at-rest encryption +uc_secretsencryption_mode: "GeneratedKey" +uc_secretsencryption_keyversion: 1 + +uc_verbosity: 8 +uc_skipvalidations: "" +# uc_skipvalidations: "--skip-validation-all" +# uc_skipvalidations: "--skip-validation-load-balancer" +# uc_skipvalidations: "--skip-validation-load-balancer --skip-validation-node-ips" + +# Optional - when using private artifact repository +private_download_base: "{{ glb_private_download_base }}" +private_download_auth_header: "{{ glb_private_download_auth_header }}" + +# Optional: ASM +asm_version: "1.13" +asm_revision: "asm-1137-3" +asm_asmcli_version: "1.13.7-asm.3-config1" +asm_network_id: "uc2-6789" +# ASM - GCP related +asm_gcp_project: "asm-test" # GCP project where user cluster is registered +asm_gcp_project_number: "123456789" +asm_gcpsa_path: "asm-meshconfig.json" +asm_user_email: "asm-sa@asm-test.iam.gserviceaccount.com" + +# Optional: ACM +acm_membership: "{{ uc_name }}" # user cluster name +acm_version: "1.12.1" # version in format #.#.# +# ACM - GCP related +acm_gcpproject: "acm-test" +acm_gcpsa_path: "acm-gcpsa.json" +# ACM - Config Sync +acm_configsync: + # Set to true to install and enable Config Sync + enabled: false + # FORMAT - unstructured, hierarchy + sourceFormat: unstructured + # REPO - git repository URL + syncRepo: "" + syncBranch: "" + # TYPE - none, ssh, cookiefile, token, gcpserviceaccount, gcenode + secretType: gcpserviceaccount + # If TYPE == gcpserviceaccount, then add its email below + gcpServiceAccountEmail: acm-sa@acm-test.iam.gserviceaccount.com + # DIRECTORY - folder in git repository for ACM + policyDir: "" + # PREVENT_DRIFT - true, false + preventDrift: true +acm_root_repo_sshkey: "" # ssh file key if using acm_configsync.secretType: ssh +# ACM - PolicyController +acm_policycontroller: + # Set to true to install and enable Policy Controller + enabled: false + +# Optional: AIS +# Uncomment out this section to specify AIS configuration +# An uncommented empty block will result in errors +# ais_install: true +# ais_authentication: +# - name: oidc-ad +# oidc: +# clientID: "" +# clientSecret: "" +# cloudConsoleRedirectURI: "" +# extraParams: prompt=consent,access_type=offline +# issuerURI: "" +# kubectlRedirectURI: "" +# scopes: "" +# userClaim: "" \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/inventory/site-a/usercluster01.yml b/tools/anthosvmware-ansible-module/inventory/site-a/usercluster01.yml new file mode 100644 index 0000000000..39a17d19f1 --- /dev/null +++ b/tools/anthosvmware-ansible-module/inventory/site-a/usercluster01.yml @@ -0,0 +1,24 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +all: + hosts: + children: + userclusters: + children: + usercluster01: + hosts: + 10.20.0.39: +# your admin workstation IP diff --git a/tools/anthosvmware-ansible-module/inventory/site-a/usercluster02.yml b/tools/anthosvmware-ansible-module/inventory/site-a/usercluster02.yml new file mode 100644 index 0000000000..77174ecbf6 --- /dev/null +++ b/tools/anthosvmware-ansible-module/inventory/site-a/usercluster02.yml @@ -0,0 +1,24 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +all: + hosts: + children: + userclusters: + children: + usercluster02: + hosts: + 10.20.0.39: +# your admin workstation IP diff --git a/tools/anthosvmware-ansible-module/playbooks/acm_install.yml b/tools/anthosvmware-ansible-module/playbooks/acm_install.yml new file mode 100644 index 0000000000..a3dce06b1d --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/acm_install.yml @@ -0,0 +1,21 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Install ACM +- name: "[acm] Installation" + hosts: userclusters + gather_facts: false + roles: + - acm diff --git a/tools/anthosvmware-ansible-module/playbooks/acm_uninstall.yml b/tools/anthosvmware-ansible-module/playbooks/acm_uninstall.yml new file mode 100644 index 0000000000..bbb771d9ec --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/acm_uninstall.yml @@ -0,0 +1,22 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Uninstall ACM +- name: "[acm] Uninstall" + hosts: userclusters + gather_facts: false + roles: + - { role: acm, acm_install: false } + diff --git a/tools/anthosvmware-ansible-module/playbooks/acm_upgrade.yml b/tools/anthosvmware-ansible-module/playbooks/acm_upgrade.yml new file mode 100644 index 0000000000..776c5dd43a --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/acm_upgrade.yml @@ -0,0 +1,21 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Upgrade ACM +- name: "[acm] Upgrade" + hosts: userclusters + gather_facts: false + roles: + - acm diff --git a/tools/anthosvmware-ansible-module/playbooks/admincluster_install.yml b/tools/anthosvmware-ansible-module/playbooks/admincluster_install.yml new file mode 100644 index 0000000000..9d43210f2a --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/admincluster_install.yml @@ -0,0 +1,23 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Install Admin Cluster +- name: "[Admin Cluster] Installation" + hosts: admincluster + gather_facts: false + roles: + - admincluster + - { role: label_membership, when: gcp_labels is defined and (gcp_labels|length > 0) and ac_install|default(true)|bool } + - { role: ais, when: (ais_install|default(false)|bool) and (ac_install|default(true)|bool) } diff --git a/tools/anthosvmware-ansible-module/playbooks/admincluster_repave.yml b/tools/anthosvmware-ansible-module/playbooks/admincluster_repave.yml new file mode 100644 index 0000000000..c3be3a1ed9 --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/admincluster_repave.yml @@ -0,0 +1,20 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Repave Admin Cluster +- name: "[Admin Cluster] Trigger repave" + hosts: admincluster + gather_facts: false + roles: + - repave \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/playbooks/admincluster_uninstall.yml b/tools/anthosvmware-ansible-module/playbooks/admincluster_uninstall.yml new file mode 100644 index 0000000000..9ae2cef0bf --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/admincluster_uninstall.yml @@ -0,0 +1,20 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Uninstall Admin Cluster +- name: "[Admin Cluster] Uninstall" + hosts: admincluster + gather_facts: false + roles: + - { role: admincluster, ac_install: false } \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/playbooks/admincluster_upgrade.yml b/tools/anthosvmware-ansible-module/playbooks/admincluster_upgrade.yml new file mode 100644 index 0000000000..20d1d94a16 --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/admincluster_upgrade.yml @@ -0,0 +1,20 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Upgrade Admin Cluster +- name: "[Admin Cluster] Upgrade" + hosts: admincluster + gather_facts: false + roles: + - { role: admincluster, ac_upgrade: true } \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/playbooks/adminws_install.yml b/tools/anthosvmware-ansible-module/playbooks/adminws_install.yml new file mode 100644 index 0000000000..03da272f54 --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/adminws_install.yml @@ -0,0 +1,21 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Install Admin Workstation +- name: "[adminws] Installation" + hosts: adminws + gather_facts: false + roles: + - adminws diff --git a/tools/anthosvmware-ansible-module/playbooks/adminws_uninstall.yml b/tools/anthosvmware-ansible-module/playbooks/adminws_uninstall.yml new file mode 100644 index 0000000000..68e5e4c369 --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/adminws_uninstall.yml @@ -0,0 +1,20 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Uninstall Admin Workstation +- name: "[adminws] Uninstall" + hosts: adminws + gather_facts: false + roles: + - { role: adminws, adminws_install: false } diff --git a/tools/anthosvmware-ansible-module/playbooks/adminws_upgrade.yml b/tools/anthosvmware-ansible-module/playbooks/adminws_upgrade.yml new file mode 100644 index 0000000000..aa864470fe --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/adminws_upgrade.yml @@ -0,0 +1,20 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Upgrade Admin Workstation +- name: "[adminws] Upgrade" + hosts: adminws + gather_facts: false + roles: + - { role: adminws, adminws_upgrade: true } diff --git a/tools/anthosvmware-ansible-module/playbooks/ais_config.yml b/tools/anthosvmware-ansible-module/playbooks/ais_config.yml new file mode 100644 index 0000000000..6f99113bf5 --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/ais_config.yml @@ -0,0 +1,21 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Configure Anthos Identity Service +- name: "[ais] Configuration" + hosts: admincluster, userclusters + gather_facts: false + roles: + - ais diff --git a/tools/anthosvmware-ansible-module/playbooks/asm_install.yml b/tools/anthosvmware-ansible-module/playbooks/asm_install.yml new file mode 100644 index 0000000000..649b63aa20 --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/asm_install.yml @@ -0,0 +1,20 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[asm] Install Anthos Service Mesh" + hosts: userclusters + gather_facts: false + roles: + - asm diff --git a/tools/anthosvmware-ansible-module/playbooks/asm_uninstall.yml b/tools/anthosvmware-ansible-module/playbooks/asm_uninstall.yml new file mode 100644 index 0000000000..07ade90ffc --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/asm_uninstall.yml @@ -0,0 +1,21 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Uninstall ASM +- name: "[asm] Uninstall Anthos Service Mesh" + hosts: userclusters + gather_facts: false + roles: + - { role: asm, asm_install: false } diff --git a/tools/anthosvmware-ansible-module/playbooks/asm_upgrade.yml b/tools/anthosvmware-ansible-module/playbooks/asm_upgrade.yml new file mode 100644 index 0000000000..564998b0a4 --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/asm_upgrade.yml @@ -0,0 +1,20 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[asm] Upgrade Anthos Service Mesh" + hosts: userclusters + gather_facts: false + roles: + - asm diff --git a/tools/anthosvmware-ansible-module/playbooks/asm_upgrade_completion.yml b/tools/anthosvmware-ansible-module/playbooks/asm_upgrade_completion.yml new file mode 100644 index 0000000000..cda98e3b22 --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/asm_upgrade_completion.yml @@ -0,0 +1,19 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[asm] Complete Anthos Service Mesh Upgrade process" + hosts: userclusters + gather_facts: false + roles: + - { role: asm, asm_upgrade_completion: true } \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/playbooks/cleanup.yml b/tools/anthosvmware-ansible-module/playbooks/cleanup.yml new file mode 100644 index 0000000000..d50280407b --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/cleanup.yml @@ -0,0 +1,21 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Clean up sensitive files +- name: "[cleanup] Clean up sensitive files" + hosts: all + gather_facts: false + roles: + - cleanup diff --git a/tools/anthosvmware-ansible-module/playbooks/copy_credentials.yml b/tools/anthosvmware-ansible-module/playbooks/copy_credentials.yml new file mode 100644 index 0000000000..aea42904c8 --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/copy_credentials.yml @@ -0,0 +1,23 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Copy sensitive files +- name: "[copy_credentials] Copy sensitive files" + hosts: all + gather_facts: false + vars: + is_standalone: true + roles: + - copy_credentials diff --git a/tools/anthosvmware-ansible-module/playbooks/multiclustermesh_install.yml b/tools/anthosvmware-ansible-module/playbooks/multiclustermesh_install.yml new file mode 100644 index 0000000000..79cde6240f --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/multiclustermesh_install.yml @@ -0,0 +1,20 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[mesh] Create mesh between multiple clusters" + hosts: userclusters + gather_facts: false + roles: + - { role: asm, mesh_install: true } diff --git a/tools/anthosvmware-ansible-module/playbooks/sakeys/.gitignore b/tools/anthosvmware-ansible-module/playbooks/sakeys/.gitignore new file mode 100644 index 0000000000..a6c57f5fb2 --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/sakeys/.gitignore @@ -0,0 +1 @@ +*.json diff --git a/tools/anthosvmware-ansible-module/playbooks/sakeys/.gitkeep b/tools/anthosvmware-ansible-module/playbooks/sakeys/.gitkeep new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tools/anthosvmware-ansible-module/playbooks/upload_artifactory_install.yml b/tools/anthosvmware-ansible-module/playbooks/upload_artifactory_install.yml new file mode 100644 index 0000000000..0440a79684 --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/upload_artifactory_install.yml @@ -0,0 +1,21 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Upload Artifacts +- name: "[upload_artifactory] Upload Artifacts" + hosts: upload + gather_facts: false + roles: + - upload_artifactory diff --git a/tools/anthosvmware-ansible-module/playbooks/upload_artifactory_uninstall.yml b/tools/anthosvmware-ansible-module/playbooks/upload_artifactory_uninstall.yml new file mode 100644 index 0000000000..1a9926963d --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/upload_artifactory_uninstall.yml @@ -0,0 +1,21 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Delete Artifacts +- name: "[upload_artifactory] Delete Artifacts" + hosts: upload + gather_facts: false + roles: + - { role: upload_artifactory, upload_artifactory_install: false } diff --git a/tools/anthosvmware-ansible-module/playbooks/usercluster_install.yml b/tools/anthosvmware-ansible-module/playbooks/usercluster_install.yml new file mode 100644 index 0000000000..1d22334b28 --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/usercluster_install.yml @@ -0,0 +1,23 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Install User Cluster +- name: "[User Cluster] Installation" + hosts: userclusters + gather_facts: false + roles: + - usercluster + - { role: label_membership, when: gcp_labels is defined and (gcp_labels|length > 0) and uc_install|default(true)|bool } + - { role: ais, when: (ais_install|default(false)|bool) and (uc_install|default(true)|bool) } diff --git a/tools/anthosvmware-ansible-module/playbooks/usercluster_repave.yml b/tools/anthosvmware-ansible-module/playbooks/usercluster_repave.yml new file mode 100644 index 0000000000..0019968732 --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/usercluster_repave.yml @@ -0,0 +1,20 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Repave User Cluster +- name: "[User Cluster] Trigger repave" + hosts: userclusters + gather_facts: false + roles: + - repave \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/playbooks/usercluster_uninstall.yml b/tools/anthosvmware-ansible-module/playbooks/usercluster_uninstall.yml new file mode 100644 index 0000000000..e65ad1c8e8 --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/usercluster_uninstall.yml @@ -0,0 +1,21 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Uninstall User Cluster +- name: "[User Cluster] Uninstall" + hosts: userclusters + gather_facts: false + roles: + - { role: usercluster, uc_install: false } \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/playbooks/usercluster_upgrade.yml b/tools/anthosvmware-ansible-module/playbooks/usercluster_upgrade.yml new file mode 100644 index 0000000000..0c0ea33b43 --- /dev/null +++ b/tools/anthosvmware-ansible-module/playbooks/usercluster_upgrade.yml @@ -0,0 +1,20 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Upgrade User Cluster +- name: "[User Cluster] Upgrade" + hosts: userclusters + gather_facts: false + roles: + - { role: usercluster, uc_upgrade: true } diff --git a/tools/anthosvmware-ansible-module/policy-controller-config/README.md b/tools/anthosvmware-ansible-module/policy-controller-config/README.md new file mode 100644 index 0000000000..27778e1fcd --- /dev/null +++ b/tools/anthosvmware-ansible-module/policy-controller-config/README.md @@ -0,0 +1,11 @@ +# Policy Configuration + +This directory contains policy configuration that should be added to the ACM Config Sync repository. + +The `cis-benchmark-bundle` contains a slightly customized version of the bundle distributed by Google. +The base policies were fetched from https://github.com/GoogleCloudPlatform/acm-policy-controller-library/tree/master/bundles/cis-k8s-v1.5.1 on August 29th, 2022. +Modification are added with a comment in line on the constraints. + +# Policy Controller Configuration + +Included is the Gatekeeper config object to configure object sync for referential policies, and to start global namespace exemptions. \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.1.1_restrict-clusteradmin-rolebindings.yaml b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.1.1_restrict-clusteradmin-rolebindings.yaml new file mode 100644 index 0000000000..b8038bfa06 --- /dev/null +++ b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.1.1_restrict-clusteradmin-rolebindings.yaml @@ -0,0 +1,62 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: constraints.gatekeeper.sh/v1beta1 +kind: K8sRestrictRoleBindings +metadata: + name: cis-k8s-v1.5.1-restrict-clusteradmin-rolebindings + labels: + policycontroller.gke.io/bundleName: cis-k8s-v1.5.1 + annotations: + # This constraint is not certified by CIS. + policycontroller.gke.io/constraintData: | + "{ + bundleName: 'cis-k8s-v1.5.1', + bundleDisplayName: 'CIS Kubernetes Benchmark 1.5.1', + bundleLink: 'https://cloud.google.com/anthos-config-management/docs/how-to/using-cis-k8s-benchmark', + bundleVersion: '202208.1', + bundleDescription: 'Use the CIS Kubernetes Benchmark 1.5.1 policy bundle with Anthos Policy Controller to evaluate the compliance of your in cluster resources against the CIS Kubernetes Benchmark, which is a set of recommendations for configuring Kubernetes to support a robust security posture.', + controlNumbers: '[5.1.1]', + severity: 'UNASSIGNED', + description: 'Restricts the use of the `cluster-admin` role.', + remediation: 'The use of the ‘cluster-admin’ `ClusterRole` for Role-based access control (RBAC) is restricted, please use a different role. See ‘Using RBAC Authorization’ for more information: https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles', + minimumTemplateLibraryVersion: '1.10.1' + }" +spec: + enforcementAction: dryrun + parameters: + restrictedRole: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: cluster-admin + allowedSubjects: + - apiGroup: rbac.authorization.k8s.io + kind: Group + name: system:masters + # for GKE Hub feature authorizer + - apiGroup: rbac.authorization.k8s.io + kind: User + name: ^service-[0-9]+@gcp-sa-gkehub.iam.gserviceaccount.com$ + regexMatch: true + # for ACM to install in-cluster resources + - apiGroup: rbac.authorization.k8s.io + kind: User + name: ^service-[0-9]+@gcp-sa-anthosconfigmanagement.iam.gserviceaccount.com$ + regexMatch: true + # for Config Sync reconciler manager service account + - kind: ServiceAccount + name: reconciler-manager + # Modification: for Config Sync root reconciler service account + - kind: ServiceAccount + name: root-reconciler diff --git a/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.1.3_prohibit-role-wildcard-access.yaml b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.1.3_prohibit-role-wildcard-access.yaml new file mode 100644 index 0000000000..0ebd07aa72 --- /dev/null +++ b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.1.3_prohibit-role-wildcard-access.yaml @@ -0,0 +1,80 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: constraints.gatekeeper.sh/v1beta1 +kind: K8sProhibitRoleWildcardAccess +metadata: + name: cis-k8s-v1.5.1-prohibit-role-wildcard-access + labels: + policycontroller.gke.io/bundleName: cis-k8s-v1.5.1 + annotations: + # This constraint is not certified by CIS. + policycontroller.gke.io/constraintData: | + "{ + bundleName: 'cis-k8s-v1.5.1', + bundleDisplayName: 'CIS Kubernetes Benchmark 1.5.1', + bundleLink: 'https://cloud.google.com/anthos-config-management/docs/how-to/using-cis-k8s-benchmark', + bundleVersion: '202208.1', + bundleDescription: 'Use the CIS Kubernetes Benchmark 1.5.1 policy bundle with Anthos Policy Controller to evaluate the compliance of your in cluster resources against the CIS Kubernetes Benchmark, which is a set of recommendations for configuring Kubernetes to support a robust security posture.', + controlNumbers: '[5.1.3]', + severity: 'UNASSIGNED', + description: 'Restricts the use of wildcards in `Roles` and `ClusterRoles`.', + remediation: 'The use of wildcards ‘*’ in `Roles` and `ClusterRoles` is restricted, please remove all wildcards from your Roles and ClusterRoles. See ‘RoleBinding and ClusterRoleBinding’ for more information: https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding', + minimumTemplateLibraryVersion: '1.12.0' + }" +spec: + enforcementAction: dryrun + match: + excludedNamespaces: + - gke-connect + - resource-group-system + parameters: + exemptions: + clusterRoles: + # Modification: Added cilium ClusterRoles + - name: cilium + - name: cilium-operator + - name: config-management-operator + - name: configsync.gke.io:ns-reconciler + - name: cluster-admin + - name: external-metrics-reader + - name: gatekeeper-manager-role + - name: ^istio-reader-clusterrole-asm-[0-9]{4}-[0-9]-istio-system$ + regexMatch: true + - name: istio-reader-istio-system + - name: ^istiod-clusterrole-asm-[0-9]{4}-[0-9]-istio-system$ + regexMatch: true + - name: istiod-istio-system + # Modification: Added istiod for GKE system ClusterRole + - name: istiod-gke-system + - name: kubelet-api-admin + - name: metering + - name: resource-group-manager-role + - name: servicemesh + # Modification: Added stackdriver operator ClusterRole + - name: stackdriver-operator + - name: system:controller:disruption-controller + - name: system:controller:generic-garbage-collector + - name: system:controller:horizontal-pod-autoscaler + - name: system:controller:namespace-controller + - name: system:controller:resourcequota-controller + - name: system:gcp-controller-manager + - name: system:gke-common-webhooks + - name: system:gke-hpa-actor + - name: system:gke-master-resourcequota + - name: system:glbc-status + - name: system:kube-controller-manager + - name: system:kubestore-collector + - name: system:kubelet-api-admin + - name: system:managed-certificate-controller diff --git a/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.1_psp-privileged-container.yaml b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.1_psp-privileged-container.yaml new file mode 100644 index 0000000000..b33c84b7e8 --- /dev/null +++ b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.1_psp-privileged-container.yaml @@ -0,0 +1,45 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: constraints.gatekeeper.sh/v1beta1 +kind: K8sPSPPrivilegedContainer +metadata: + name: cis-k8s-v1.5.1-psp-privileged-container + labels: + policycontroller.gke.io/bundleName: cis-k8s-v1.5.1 + annotations: + # This constraint is not certified by CIS. + policycontroller.gke.io/constraintData: | + "{ + bundleName: 'cis-k8s-v1.5.1', + bundleDisplayName: 'CIS Kubernetes Benchmark 1.5.1', + bundleLink: 'https://cloud.google.com/anthos-config-management/docs/how-to/using-cis-k8s-benchmark', + bundleVersion: '202208.1', + bundleDescription: 'Use the CIS Kubernetes Benchmark 1.5.1 policy bundle with Anthos Policy Controller to evaluate the compliance of your in cluster resources against the CIS Kubernetes Benchmark, which is a set of recommendations for configuring Kubernetes to support a robust security posture.', + controlNumbers: '[5.2.1]', + severity: 'UNASSIGNED', + description: 'Restricts containers with `securityContext.privileged` set to `true`.', + remediation: 'Containers are restricted from running with `securityContext.privileged` set to `true`, please remove the `securityContext.privileged: true` from your containers. See ‘Configure a Security Context for a Pod or Container’ for more information: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/', + minimumTemplateLibraryVersion: '1.11.1' + }" +spec: + enforcementAction: dryrun + match: + kinds: + - apiGroups: + - "" + kinds: + - Pod + excludedNamespaces: + - kube-system diff --git a/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.2-5.2.3_psp-host-namespace.yaml b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.2-5.2.3_psp-host-namespace.yaml new file mode 100644 index 0000000000..1c8cfeed35 --- /dev/null +++ b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.2-5.2.3_psp-host-namespace.yaml @@ -0,0 +1,45 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: constraints.gatekeeper.sh/v1beta1 +kind: K8sPSPHostNamespace +metadata: + name: cis-k8s-v1.5.1-psp-host-namespace + labels: + policycontroller.gke.io/bundleName: cis-k8s-v1.5.1 + annotations: + # This constraint is not certified by CIS. + policycontroller.gke.io/constraintData: | + "{ + bundleName: 'cis-k8s-v1.5.1', + bundleDisplayName: 'CIS Kubernetes Benchmark 1.5.1', + bundleLink: 'https://cloud.google.com/anthos-config-management/docs/how-to/using-cis-k8s-benchmark', + bundleVersion: '202208.1', + bundleDescription: 'Use the CIS Kubernetes Benchmark 1.5.1 policy bundle with Anthos Policy Controller to evaluate the compliance of your in cluster resources against the CIS Kubernetes Benchmark, which is a set of recommendations for configuring Kubernetes to support a robust security posture.', + controlNumbers: '[5.2.2,5.2.3]', + severity: 'UNASSIGNED', + description: 'Prohibits containers from running with `hostPID` or `hostIPC` set to `true`.', + remediation: 'Containers are restricted from running with `hostPID` or `hostIPC` set to `true`, please remove the `hostPID: true` and/or `hostIPC: true` from your containers.', + minimumTemplateLibraryVersion: '1.11.1' + }" +spec: + enforcementAction: dryrun + match: + kinds: + - apiGroups: + - "" + kinds: + - Pod + excludedNamespaces: + - kube-system diff --git a/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.4_psp-host-network-ports.yaml b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.4_psp-host-network-ports.yaml new file mode 100644 index 0000000000..40b035cfd3 --- /dev/null +++ b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.4_psp-host-network-ports.yaml @@ -0,0 +1,49 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: constraints.gatekeeper.sh/v1beta1 +kind: K8sPSPHostNetworkingPorts +metadata: + name: cis-k8s-v1.5.1-psp-host-network-ports + labels: + policycontroller.gke.io/bundleName: cis-k8s-v1.5.1 + annotations: + # This constraint is not certified by CIS. + policycontroller.gke.io/constraintData: | + "{ + bundleName: 'cis-k8s-v1.5.1', + bundleDisplayName: 'CIS Kubernetes Benchmark 1.5.1', + bundleLink: 'https://cloud.google.com/anthos-config-management/docs/how-to/using-cis-k8s-benchmark', + bundleVersion: '202208.1', + bundleDescription: 'Use the CIS Kubernetes Benchmark 1.5.1 policy bundle with Anthos Policy Controller to evaluate the compliance of your in cluster resources against the CIS Kubernetes Benchmark, which is a set of recommendations for configuring Kubernetes to support a robust security posture.', + controlNumbers: '[5.2.4]', + severity: 'UNASSIGNED', + description: 'Restricts containers from running with the `hostNetwork` flag set to `true`.', + remediation: 'Containers are restricted from running with `hostNetwork` set to `true`, please remove the `hostNetwork: true` from your containers.', + minimumTemplateLibraryVersion: '1.11.1' + }" +spec: + enforcementAction: dryrun + match: + kinds: + - apiGroups: + - "" + kinds: + - Pod + excludedNamespaces: + - kube-system + parameters: + hostNetwork: false + min: 1 + max: 65535 diff --git a/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.5_psp-allow-privilege-escalation-container.yaml b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.5_psp-allow-privilege-escalation-container.yaml new file mode 100644 index 0000000000..d43cba7ddc --- /dev/null +++ b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.5_psp-allow-privilege-escalation-container.yaml @@ -0,0 +1,51 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: constraints.gatekeeper.sh/v1beta1 +kind: K8sPSPAllowPrivilegeEscalationContainer +metadata: + name: cis-k8s-v1.5.1-psp-allow-privilege-escalation + labels: + policycontroller.gke.io/bundleName: cis-k8s-v1.5.1 + annotations: + # This constraint is not certified by CIS. + policycontroller.gke.io/constraintData: | + "{ + bundleName: 'cis-k8s-v1.5.1', + bundleDisplayName: 'CIS Kubernetes Benchmark 1.5.1', + bundleLink: 'https://cloud.google.com/anthos-config-management/docs/how-to/using-cis-k8s-benchmark', + bundleVersion: '202208.1', + bundleDescription: 'Use the CIS Kubernetes Benchmark 1.5.1 policy bundle with Anthos Policy Controller to evaluate the compliance of your in cluster resources against the CIS Kubernetes Benchmark, which is a set of recommendations for configuring Kubernetes to support a robust security posture.', + controlNumbers: '[5.2.5]', + severity: 'UNASSIGNED', + description: 'Restricts containers with `allowPrivilegeEscalation` set to `true`.', + remediation: 'Containers are restricted from running with `allowPrivilegeEscalation` set to `true`, please remove the `allowPrivilegeEscalation: true` from your containers.', + minimumTemplateLibraryVersion: '1.11.1' + }" +spec: + enforcementAction: dryrun + match: + kinds: + - apiGroups: + - "" + kinds: + - Pod + excludedNamespaces: + - kube-system + - istio-system + - asm-system + - config-management-system + - config-management-monitoring + - resource-group-system + - cert-manager diff --git a/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.6_psp-restrict_root_containers.yaml b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.6_psp-restrict_root_containers.yaml new file mode 100644 index 0000000000..f29d434456 --- /dev/null +++ b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.6_psp-restrict_root_containers.yaml @@ -0,0 +1,57 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: constraints.gatekeeper.sh/v1beta1 +kind: K8sPSPAllowedUsers +metadata: + name: cis-k8s-v1.5.1-psp-pods-must-run-as-nonroot + labels: + policycontroller.gke.io/bundleName: cis-k8s-v1.5.1 + annotations: + # This constraint is not certified by CIS. + policycontroller.gke.io/constraintData: | + "{ + bundleName: 'cis-k8s-v1.5.1', + bundleDisplayName: 'CIS Kubernetes Benchmark 1.5.1', + bundleLink: 'https://cloud.google.com/anthos-config-management/docs/how-to/using-cis-k8s-benchmark', + bundleVersion: '202208.1', + bundleDescription: 'Use the CIS Kubernetes Benchmark 1.5.1 policy bundle with Anthos Policy Controller to evaluate the compliance of your in cluster resources against the CIS Kubernetes Benchmark, which is a set of recommendations for configuring Kubernetes to support a robust security posture.', + controlNumbers: '[5.2.6]', + severity: 'UNASSIGNED', + description: 'Restricts containers from running as the root user.', + remediation: 'Containers are restricted from running with `runAsUser` set to `root`, please set your containers `runAsUser: ` to a non-root user ID. See ‘Set the security context for a Pod’ for more information: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod', + minimumTemplateLibraryVersion: '1.11.1' + }" +spec: + enforcementAction: dryrun + match: + kinds: + - apiGroups: + - "" + kinds: + - Pod + excludedNamespaces: + - kube-system + - asm-system + - config-management-system + - cert-manager + parameters: + # Modification: Added parameters.exemptImages to ensure ASM proxies initContainers do not cause violations + exemptImages: + - "gcr.io/gke-release/asm/proxyv2:*" + - "artifactory-dev.example.com/anthosdockerlocal/proxyv2:*" + - "artifactory-uat.example.com/anthosdockerlocal/proxyv2:*" + - "artifactory.example.com/anthosdockerlocal/proxyv2:*" + runAsUser: + rule: MustRunAsNonRoot diff --git a/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.7-5.2.8-5.2.9_psp-capabilities.yaml b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.7-5.2.8-5.2.9_psp-capabilities.yaml new file mode 100644 index 0000000000..3a1ab8b926 --- /dev/null +++ b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.2.7-5.2.8-5.2.9_psp-capabilities.yaml @@ -0,0 +1,61 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: constraints.gatekeeper.sh/v1beta1 +kind: K8sPSPCapabilities +metadata: + name: cis-k8s-v1.5.1-psp-capabilities + labels: + policycontroller.gke.io/bundleName: cis-k8s-v1.5.1 + annotations: + # This constraint is not certified by CIS. + policycontroller.gke.io/constraintData: | + "{ + bundleName: 'cis-k8s-v1.5.1', + bundleDisplayName: 'CIS Kubernetes Benchmark 1.5.1', + bundleLink: 'https://cloud.google.com/anthos-config-management/docs/how-to/using-cis-k8s-benchmark', + bundleVersion: '202208.1', + bundleDescription: 'Use the CIS Kubernetes Benchmark 1.5.1 policy bundle with Anthos Policy Controller to evaluate the compliance of your in cluster resources against the CIS Kubernetes Benchmark, which is a set of recommendations for configuring Kubernetes to support a robust security posture.', + controlNumbers: '[5.2.7,5.2.8,5.2.9]', + severity: 'UNASSIGNED', + description: 'Requires containers to drop the `NET_RAW` capability; containers may not have allowedCapabilities set to anything other than an empty array.', + remediation: 'Secrets as environment variables are prohibited in container definitions, please use a mounted secret file in a data volume. See ‘Using Secrets as files from a Pod’ for more information: https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-files-from-a-pod', + minimumTemplateLibraryVersion: '1.10.2' + }" +spec: + enforcementAction: dryrun + match: + kinds: + - apiGroups: + - "" + kinds: + - Pod + excludedNamespaces: + - kube-system + - config-management-system + - config-management-monitoring + - resource-group-system + - asm-system + - gke-connect + - cert-manager + parameters: + # Modification: Added parameters.exemptImages to ensure ASM proxies initContainers do not cause violations + exemptImages: + - "gcr.io/gke-release/asm/proxyv2:*" + - "artifactory-dev.example.com/anthosdockerlocal/proxyv2:*" + - "artifactory-uat.example.com/anthosdockerlocal/proxyv2:*" + - "artifactory.example.com/anthosdockerlocal/proxyv2:*" + allowedCapabilities: [] + requiredDropCapabilities: + - NET_RAW diff --git a/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.3.2_require-namespace-network-policies.yaml b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.3.2_require-namespace-network-policies.yaml new file mode 100644 index 0000000000..1266059da1 --- /dev/null +++ b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.3.2_require-namespace-network-policies.yaml @@ -0,0 +1,51 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: constraints.gatekeeper.sh/v1beta1 +kind: K8sRequireNamespaceNetworkPolicies +metadata: + name: cis-k8s-v1.5.1-require-namespace-network-policies + labels: + policycontroller.gke.io/bundleName: cis-k8s-v1.5.1 + annotations: + # This constraint is not certified by CIS. + policycontroller.gke.io/constraintData: | + "{ + bundleName: 'cis-k8s-v1.5.1', + bundleDisplayName: 'CIS Kubernetes Benchmark 1.5.1', + bundleLink: 'https://cloud.google.com/anthos-config-management/docs/how-to/using-cis-k8s-benchmark', + bundleVersion: '202208.1', + bundleDescription: 'Use the CIS Kubernetes Benchmark 1.5.1 policy bundle with Anthos Policy Controller to evaluate the compliance of your in cluster resources against the CIS Kubernetes Benchmark, which is a set of recommendations for configuring Kubernetes to support a robust security posture.', + controlNumbers: '[5.3.2]', + severity: 'UNASSIGNED', + description: 'Requires that every namespace defined in the cluster has a NetworkPolicy.', + remediation: 'The use of a namespace in the cluster without a NetworkPolicy is resticted, please add a NetworkPolicy to your namespace. See ‘Network Policies’ for more information: https://kubernetes.io/docs/concepts/services-networking/network-policies/', + minimumTemplateLibraryVersion: '1.10.1' + }" +spec: + # This should always stay 'dryrun' since it is an audit only policy. A NetworkPolicy can only be created after the Namespace. + enforcementAction: dryrun + match: + excludedNamespaces: + - kube-system + - kube-node-lease + - kube-public + - gatekeeper-system + - config-management-system + - config-management-monitoring + - resource-group-system + - gke-connect + - istio-system + - asm-system + - default diff --git a/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.4.1_no-secrets-as-env-vars.yaml b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.4.1_no-secrets-as-env-vars.yaml new file mode 100644 index 0000000000..f1f26e1769 --- /dev/null +++ b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.4.1_no-secrets-as-env-vars.yaml @@ -0,0 +1,41 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: constraints.gatekeeper.sh/v1beta1 +kind: K8sNoEnvVarSecrets +metadata: + name: cis-k8s-v1.5.1-no-secrets-as-env-vars + labels: + policycontroller.gke.io/bundleName: cis-k8s-v1.5.1 + annotations: + # This constraint is not certified by CIS. + policycontroller.gke.io/constraintData: | + "{ + bundleName: 'cis-k8s-v1.5.1', + bundleDisplayName: 'CIS Kubernetes Benchmark 1.5.1', + bundleLink: 'https://cloud.google.com/anthos-config-management/docs/how-to/using-cis-k8s-benchmark', + bundleVersion: '202208.1', + bundleDescription: 'Use the CIS Kubernetes Benchmark 1.5.1 policy bundle with Anthos Policy Controller to evaluate the compliance of your in cluster resources against the CIS Kubernetes Benchmark, which is a set of recommendations for configuring Kubernetes to support a robust security posture.', + controlNumbers: '[5.4.1]', + severity: 'UNASSIGNED', + description: 'Prohibits secrets as environment variables in container definitions; instead, use mounted secret files in data volumes.', + remediation: 'Secrets as environment variables are prohibited in container definitions, please use a mounted secret file in a data volume. See "Using Secrets as files from a Pod" for more information https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-files-from-a-pod', + minimumTemplateLibraryVersion: '1.10.2' + }" +spec: + enforcementAction: dryrun + match: + excludedNamespaces: + - config-management-system + - gke-connect diff --git a/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.7.2_seccomp-default.yaml b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.7.2_seccomp-default.yaml new file mode 100644 index 0000000000..9942a5680c --- /dev/null +++ b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.7.2_seccomp-default.yaml @@ -0,0 +1,63 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: constraints.gatekeeper.sh/v1beta1 +kind: K8sPSPSeccomp +metadata: + name: cis-k8s-v1.5.1-psp-seccomp-default + labels: + policycontroller.gke.io/bundleName: cis-k8s-v1.5.1 + annotations: + # This constraint is not certified by CIS. + policycontroller.gke.io/constraintData: | + "{ + bundleName: 'cis-k8s-v1.5.1', + bundleDisplayName: 'CIS Kubernetes Benchmark 1.5.1', + bundleLink: 'https://cloud.google.com/anthos-config-management/docs/how-to/using-cis-k8s-benchmark', + bundleVersion: '202208.1', + bundleDescription: 'Use the CIS Kubernetes Benchmark 1.5.1 policy bundle with Anthos Policy Controller to evaluate the compliance of your in cluster resources against the CIS Kubernetes Benchmark, which is a set of recommendations for configuring Kubernetes to support a robust security posture.', + controlNumbers: '[5.7.2]', + severity: 'UNASSIGNED', + description: 'Requires that Pods have seccomp profile set to runtime/default or docker/default.', + remediation: 'Containers are restricted from running with `seccomp` other than `runtime/default` or `docker/default`, please remote or set your containers `seccomp` annotation to `runtime/default`. See ‘Set the Seccomp Profile for a Container’ for more information: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-seccomp-profile-for-a-containe', + minimumTemplateLibraryVersion: '1.11.1' + }" +spec: + enforcementAction: dryrun + match: + kinds: + - apiGroups: + - "" + kinds: + - Pod + excludedNamespaces: + - kube-system + - gatekeeper-system + - config-management-monitoring + - config-management-system + - cert-manager + - asm-system + - resource-group-system + - istio-system + - gke-connect + parameters: + # Modification: Added parameters.exemptImages to ensure ASM proxies initContainers do not cause violations + exemptImages: + - "gcr.io/gke-release/asm/proxyv2:*" + - "artifactory-dev.example.com/anthosdockerlocal/proxyv2:*" + - "artifactory-uat.example.com/anthosdockerlocal/proxyv2:*" + - "artifactory.example.com/anthosdockerlocal/proxyv2:*" + allowedProfiles: + - runtime/default + - docker/default diff --git a/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.7.3_pods-require-security-context.yaml b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.7.3_pods-require-security-context.yaml new file mode 100644 index 0000000000..ed6c011947 --- /dev/null +++ b/tools/anthosvmware-ansible-module/policy-controller-config/cis-benchmark-bundle/5.7.3_pods-require-security-context.yaml @@ -0,0 +1,42 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: constraints.gatekeeper.sh/v1beta1 +kind: K8sPodsRequireSecurityContext +metadata: + name: cis-k8s-v1.5.1-pods-require-security-context + labels: + policycontroller.gke.io/bundleName: cis-k8s-v1.5.1 + annotations: + # This constraint is not certified by CIS. + policycontroller.gke.io/constraintData: | + "{ + bundleName: 'cis-k8s-v1.5.1', + bundleDisplayName: 'CIS Kubernetes Benchmark 1.5.1', + bundleLink: 'https://cloud.google.com/anthos-config-management/docs/how-to/using-cis-k8s-benchmark', + bundleVersion: '202208.1', + bundleDescription: 'Use the CIS Kubernetes Benchmark 1.5.1 policy bundle with Anthos Policy Controller to evaluate the compliance of your in cluster resources against the CIS Kubernetes Benchmark, which is a set of recommendations for configuring Kubernetes to support a robust security posture.', + controlNumbers: '[5.7.3]', + severity: 'UNASSIGNED', + description: 'Requires that Pods must have a `securityContext` defined.', + remediation: 'Containers are restricted from running without a `securityContext` defined, please set your containers `securityContext: `. See ‘Set the security context for a Pod’ for more information: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod', + minimumTemplateLibraryVersion: '1.10.2' + }" +spec: + enforcementAction: dryrun + match: + excludedNamespaces: + - kube-system + - config-management-system + - asm-system diff --git a/tools/anthosvmware-ansible-module/policy-controller-config/policy-controller-config.yaml b/tools/anthosvmware-ansible-module/policy-controller-config/policy-controller-config.yaml new file mode 100644 index 0000000000..e00e9f77bc --- /dev/null +++ b/tools/anthosvmware-ansible-module/policy-controller-config/policy-controller-config.yaml @@ -0,0 +1,41 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: config.gatekeeper.sh/v1alpha1 +kind: Config +metadata: + name: config + namespace: gatekeeper-system +spec: + match: + - excludedNamespaces: + - anthos-identity-service + - asm-ingress + - asm-system + - config-management-monitoring + - config-management-system + - gatekeeper-system + - gke-connect + - gke-system + - istio-system + - kube-node-lease + - kube-system + - resource-group-system + processes: ["audit", "webhook", "sync","mutation-webhook"] + sync: + syncOnly: + # Required for CIS namespaces require network policies + - group: "networking.k8s.io" + version: "v1" + kind: "NetworkPolicy" diff --git a/tools/anthosvmware-ansible-module/roles/acm/README.md b/tools/anthosvmware-ansible-module/roles/acm/README.md new file mode 100644 index 0000000000..cd1847c159 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/acm/README.md @@ -0,0 +1,118 @@ +# Ansible Role: acm + +Install and configures Anthos Config Management (ACM). Can also upgrade ACM. + +## Requirements + +No special requirements. + +## Role Variables + +Available variables are listed below, along with default values (see `defaults/main.yml`): + +``` +acm_install: true +``` + +Toggles installation and deinstallation. + +``` +acm_membership: "" # user cluster name +``` + +Usually, this equals the user cluster name. + +``` +acm_config: "/home/ubuntu/{{ acm_membership }}-apply-spec.yaml" +``` + +ACM configuration file that is applied with `gcloud`. + +``` +acm_version: "" # version in format #.##.# +``` + +ACM version in the format `#.##.#`, e.g., `1.12.1`. + +``` +acm_gcpproject: "" +``` + +Google Cloud Project where the user cluster is registered. + +``` +acm_gcpsa: "" +acm_gcpsa_path: "acm-gcpsa.json" +``` + +Source and destination for the Google Service Account that can enable the ACM feature. + +``` +acm_root_repo_sshkey: "" +``` + +SSH private key to access the ACM git repository. + +``` +acm_configsync: + # Set to true to install and enable Config Sync + enabled: false + # If you don't have a Git repository, omit the following fields. You + # can configure them later. + # FORMAT - unstructured, hierarchy + sourceFormat: FORMAT + # REPO - git repository URL + syncRepo: REPO + syncBranch: BRANCH + # TYPE - none, ssh, cookiefile, token, gcpserviceaccount, gcenode + secretType: TYPE + # If TYPE == gcpserviceaccount, then add its email below + gcpServiceAccountEmail: EMAIL + # DIRECTORY - folder in git repository for ACM + policyDir: DIRECTORY + # the `preventDrift` field is supported in Anthos Config Management version 1.10.0 and later. + # PREVENT_DRIFT - true, false + preventDrift: PREVENT_DRIFT +``` + +Configures the Config Sync component of ACM. + +``` +acm_policycontroller: + # Set to true to install and enable Policy Controller + enabled: false + # Uncomment to prevent the template library from being installed + # templateLibraryInstalled: false + # Uncomment to enable support for referential constraints + # referentialRulesEnabled: true + # Uncomment to disable audit, adjust value to set audit interval + # auditIntervalSeconds: 0 + # Uncomment to log all denies and dryrun failures + # logDeniesEnabled: true + # Uncomment to exempt namespaces + # exemptableNamespaces: ["namespace-name"] + # Uncomment to enable mutation + # mutation: + # # enabled: true +``` + +Configures the Policy Controller component of ACM. + +## Dependencies + +None. + +## Example Playbook + +``` +- name: "[ACM] Installation" + hosts: all + gather_facts: False + roles: + - acm +``` + +## **License** + +Copyright 2022 Google LLC. This software is provided as-is, without warranty or representation for any use or purpose. +Your use of it is subject to your agreement with Google. diff --git a/tools/anthosvmware-ansible-module/roles/acm/defaults/main.yml b/tools/anthosvmware-ansible-module/roles/acm/defaults/main.yml new file mode 100644 index 0000000000..ac3f18432b --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/acm/defaults/main.yml @@ -0,0 +1,97 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Role default values +acm_install: true +acm_membership: "" # user cluster name +acm_config: "/home/ubuntu/{{ acm_membership }}-apply-spec.yaml" +acm_version: "" # version in format #.#.# +acm_gcpproject: "" +acm_gcpsa: '{{ lookup("env", "GCPSA_ASMCFG_FILE") }}' +acm_gcpsa_path: "acm-gcpsa.json" +acm_root_repo_sshkey: "" + +# Config Sync details +acm_configsync: + # Set to true to install and enable Config Sync + enabled: false + # If you don't have a Git repository, omit the following fields. You + # can configure them later. + # FORMAT - unstructured, hierarchy + sourceFormat: FORMAT + # REPO - git repository URL + syncRepo: REPO + syncBranch: BRANCH + # TYPE - none, ssh, cookiefile, token, gcpserviceaccount, gcenode + secretType: TYPE + # If TYPE == gcpserviceaccount, then add its email below + gcpServiceAccountEmail: EMAIL + # DIRECTORY - folder in git repository for ACM + policyDir: DIRECTORY + # the `preventDrift` field is supported in Anthos Config Management version 1.10.0 and later. + # PREVENT_DRIFT - true, false + preventDrift: PREVENT_DRIFT + +# Policy controller configuration +acm_policycontroller: + # Set to true to install and enable Policy Controller + enabled: false + # Uncomment to prevent the template library from being installed + # templateLibraryInstalled: false + # Uncomment to enable support for referential constraints + # referentialRulesEnabled: true + # Uncomment to disable audit, adjust value to set audit interval + # auditIntervalSeconds: 0 + # Uncomment to log all denies and dryrun failures + # logDeniesEnabled: true + # Uncomment to exempt namespaces + # exemptableNamespaces: ["namespace-name"] + # Uncomment to enable mutation + # mutation: + # # enabled: true + ## private registry + +# Source: https://cloud.google.com/anthos-config-management/docs/how-to/updating-private-registry +acm_container_images: + "1.12.1": + - gcr.io/config-management-release/config-management-operator:20220707000119-op + - gcr.io/config-management-release/admission-webhook:v1.12.1-rc.3 + - gcr.io/config-management-release/gatekeeper:anthos1.12.1-c370036.g0 + - gcr.io/config-management-release/git-sync:v3.5.1-gke.0__linux_amd64 + - gcr.io/config-management-release/gke-hc-manager:v0.11.0-hc.2 + - gcr.io/config-management-release/hnc-manager:hnc-v0.8.0-hc.3 + - gcr.io/config-management-release/hydration-controller:v1.12.1-rc.3 + - gcr.io/config-management-release/kube-rbac-proxy:v0.5.0 + - gcr.io/config-management-release/nomos:v1.12.1-rc.3 + - gcr.io/config-management-release/oci-sync:v1.12.1-rc.3 + - gcr.io/config-management-release/otelcontribcol:v0.38.0 + - gcr.io/config-management-release/reconciler-manager:v1.12.1-rc.3 + - gcr.io/config-management-release/reconciler:v1.12.1-rc.3 + - gcr.io/config-management-release/resource-group-controller:v1.0.5 + "1.13.0": + - gcr.io/config-management-release/config-management-operator:20220831042821-op + - gcr.io/config-management-release/admission-webhook:v1.13.0-rc.7 + - gcr.io/config-management-release/gatekeeper:anthos1.13.0-368961a.g0 + - gcr.io/config-management-release/git-sync:v3.6.1-gke.0__linux_amd64 + - gcr.io/config-management-release/gke-hc-manager:v0.11.0-hc.2 + - gcr.io/config-management-release/hnc-manager:hnc-v0.8.0-hc.3 + - gcr.io/config-management-release/hydration-controller:v1.13.0-rc.7 + - gcr.io/config-management-release/kube-rbac-proxy:v0.5.0 + - gcr.io/config-management-release/nomos:v1.13.0-rc.7 + - gcr.io/config-management-release/oci-sync:v1.13.0-rc.7 + - gcr.io/config-management-release/otelcontribcol:v0.54.0 + - gcr.io/config-management-release/reconciler-manager:v1.13.0-rc.7 + - gcr.io/config-management-release/reconciler:v1.13.0-rc.7 + - gcr.io/config-management-release/resource-group-controller:v1.0.8 diff --git a/tools/anthosvmware-ansible-module/roles/acm/files/namespace.yaml b/tools/anthosvmware-ansible-module/roles/acm/files/namespace.yaml new file mode 100644 index 0000000000..f188df3764 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/acm/files/namespace.yaml @@ -0,0 +1,19 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +apiVersion: v1 +kind: Namespace +metadata: + name: config-management-system diff --git a/tools/anthosvmware-ansible-module/roles/acm/tasks/asserts.yml b/tools/anthosvmware-ansible-module/roles/acm/tasks/asserts.yml new file mode 100644 index 0000000000..37e89694b6 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/acm/tasks/asserts.yml @@ -0,0 +1,43 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[acm] Sanity Checks" + ansible.builtin.assert: + that: + - yamldestpath | length >= 6 + - sakeyfolder_base is defined + - acm_membership | length >= 3 + - acm_config | length >= 6 + - acm_gcpproject | length >= 6 + - acm_version | length >= 3 + fail_msg: "Undefined variables." + success_msg: "All required variables are defined." + +- name: "[acm] Ensure container images are available for '{{ acm_version }}' when private registry" + ansible.builtin.assert: + that: + - acm_container_images is defined + - acm_version in acm_container_images + - acm_container_images[acm_version] | type_debug == 'list' + - acm_container_images[acm_version] | length > 0 + fail_msg: "Please add ACM version '{{ acm_version }}' container images to acm_container_images map" + success_msg: "ACM version '{{ acm_version }}' found in acm_container_images map" + when: + - glb_privatereg_url is defined + - glb_privatereg_url|length > 3 + - glb_privatereg_username is defined + - glb_privatereg_username | length > 3 + - glb_privatereg_password is defined + - glb_privatereg_password | length > 3 diff --git a/tools/anthosvmware-ansible-module/roles/acm/tasks/install.yml b/tools/anthosvmware-ansible-module/roles/acm/tasks/install.yml new file mode 100644 index 0000000000..253d3c7cfb --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/acm/tasks/install.yml @@ -0,0 +1,168 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[acm] Copy SSH key for authenticating to root repository" + ansible.builtin.template: + src: ssh-key.j2 + dest: "{{ yamldestpath }}/{{ uc_name }}/acm-sshkey" + mode: 0600 + when: + - acm_configsync.secretType is defined + - acm_configsync.secretType == "ssh" + +- name: "[acm] Block - Create NS, git-creds secret, and configure ACM" # noqa var-spacing + block: + - name: "[acm] Copy ACM namespace file" + ansible.builtin.copy: + src: namespace.yaml + dest: "{{ yamldestpath }}/{{ uc_name }}" + mode: 0600 + + - name: "[acm] Create ACM namespace" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ yamldestpath }}/{{ uc_name }}-kubeconfig" + - apply + - -f + - "{{ yamldestpath }}/{{ uc_name }}/namespace.yaml" + + - name: "[acm] Check for root repository SSH key secret" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ yamldestpath }}/{{ uc_name }}-kubeconfig" + - get + - secret + - -n + - config-management-system + - git-creds + changed_when: false + failed_when: false + when: + - acm_configsync.secretType is defined + - acm_configsync.secretType == "ssh" + register: git_secret_status + + - name: "[acm] Add root repository SSH key secret" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ yamldestpath }}/{{ uc_name }}-kubeconfig" + - create + - secret + - generic + - git-creds + - -n + - config-management-system + - "--from-file=ssh={{ yamldestpath }}/{{ uc_name }}/acm-sshkey" + when: + - acm_configsync.secretType is defined + - acm_configsync.secretType == "ssh" + - "'secrets \"git-creds\" not found' in git_secret_status.stderr" + + - name: "[acm] Upload ACM Images to Private Registry" + include_tasks: private-upload.yml + when: + - glb_privatereg_url is defined + - glb_privatereg_url|length > 3 + - glb_privatereg_username is defined + - glb_privatereg_username | length > 3 + - glb_privatereg_password is defined + - glb_privatereg_password | length > 3 + + - name: "[acm] Templating YAML files" + ansible.builtin.template: + src: apply-spec.yaml.j2 + dest: "{{ acm_config }}" + mode: 0600 + + - name: "[acm] Check ACM Feature Status" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gcloud + - container + - hub + - features + - list + - --filter + - 'name:configmanagement' + - --format + - 'value(name)' + - "--project={{ acm_gcpproject }}" + register: acm_feature_status + + - name: "[acm] Enable ACM Feature" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gcloud + - beta + - container + - hub + - config-management + - enable + - "--project={{ acm_gcpproject }}" + when: + - acm_feature_status.rc | default('') == 0 + - acm_feature_status.stdout | length == 0 + + - name: "[acm] Enable ACM" + ansible.builtin.command: + argv: + - gcloud + - beta + - container + - hub + - config-management + - apply + - "--membership={{ acm_membership }}" + - "--config={{ acm_config }}" + - "--version={{ acm_version }}" + - "--project={{ acm_gcpproject }}" + register: _results + retries: 5 + delay: 10 + until: _results.rc == 0 + + - name: "[acm] Wait for Policy Controller to be installed" + ansible.builtin.shell: + cmd: gcloud beta container hub config-management status --project={{ acm_gcpproject }} --format=json | jq -c '.acm_status[] | select(.name | contains("{{ acm_membership }}"))' # noqa yaml[line-length] + register: _acmpc_status + retries: 30 + until: "'\"policy_controller_state\":\"INSTALLED\"' in _acmpc_status.stdout" + delay: 20 + when: + - acm_policycontroller.enabled is defined + - acm_policycontroller.enabled | bool + + - name: "[acm] Wait for Config Sync to be synced" + ansible.builtin.shell: + cmd: gcloud beta container hub config-management status --project={{ acm_gcpproject }} --format=json | jq -c '.acm_status[] | select(.name | contains("{{ acm_membership }}"))' # noqa yaml[line-length] + register: _acmcs_status + retries: 30 + until: "'\"config_sync\":\"SYNCED\"' in _acmcs_status.stdout" + delay: 20 + when: + - acm_configsync.enabled is defined + - acm_configsync.enabled | bool + + always: + - name: "[acm] Clean up root repository ssh key file" + ansible.builtin.file: + path: "{{ yamldestpath }}/{{ uc_name }}/acm-sshkey" + state: absent diff --git a/tools/anthosvmware-ansible-module/roles/acm/tasks/main.yml b/tools/anthosvmware-ansible-module/roles/acm/tasks/main.yml new file mode 100644 index 0000000000..27f63b7e44 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/acm/tasks/main.yml @@ -0,0 +1,98 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[acm] Clean up old SSH host keys from incomplete runs" + ansible.builtin.file: + path: "/home/ubuntu/.ssh/known_hosts" + state: absent + +- name: "[acm] Include tasks - asserts.yml" + include_tasks: asserts.yml + +- name: "Include role for GCP SA JSON key files" + ansible.builtin.include_role: + name: copy_credentials + vars: + is_acm: true + +- name: "[acm] Block - Manage ACM" # noqa var-spacing + block: + - name: "[acm] Switch to ACM Service Account Name" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gcloud + - auth + - activate-service-account + - --key-file + - "{{ job_sakeyfolder }}/{{ acm_gcpsa_path }}" + + - name: "[acm] Verify cluster is registered" + ansible.builtin.command: + argv: + - gcloud + - container + - hub + - memberships + - list + - --project={{ acm_gcpproject }} + register: memberships + failed_when: not acm_membership in memberships.stdout + changed_when: false + + - name: "[acm] Get current ACM version" + ansible.builtin.command: + argv: + - gcloud + - beta + - container + - hub + - config-management + - version + - --project={{ acm_gcpproject }} + - --format=value(version) + - --filter=name:{{ acm_membership }} + register: acm_live_version + changed_when: false + + - name: "[acm] Compare current version with requested" + ansible.builtin.set_fact: + acm_upgrade: "{{ true if acm_live_version.stdout != 'NA' and acm_live_version.stdout != acm_version else false }}" + + - name: "[acm] Include tasks - install.yml" + include_tasks: install.yml + when: + - acm_install|default(false)|bool + - not acm_upgrade|bool + + - name: "[acm] Include tasks - upgrade.yml" + include_tasks: upgrade.yml + when: + - acm_install|default(false)|bool + - acm_upgrade|bool + + - name: "[acm] Include tasks - uninstall.yml" + include_tasks: uninstall.yml + when: not acm_install|default(false)|bool + + environment: + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" + http_proxy: "{{ glb_proxyurl }}" + https_proxy: "{{ glb_proxyurl }}" + no_proxy: "{{ glb_noproxy }}" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup diff --git a/tools/anthosvmware-ansible-module/roles/acm/tasks/private-upload.yml b/tools/anthosvmware-ansible-module/roles/acm/tasks/private-upload.yml new file mode 100644 index 0000000000..8a9a7f896f --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/acm/tasks/private-upload.yml @@ -0,0 +1,72 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[acm] Login to private container registry" + ansible.builtin.command: + argv: + - docker + - login + - -u + - '{{ glb_privatereg_username }}' + - -p + - '{{ glb_privatereg_password }}' + - '{{ glb_privatereg_url }}' + no_log: true + +- name: "[acm] Check if images are already pushed to private container registry" + ansible.builtin.command: + argv: + - docker + - manifest + - inspect + - "{{ glb_privatereg_url }}/{{ item.split('/')[-1] }}" + loop: "{{ acm_container_images[acm_version] }}" + loop_control: + label: "{{ glb_privatereg_url }}/{{ item.item.split('/')[-1] }}" + register: r_image_check + failed_when: false + +- name: "[acm] Download new images from gcr" + ansible.builtin.command: + argv: + - docker + - pull + - "{{ item.item }}" + loop: "{{ r_image_check.results }}" + loop_control: + label: "{{ item.item }}" + when: item.rc != 0 + +- name: "[acm] Change new image tags to private registry" + ansible.builtin.command: + argv: + - docker + - tag + - "{{ item.item }}" + - "{{ glb_privatereg_url }}/{{ item.item.split('/')[-1] }}" + loop: "{{ r_image_check.results }}" + loop_control: + label: "{{ glb_privatereg_url }}/{{ item.item.split('/')[-1] }}" + when: item.rc != 0 + +- name: "[acm] Push new images to private registry" + ansible.builtin.command: + argv: + - docker + - push + - "{{ glb_privatereg_url }}/{{ item.item.split('/')[-1] }}" + loop: "{{ r_image_check.results }}" + loop_control: + label: "{{ glb_privatereg_url }}/{{ item.item.split('/')[-1] }}" + when: item.rc != 0 \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/acm/tasks/uninstall.yml b/tools/anthosvmware-ansible-module/roles/acm/tasks/uninstall.yml new file mode 100644 index 0000000000..1297cdfa21 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/acm/tasks/uninstall.yml @@ -0,0 +1,64 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[acm] Disable ACM" + ansible.builtin.command: + argv: + - gcloud + - beta + - container + - hub + - config-management + - unmanage + - "--membership={{ acm_membership }}" + - "--project={{ acm_gcpproject }}" + register: _results + retries: 5 + delay: 10 + until: _results.rc == 0 + +- name: "[acm] Remove ACM operator configuration" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ yamldestpath }}/{{ uc_name }}-kubeconfig" + - delete + - configmanagement + - --all + +- name: "[acm] Remove ACM namespaces" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ yamldestpath }}/{{ uc_name }}-kubeconfig" + - delete + - ns + - config-management-system + - config-management-monitoring + - --ignore-not-found + +- name: "[acm] Remove ACM operator CRD" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ yamldestpath }}/{{ uc_name }}-kubeconfig" + - delete + - crd + - configmanagements.configmanagement.gke.io + - --ignore-not-found + diff --git a/tools/anthosvmware-ansible-module/roles/acm/tasks/upgrade.yml b/tools/anthosvmware-ansible-module/roles/acm/tasks/upgrade.yml new file mode 100644 index 0000000000..f55a4ccdba --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/acm/tasks/upgrade.yml @@ -0,0 +1,67 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[acm] Upload ACM Images to Private Registry" + include_tasks: private-upload.yml + when: + - glb_privatereg_url is defined + - glb_privatereg_url|length > 3 + - glb_privatereg_username is defined + - glb_privatereg_username | length > 3 + - glb_privatereg_password is defined + - glb_privatereg_password | length > 3 + +- name: "[acm] Upgrade ACM" + ansible.builtin.command: + argv: + - gcloud + - beta + - container + - hub + - config-management + - upgrade + - --membership={{ acm_membership }} + - --version={{ acm_version }} + - --project={{ acm_gcpproject }} + - --quiet + register: _results + retries: 5 + delay: 10 + until: _results.rc == 0 + +- name: "[acm] Wait for upgrade to start" + ansible.builtin.pause: + seconds: 30 + +- name: "[acm] Wait for Policy Controller" + ansible.builtin.shell: + cmd: gcloud beta container hub config-management status --project={{ acm_gcpproject }} --format=json | jq -c '.acm_status[] | select(.name | contains("{{ acm_membership }}"))' # noqa yaml[line-length] + register: _acmpc_status + retries: 30 + until: "'\"policy_controller_state\":\"INSTALLED\"' in _acmpc_status.stdout" + delay: 20 + when: + - acm_policycontroller.enabled is defined + - acm_policycontroller.enabled | bool + +- name: "[acm] Wait for Config Sync" + ansible.builtin.shell: + cmd: gcloud beta container hub config-management status --project={{ acm_gcpproject }} --format=json | jq -c '.acm_status[] | select(.name | contains("{{ acm_membership }}"))' # noqa yaml[line-length] + register: _acmcs_status + retries: 30 + until: "'\"config_sync\":\"SYNCED\"' in _acmcs_status.stdout" + delay: 20 + when: + - acm_configsync.enabled is defined + - acm_configsync.enabled | bool diff --git a/tools/anthosvmware-ansible-module/roles/acm/templates/apply-spec.yaml.j2 b/tools/anthosvmware-ansible-module/roles/acm/templates/apply-spec.yaml.j2 new file mode 100644 index 0000000000..58ec3964b1 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/acm/templates/apply-spec.yaml.j2 @@ -0,0 +1,25 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +applySpecVersion: 1 +spec: +{% if acm_configsync.enabled %} + configSync: +{{ acm_configsync | to_nice_yaml | indent(width=4, indentfirst=true) }} +{% endif %} +{% if acm_policycontroller.enabled %} + policyController: +{{ acm_policycontroller | to_nice_yaml | indent(width=4, indentfirst=true) }} +{% endif %} diff --git a/tools/anthosvmware-ansible-module/roles/acm/templates/ssh-key.j2 b/tools/anthosvmware-ansible-module/roles/acm/templates/ssh-key.j2 new file mode 100644 index 0000000000..a247a35148 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/acm/templates/ssh-key.j2 @@ -0,0 +1 @@ +{{ acm_root_repo_sshkey_content if acm_root_repo_sshkey_content is defined else lookup('file', acm_root_repo_sshkey) }} diff --git a/tools/anthosvmware-ansible-module/roles/admincluster/README.md b/tools/anthosvmware-ansible-module/roles/admincluster/README.md new file mode 100644 index 0000000000..63ea267204 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/admincluster/README.md @@ -0,0 +1,149 @@ +# Ansible Role: admincluster + +Installs and configures the Admin Cluster. + +## Requirements + +Optionally include the `ais` role to set up the Anthos Identity Service (AIS) with OIDC for user login. + +## Role Variables + +Available variables are listed below, along with default values (see `defaults/main.yml`): + +``` +ac_name: "ac01" # Name of admin cluster +ac_install: true +``` + +Name, version, and install/uninstall toggle. + +``` +component_access_gcpsa: "" +component_access_gcpsa_path: "" +``` + +Creates a unique folder to temporary store all Google Service Account (GSA) JSON key files. +Source and destination of the component access GSA. + +``` +# vSphere/vCenter +ac_vc_fqdn: "" +ac_vc_username: "" +ac_vc_password: "" +ac_vc_credfile: "credential.yaml" +ac_vc_credentry: "vCenter" +ac_vc_datacenter: "" +ac_vc_datastore: "" +ac_vc_cluster: "" +ac_vc_folder: "" # optional +ac_vc_respool: "" # if default resourcePool use /Resources +ac_vc_cacertpath: "/home/ubuntu/vcenter.pem" # Default location with automatically downloaded cert file name +# for vSAN: ac_vc_datadisk must be created inside a folder when using vSAN. Folder must be created manually +ac_vc_datadisk: "{{ ac_name }}-admin-cluster.vmdk" +``` + +The vSphere/vCenter specific settings, including credentials and object names. + +``` +# Networking +ac_nw_ipallocmode: "" # dhcp or static +ac_nw_ipfile: "{{ ac_name }}-ip-block.yaml" +ac_nw_gw: "" # gateway +ac_nw_nm: "" # netmask 255.255.255.0 or similar +ac_nw_ntp: ["",""] # list from group_vars/all +ac_nw_dns: ["","",""] # list of DNS servers +ac_nw_searchdomains: [""] # list from group_vars/all +ac_nw_servicecidr: "10.96.232.0/24" +ac_nw_podcidr: "192.168.0.0/16" +ac_nw_vc_net: "VM Network" + +# values for the file content of network.ipMode.ipBlockFilePath +ac_ipblock_netmask: 255.255.255.0 +ac_ipblock_gateway: a.b.c.d +ac_ipblock_ips: [""] +``` + +Network settings. + +``` +# Load balancing +ac_lb_kind: MetalLB +ac_lb_vips_cp: "" +ac_lb_vips_addons: "" +``` + +Load balancing and VIPs. +Options for load balancing is limited to MetalLB + +``` +# masternode sizing +ac_masternode_cpus: 4 +ac_masternode_mem: 8192 +``` + +Master node sizing with above default values. + +``` +ac_antiaffinitygroups: true +``` + +Toggle distribution of VMs onto different virtualization hosts. + + +``` +# GCP project IDs +ac_stackdriver_projectid: "" +ac_stackdriver_clusterlocation: "" +ac_stackdriver_enablevpc: false +logging_monitoring_gcpsa_path: "" +ac_stackdriver_disablevsphereresourcemetrics: false +ac_gkeconnect_projectid: "" +connect_register_gcpsa_path: "" +ac_cloudauditlogging_projectid: "" +ac_cloudauditlogging_clusterlocation: "" +audit_logging_gcpsa_path: "" +``` + +Google Cloud Project IDs, regions, and GSA file paths. + +``` +ac_autorepair: true +``` + +Toggle the Autorepair feature. + +``` +# Kubernetes Secrets at-rest encryption +ac_secretsencryption_mode: "GeneratedKey" # optional +ac_secretsencryption_keyversion: 1 # optional +``` + +Enable encryption at-rest for Kubernetes Secrets. + +``` +# optional --skip-validation for gkectl +ac_skipvalidations: "" +ac_verbosity: 5 +``` + +Verbosity of `gkectl` command outputs and skipping all or specific validation preflight checks. + +## Dependencies + +None. + +## Example Playbook + +``` +- name: "[Admin Cluster] Installation" + hosts: all + gather_facts: False + roles: + - admincluster + - { role: ais, when: ais_authentication is defined and (ais_authentication|length > 0) and ac_install|default(true)|bool } +``` + +## **License** + +Copyright 2022 Google LLC. This software is provided as-is, without warranty or representation for any use or purpose. +Your use of it is subject to your agreement with Google. diff --git a/tools/anthosvmware-ansible-module/roles/admincluster/defaults/main.yml b/tools/anthosvmware-ansible-module/roles/admincluster/defaults/main.yml new file mode 100644 index 0000000000..451f8a91cb --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/admincluster/defaults/main.yml @@ -0,0 +1,87 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# role ac default values +ac_name: "ac01" # Name of admin cluster +ac_install: true +component_access_gcpsa: "" +component_access_gcpsa_path: "" + +# vSphere/vCenter +ac_vc_fqdn: "" +ac_vc_credfile: "credential.yaml" +ac_vc_credentry: "vCenter" +ac_vc_username: "" +ac_vc_password: "" +ac_vc_datacenter: "" +ac_vc_datastore: "" +ac_vc_cluster: "" +ac_vc_folder: "" # optional +ac_vc_respool: "" # if default resourcePool use /Resources +ac_vc_cacertpath: "/home/ubuntu/vcenter.pem" # Default location with automatically downloaded cert file name +# for vSAN: ac_vc_datadisk must be created inside a folder when using vSAN. Folder must be created manually +ac_vc_datadisk: "{{ ac_name }}-admin-cluster.vmdk" + +# Networking +ac_nw_ipallocmode: "" # dhcp or static +ac_nw_ipfile: "{{ ac_name }}-ip-block.yaml" +ac_nw_gw: "" # gateway +ac_nw_nm: "" # netmask 255.255.255.0 or similar +ac_nw_ntp: ["", ""] # list from group_vars/all +ac_nw_dns: ["", "", ""] # list of DNS servers +ac_nw_searchdomains: [""] # list from group_vars/all +ac_nw_servicecidr: "10.96.232.0/24" +ac_nw_podcidr: "192.168.0.0/16" +ac_nw_vc_net: "VM Network" + +# values for the file content of network.ipMode.ipBlockFilePath +ac_ipblock_netmask: 255.255.255.0 +ac_ipblock_gateway: a.b.c.d +ac_ipblock_ips: [""] + +# Load balancing +ac_lb_kind: MetalLB +ac_lb_vips_cp: "" +ac_lb_vips_addons: "" + +# masternode sizing +ac_masternode_cpus: 4 +ac_masternode_mem: 8192 +ac_antiaffinitygroups: true + +# GCP project IDs +ac_stackdriver_projectid: "" +ac_stackdriver_clusterlocation: "" +ac_stackdriver_enablevpc: false +logging_monitoring_gcpsa_path: "" +ac_stackdriver_disablevsphereresourcemetrics: false +ac_gkeconnect_projectid: "" +connect_register_gcpsa_path: "" +ac_cloudauditlogging_projectid: "" +ac_cloudauditlogging_clusterlocation: "" +audit_logging_gcpsa_path: "" + +ac_autorepair: true + +# Kubernetes Secrets at-rest encryption +ac_secretsencryption_mode: "GeneratedKey" # optional +ac_secretsencryption_keyversion: 1 # optional + +# optional --skip-validation for gkectl +ac_skipvalidations: "" +ac_verbosity: 5 + +# upgrades +ac_upgrade: false diff --git a/tools/anthosvmware-ansible-module/roles/admincluster/tasks/asserts.yml b/tools/anthosvmware-ansible-module/roles/admincluster/tasks/asserts.yml new file mode 100644 index 0000000000..5e451e1dd7 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/admincluster/tasks/asserts.yml @@ -0,0 +1,73 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[ac] Sanity Checks" + ansible.builtin.assert: + that: + - yamldestpath | length >= 6 + - sakeyfolder_base is defined + - component_access_gcpsa_path | length >= 6 + - ac_vc_fqdn | length >= 4 + - ac_vc_credentry | length >= 2 + - ac_vc_username | length >= 4 + - ac_vc_password | length >= 8 + - ac_vc_datacenter | length >= 2 + - ac_vc_datastore | length >= 2 + - ac_vc_cluster | length >= 2 + - ac_vc_respool | length >= 4 + - ac_vc_cacertpath is defined + - ac_nw_ipallocmode is search("dhcp") or ac_nw_ipallocmode is search("static") + - ac_nw_gw | length >= 7 + - ac_nw_nm | length >= 7 + - ac_nw_dns is defined and ac_nw_dns | type_debug == "list" + - ac_nw_servicecidr | length >= 7 + - ac_nw_podcidr | length >= 7 + - ac_nw_vc_net | length >= 2 + - ac_nw_ntp is defined and ac_nw_ntp | type_debug == "list" + - ac_nw_searchdomains is defined and ac_nw_searchdomains | type_debug == "list" + - ac_ipblock_netmask | length >= 7 + - ac_ipblock_gateway | length >= 7 + - ac_lb_kind is search("MetalLB") + - ac_ipblock_ips is defined and ac_ipblock_ips | type_debug == "list" + - ac_lb_vips_cp | length >= 7 + # - ac_lb_vips_addons | length >= 7 + - ac_masternode_cpus | int + - ac_masternode_mem | int + - ac_antiaffinitygroups == ac_antiaffinitygroups|bool + - ac_stackdriver_projectid | length >= 6 + - ac_stackdriver_clusterlocation | length >= 6 + - ac_stackdriver_enablevpc == ac_stackdriver_enablevpc|bool + - logging_monitoring_gcpsa_path | length >= 6 + - ac_stackdriver_disablevsphereresourcemetrics == ac_stackdriver_disablevsphereresourcemetrics|bool + - ac_gkeconnect_projectid | length >= 6 + - connect_register_gcpsa_path | length >= 6 + # - ac_cloudauditlogging_projectid | length >= 6 + # - ac_cloudauditlogging_clusterlocation | length >= 6 + # - audit_logging_gcpsa_path | length >= 6 + - ac_autorepair == ac_autorepair|bool + fail_msg: "Undefined variables." + success_msg: "All required variables are defined." + +- name: "[ac] Sanity Checks" + ansible.builtin.assert: + that: + - ac_ipblock_ips is defined and ac_ipblock_ips | type_debug == "list" + fail_msg: "Undefined variables." + success_msg: "All required variables are defined." + when: ac_nw_ipallocmode is search("dhcp") + +- name: "[ac] Note on optional values" + ansible.builtin.debug: + msg: "Optional values ac_name, ac_vc_folder are not checked. Please double check if required." diff --git a/tools/anthosvmware-ansible-module/roles/admincluster/tasks/install.yml b/tools/anthosvmware-ansible-module/roles/admincluster/tasks/install.yml new file mode 100644 index 0000000000..4f415445c5 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/admincluster/tasks/install.yml @@ -0,0 +1,153 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[ac] Block - Preflight checks, deploy cluster and delete GCP SA JSON key files" + block: + + - name: "[ac] Setup cluster config" + include_tasks: setup-config.yml + + - name: "[ac] Check existence of datastore folder for admin cluster" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - govc + - datastore.ls + - -json + register: r_ac_folder + + - name: "[ac] Create datastore folder for admin cluster" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - govc + - datastore.mkdir + - "{{ ac_name }}" + when: not r_ac_folder.stdout is search(ac_name) + + - name: "[ac] Check existence of kubevols folder" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - govc + - datastore.ls + - -json + register: r_kubevols_folder + + - name: "[ac] Create datastore folder kubevols" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - govc + - datastore.mkdir + - kubevols + when: not r_kubevols_folder.stdout is search("kubevols") + + - name: "[ac] prepare check status file" + ansible.builtin.stat: + path: "{{ yamldestpath }}/{{ ac_name }}/{{ ac_name }}-prepare" + register: prepare_file + tags: prepare + + - name: "[ac] Upload OVAs to vSphere and optionally upload system images to private registry" + async: 7200 + poll: 5 + ansible.builtin.command: # noqa 305 301 no-changed-when + chdir: "{{ yamldestpath }}/{{ ac_name }}" + argv: "{{ _argv }}" + vars: + _req_args: + - gkectl + - prepare + - --config + - "{{ ac_name }}-cluster.yaml" + _skips: "{{ ac_skipvalidations if ac_skipvalidations | type_debug == 'list' else (ac_skipvalidations.split(' ')) }}" + _argv: "{{ _req_args + _skips }}" + register: prepare_result + when: not prepare_file.stat.exists + tags: prepare + + - name: "[ac] Prepare results" + ansible.builtin.file: + path: "{{ yamldestpath }}/{{ ac_name }}/{{ ac_name }}-prepare" + state: touch + mode: 0600 + when: prepare_result.rc | default('') == 0 + tags: prepare + + - name: "[ac] preflight check status file" + ansible.builtin.stat: + path: "{{ yamldestpath }}/{{ ac_name }}/{{ ac_name }}-preflight" + register: preflight_file + tags: preflight + + - name: "[ac] Preflight check" + async: 2700 + poll: 5 + ansible.builtin.command: # noqa 305 301 no-changed-when + chdir: "{{ yamldestpath }}/{{ ac_name }}" + argv: "{{ _argv }}" + vars: + _req_args: + - gkectl + - check-config + - --config + - "{{ ac_name }}-cluster.yaml" + _skips: "{{ ac_skipvalidations if ac_skipvalidations | type_debug == 'list' else (ac_skipvalidations.split(' ')) }}" + _argv: "{{ _req_args + _skips }}" + register: preflight_result + when: not preflight_file.stat.exists + tags: preflight + + - name: "[ac] Preflight results" + ansible.builtin.file: + path: "{{ yamldestpath }}/{{ ac_name }}/{{ ac_name }}-preflight" + state: touch + mode: 0600 + when: preflight_result.rc | default('') == 0 + tags: preflight + + - name: "[ac] Create cluster" + async: 3600 + poll: 5 + ansible.builtin.command: # noqa 305 301 no-changed-when + chdir: "{{ yamldestpath }}" + argv: "{{ _argv }}" + creates: "~/kubeconfig" + vars: + _req_args: + - gkectl + - create + - admin + - --config + - "{{ ac_name }}/{{ ac_name }}-cluster.yaml" + - --alsologtostderr + - -v + - "{{ ac_verbosity }}" + _skips: "{{ ac_skipvalidations if ac_skipvalidations | type_debug == 'list' else (ac_skipvalidations.split(' ')) }}" + _argv: "{{ _req_args + _skips }}" + + environment: + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" + http_proxy: "{{ glb_proxyurl }}" + https_proxy: "{{ glb_proxyurl }}" + no_proxy: "{{ glb_noproxy }}" + GOVC_URL: "https://{{ ac_vc_fqdn }}/sdk" + GOVC_USERNAME: "{{ ac_vc_username }}" + GOVC_PASSWORD: "{{ ac_vc_password }}" + GOVC_DATASTORE: "{{ ac_vc_datastore }}" + GOVC_DATACENTER: "{{ ac_vc_datacenter }}" + GOVC_RESOURCE_POOL: "{{ ac_vc_respool }}" + GOVC_INSECURE: "true" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup diff --git a/tools/anthosvmware-ansible-module/roles/admincluster/tasks/main.yml b/tools/anthosvmware-ansible-module/roles/admincluster/tasks/main.yml new file mode 100644 index 0000000000..6a0b4b0694 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/admincluster/tasks/main.yml @@ -0,0 +1,67 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[ac] Clean up old SSH host keys from incomplete runs" + ansible.builtin.file: + path: "/home/ubuntu/.ssh/known_hosts" + state: absent + +- name: "[ac] Include tasks - asserts.yml" + include_tasks: asserts.yml + +- name: "[ac] Detect if Admin Cluster kubeconfig exists" + ansible.builtin.stat: + path: "{{ yamldestpath }}/kubeconfig" + register: ac_kubeconfig_file + +- name: "[ac] Detect if Admin Cluster already exists" + ansible.builtin.command: + chdir: "{{ yamldestpath }}" + argv: + - kubectl + - --kubeconfig + - kubeconfig + - cluster-info + register: r_ac_exists + ignore_errors: true + when: ac_kubeconfig_file.stat.exists + changed_when: false + +- name: "[ac] Set ac_exists flag if Admin Cluster already exists" + ansible.builtin.set_fact: + ac_exists: true + when: + - r_ac_exists.stdout_lines is defined + - r_ac_exists.stdout_lines is search('control plane') + - r_ac_exists.stdout_lines is search('is running') + +- name: "[ac] Include tasks - install.yml" + include_tasks: install.yml + when: + - ac_install|default(false)|bool + - not ac_upgrade|default(false)|bool + - not ac_exists|default(false)|bool + +- name: "[ac] Include tasks - uninstall.yml" + include_tasks: uninstall.yml + when: + - not ac_install|default(false)|bool + - not ac_upgrade|default(false)|bool + - ac_exists|default(false)|bool + +- name: "[ac] Include tasks - upgrade.yml" + include_tasks: upgrade.yml + when: + - ac_upgrade|default(false)|bool + - ac_exists|default(false)|bool diff --git a/tools/anthosvmware-ansible-module/roles/admincluster/tasks/setup-config.yml b/tools/anthosvmware-ansible-module/roles/admincluster/tasks/setup-config.yml new file mode 100644 index 0000000000..4e171739f5 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/admincluster/tasks/setup-config.yml @@ -0,0 +1,95 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "Include role for GCP SA JSON key files" + ansible.builtin.include_role: + name: copy_credentials + vars: + is_admin_cluster: true + +- name: "[ac] Create folder on Admin Workstation for YAML files" + ansible.builtin.file: + path: "{{ yamldestpath }}/{{ ac_name }}" + state: directory + mode: 0700 + +- name: "[ac] Templating YAML files" + ansible.builtin.template: + src: admin-cluster-{{ glb_major_version }}.yaml.j2 + dest: "{{ yamldestpath }}/{{ ac_name }}/{{ ac_name }}-cluster.yaml" + mode: 0600 + register: ac_template + +- name: "[ac] Templating YAML files - if using static IPs" + include_tasks: static-ips.yml + when: ac_nw_ipallocmode == "static" + +- name: "[ac] Copy Private Registry CA to admin workstation from URL" + ansible.builtin.command: + argv: + - curl + - -k + - -L + - "{{ glb_privatereg_cacerturl }}" + - -o + - "{{ glb_privatereg_cacertpath }}" + creates: "{{ glb_privatereg_cacertpath }}" + when: + - (glb_privatereg_cacertpath is defined) and (glb_privatereg_cacertpath|length > 0) + - (glb_privatereg_cacerturl is defined) and (glb_privatereg_cacerturl|length > 0) + +- name: "[ac] Copy Private Registry CA to admin workstation from Tower Vault" + ansible.builtin.copy: + src: '{{ glb_privatereg_cacertfile }}' + dest: "{{ glb_privatereg_cacertpath }}" + mode: 0644 + when: + - (glb_privatereg_cacertfile is defined) and (glb_privatereg_cacertfile|length > 0) + - (glb_privatereg_cacertpath is defined) and (glb_privatereg_cacertpath|length > 0) + +- name: "[ac] Copy Private Registry CA to admin workstation from Inventory" + ansible.builtin.template: + src: private-reg-ca.crt.j2 + dest: "{{ glb_privatereg_cacertpath }}" + mode: 0644 + when: + - (glb_privatereg_cacontent is defined) and (glb_privatereg_cacontent|length > 0) + - (glb_privatereg_cacertpath is defined) and (glb_privatereg_cacertpath|length > 0) + +- name: "[ac] Create private registry host directory under certs.d" + ansible.builtin.command: + argv: + - sudo + - mkdir + - -p + - /etc/docker/certs.d/{{ glb_privatereg_url.split("/")[0] }} + ignore_errors: true + when: + - (glb_privatereg_url is defined) and (glb_privatereg_url|length > 0) + - (glb_privatereg_cacertfile is defined) and (glb_privatereg_cacertfile|length > 0) + - (glb_privatereg_cacertpath is defined) and (glb_privatereg_cacertpath|length > 0) + +- name: "[ac] Add permission to private registry cert directory" + ansible.builtin.command: + argv: + - sudo + - chmod + - 755 + - /etc/docker/certs.d/{{ glb_privatereg_url.split("/")[0] }} + ignore_errors: true + when: + - (glb_privatereg_url is defined) and (glb_privatereg_url|length > 0) + - (glb_privatereg_cacertfile is defined) and (glb_privatereg_cacertfile|length > 0) + - (glb_privatereg_cacertpath is defined) and (glb_privatereg_cacertpath|length > 0) diff --git a/tools/anthosvmware-ansible-module/roles/admincluster/tasks/static-ips.yml b/tools/anthosvmware-ansible-module/roles/admincluster/tasks/static-ips.yml new file mode 100644 index 0000000000..69e663c403 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/admincluster/tasks/static-ips.yml @@ -0,0 +1,19 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[ac] Templating YAML files - IP block" + ansible.builtin.template: + src: cluster-ip-block.yaml.j2 + dest: "{{ yamldestpath }}/{{ ac_name }}/{{ ac_name }}-ip-block.yaml" + mode: 0600 diff --git a/tools/anthosvmware-ansible-module/roles/admincluster/tasks/uninstall.yml b/tools/anthosvmware-ansible-module/roles/admincluster/tasks/uninstall.yml new file mode 100644 index 0000000000..b4b8c7e14f --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/admincluster/tasks/uninstall.yml @@ -0,0 +1,263 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "Include role for GCP SA JSON key files" + ansible.builtin.include_role: + name: copy_credentials + vars: + is_admin_cluster: true + +- name: "[ac] Block - delete VMs, data disk, cluster and GCP SA JSON key files" + block: + - name: "[ac] Get Register Service Account Name" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gcloud + - auth + - list + - --filter + - 'account~reg AND account~{{ ac_gkeconnect_projectid }}' + - --format + - 'value(account)' + register: register_sa + tags: delete + + - name: "[ac] Switch to Register Service Account Name" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gcloud + - config + - set + - account + - "{{ register_sa.stdout }}" + tags: delete + + - name: "[ac] Unregister Admin Cluster from GKE Hub" + ansible.builtin.command: # noqa 305 301 no-changed-when + chdir: "{{ yamldestpath }}" + argv: + - gcloud + - container + - hub + - memberships + - delete + - "{{ ac_name }}" + - --project + - "{{ ac_gkeconnect_projectid }}" + - --quiet + tags: delete + + - name: "[ac] Delete Monitoring System Pods" + ansible.builtin.command: # noqa 305 301 no-changed-when + chdir: "{{ yamldestpath }}" + argv: + - kubectl + - delete + - monitoring + - --all + - -n + - kube-system + - --kubeconfig + - kubeconfig + tags: delete + + - name: "[ac] Delete Logging System Pods" + ansible.builtin.command: # noqa 305 301 no-changed-when + chdir: "{{ yamldestpath }}" + argv: + - kubectl + - delete + - stackdriver + - --all + - -n + - kube-system + - --kubeconfig + - kubeconfig + tags: delete + + - name: "[ac] Get vSphere VM Names" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - kubectl + - get + - machines + - --kubeconfig + - kubeconfig + - --no-headers + register: vm_names + tags: delete + + - name: "[ac] Get Admin Cluster Master VM Name" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - kubectl + - get + - machines + - --kubeconfig + - kubeconfig + - -l + - set=master + - --no-headers + register: ac_master_vm_name + tags: delete + + - name: "[ac] admin cluster debug kubeconfig file status" + ansible.builtin.stat: + path: "{{ yamldestpath }}/internal-cluster-kubeconfig-debug" + register: ac_kubeconfig_debug_file + + - name: "[ac] admin cluster kubeconfig file status" + ansible.builtin.stat: + path: "~/kubeconfig" + register: ac_kubeconfig_file + + - name: "[ac] Delete VMs from vCenter" + with_items: "{{ vm_names.stdout_lines }}" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - govc + - vm.destroy + - "{{ item }}" + register: ac_vms_delete_status + when: ac_kubeconfig_file.stat.exists + tags: delete + + - name: "[ac] Check for Admin Cluster Master Template" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - govc + - ls + - "vm/{{ ac_master_vm_name.stdout }}-tmpl" + register: ac_master_vm_tmpl_full_name + when: ac_master_vm_name.rc | default('') == 0 + tags: delete + + - name: "[ac] Delete Admin Cluster Master Template" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - govc + - vm.destroy + - "{{ ac_master_vm_tmpl_full_name.stdout }}" + when: + - ac_master_vm_tmpl_full_name.rc | default('') == 0 + - ac_master_vm_tmpl_full_name.stdout | length > 0 + tags: delete + + - name: "[ac] Delete Admin Cluster Debug Kubeconfig" + ansible.builtin.file: + path: "{{ yamldestpath }}/internal-cluster-kubeconfig-debug" + state: absent + when: ac_vms_delete_status.results[0].rc | default('') == 0 + + - name: "[ac] Delete Admin Cluster Kubeconfig" + ansible.builtin.file: + path: "~/kubeconfig" + state: absent + when: ac_vms_delete_status.results[0].rc | default('') == 0 + + - name: "[ac] Check for Admin Cluster Data Disk vmdk" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - govc + - datastore.ls + - "{{ ac_vc_datadisk }}" + ignore_errors: true + register: ac_datadisk_status + when: ac_vms_delete_status.results[0].rc | default('') == 0 + tags: delete + + - name: "[ac] Delete Data Disk vmdk" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - govc + - datastore.rm + - "{{ ac_vc_datadisk }}" + when: + - ac_datadisk_status.stdout | default('') | length > 0 + tags: delete + + - name: "[ac] Check Data Disk Checkpoint file" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - govc + - datastore.ls + - "{{ ac_name }}/{{ ac_name }}-checkpoint.yaml" + ignore_errors: true + register: ac_datadisk_checkpoint_status + when: ac_vms_delete_status.results[0].rc | default('') == 0 + tags: delete + + - name: "[ac] Delete Data Disk Checkpoint file" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - govc + - datastore.rm + - "{{ ac_name }}/{{ ac_name }}-checkpoint.yaml" + when: + - ac_datadisk_checkpoint_status.stdout | default('') | length > 0 + tags: delete + + - name: "[ac] Check for datastore folder kubevols" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - govc + - datastore.ls + - -p + register: r_kubevols_folder + tags: delete + + - name: "[ac] Delete datastore folder kubevols" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - govc + - datastore.rm + - kubevols + when: r_kubevols_folder is search("kubevols") + tags: delete + + - name: "[ac] Check existence of datastore folder for admin cluster" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - govc + - datastore.ls + - -p + register: r_ac_folder + tags: delete + + - name: "[ac] Delete datastore folder for admin cluster" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - govc + - datastore.rm + - "{{ ac_name }}" + when: r_ac_folder is search(ac_name) + tags: delete + + environment: + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" + http_proxy: "{{ glb_proxyurl }}" + https_proxy: "{{ glb_proxyurl }}" + no_proxy: "{{ glb_noproxy }}" + GOVC_URL: "https://{{ ac_vc_fqdn }}/sdk" + GOVC_USERNAME: "{{ ac_vc_username }}" + GOVC_PASSWORD: "{{ ac_vc_password }}" + GOVC_DATASTORE: "{{ ac_vc_datastore }}" + GOVC_DATACENTER: "{{ ac_vc_datacenter }}" + GOVC_RESOURCE_POOL: "{{ ac_vc_respool }}" + GOVC_INSECURE: "true" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup diff --git a/tools/anthosvmware-ansible-module/roles/admincluster/tasks/upgrade.yml b/tools/anthosvmware-ansible-module/roles/admincluster/tasks/upgrade.yml new file mode 100644 index 0000000000..15e8c2a106 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/admincluster/tasks/upgrade.yml @@ -0,0 +1,108 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "Include role for GCP SA JSON key files" + ansible.builtin.include_role: + name: copy_credentials + vars: + is_admin_cluster: true + +- name: "[ac] Block - Upload new bundle, upgrade cluster and delete GCP SA JSON key files" + block: + - name: "[ac] Get current admin cluster version" + ansible.builtin.command: + chdir: "{{ yamldestpath }}" + argv: + - kubectl + - --kubeconfig + - kubeconfig + - get + - onpremadmincluster + - "{{ ac_name }}" + - -n + - kube-system + - "-o=jsonpath={.spec.gkeOnPremVersion}" + register: live_version + + - name: "[ac] Verify inventory version not equal current version" + ansible.builtin.assert: + that: glb_anthos_version != live_version.stdout + fail_msg: "Requested Admin cluster version {{ glb_anthos_version }} equals current live version {{ live_version.stdout }}." + success_msg: "Requested Admin cluster version different from live." + + - name: "[ac] Get gkectl version from Admin Workstation" + ansible.builtin.shell: "gkectl version | head -n1 | awk '{print $2;}'" + register: gkectl_version + + - name: "[ac] Verify inventory version matches Admin workstation version" + ansible.builtin.assert: + that: glb_anthos_version == gkectl_version.stdout + fail_msg: "Requested Admin cluster version {{ glb_anthos_version }} does not match Admin workstation version {{ gkectl_version.stdout }}." + success_msg: "Requested Admin cluster version matched Admin workstation version" + + - name: "[ac] Replace bundlePath in admin cluster config YAML" + ansible.builtin.lineinfile: + path: "{{ yamldestpath }}/{{ ac_name }}/{{ ac_name }}-cluster.yaml" + regex: '(^bundlePath: )(.*?)\/(gke-onprem-vsphere-)(.*?)(\.tgz*)' + line: 'bundlePath: "/var/lib/gke/bundles/gke-onprem-vsphere-{{ glb_anthos_version }}-full.tgz"' + + - name: "[ac] Upload new bundle to vSphere" + ansible.builtin.command: + chdir: "{{ yamldestpath }}/{{ ac_name }}" + argv: "{{ _argv }}" + vars: + _req_args: + - gkectl + - prepare + - --bundle-path + - "/var/lib/gke/bundles/gke-onprem-vsphere-{{ glb_anthos_version }}-full.tgz" + - --kubeconfig + - ~/kubeconfig + - --alsologtostderr + - -v + - "{{ ac_verbosity }}" + _skips: "{{ ac_skipvalidations if ac_skipvalidations | type_debug == 'list' else (ac_skipvalidations.split(' ')) }}" + _argv: "{{ _req_args + _skips }}" + + - name: "[ac] Upgrade admin cluster" + async: 5400 + poll: 5 + ansible.builtin.command: # noqa 305 no-changed-when + chdir: "{{ yamldestpath }}/{{ ac_name }}" + argv: "{{ _argv }}" + vars: + _req_args: + - gkectl + - upgrade + - admin + - --config + - "{{ ac_name }}-cluster.yaml" + - --kubeconfig + - ~/kubeconfig + - --alsologtostderr + - -v + - "{{ ac_verbosity }}" + _skips: "{{ ac_skipvalidations if ac_skipvalidations | type_debug == 'list' else (ac_skipvalidations.split(' ')) }}" + _argv: "{{ _req_args + _skips }}" + + environment: + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" + http_proxy: "{{ glb_proxyurl }}" + https_proxy: "{{ glb_proxyurl }}" + no_proxy: "{{ glb_noproxy }}" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup diff --git a/tools/anthosvmware-ansible-module/roles/admincluster/templates/admin-cluster-1.11.yaml.j2 b/tools/anthosvmware-ansible-module/roles/admincluster/templates/admin-cluster-1.11.yaml.j2 new file mode 100644 index 0000000000..3ddd4aaa3f --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/admincluster/templates/admin-cluster-1.11.yaml.j2 @@ -0,0 +1,128 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +apiVersion: v1 +kind: AdminCluster +# (Optional) A unique name for this admin cluster. This will default to a random name +# prefixed with 'gke-admin-' +name: "{{ ac_name }}" +# (Required) Absolute path to a GKE bundle on disk +#bundlePath: /var/lib/gke/bundles/gke-onprem-vsphere-{{ glb_anthos_version }}-full.tgz +bundlePath: /var/lib/gke/bundles/gke-onprem-vsphere-{{ glb_anthos_version }}.tgz +# (Required) vCenter configuration +vCenter: + address: {{ ac_vc_fqdn }} + datacenter: {{ ac_vc_datacenter }} + folder: {{ ac_vc_folder }} + cluster: {{ ac_vc_cluster }} + resourcePool: {{ ac_vc_respool }} + datastore: {{ ac_vc_datastore }} + caCertPath: {{ ac_vc_cacertpath }} + credentials: + fileRef: + path: {{ ac_vc_credfile }} + entry: {{ ac_vc_credentry }} + dataDisk: "{{ ac_vc_datadisk }}" +network: + hostConfig: + dnsServers: +{% for ip in ac_nw_dns %} + - "{{ ip }}" +{% endfor %} + ntpServers: +{% for ntp in ac_nw_ntp %} + - "{{ ntp }}" +{% endfor %} + searchDomainsForDNS: +{% for domain in ac_nw_searchdomains %} + - "{{ domain }}" +{% endfor %} + ipMode: + type: {{ ac_nw_ipallocmode }} +{% if (ac_nw_ipallocmode is defined) and ('static' == ac_nw_ipallocmode) %} + ipBlockFilePath: "{{ ac_nw_ipfile }}" +{% endif %} + serviceCIDR: {{ ac_nw_servicecidr }} + podCIDR: {{ ac_nw_podcidr }} + vCenter: + networkName: {{ ac_nw_vc_net }} +loadBalancer: + vips: + controlPlaneVIP: "{{ ac_lb_vips_cp }}" + addonsVIP: "{{ ac_lb_vips_addons }}" + kind: {{ ac_lb_kind }} +antiAffinityGroups: + enabled: {{ ac_antiaffinitygroups }} +connectivity: connected +# (Optional) Specify the proxy configuration +proxy: + url: "{{ glb_proxyurl }}" + noProxy: "{{ glb_noproxy }}" +{% if (glb_privatereg_url is defined) and (glb_privatereg_url|length > 0) %} +# # (Optional) Use a private Docker registry to host GKE images +privateRegistry: + address: "{{ glb_privatereg_url }}" + credentials: + fileRef: + path: credential.yaml + entry: privateRegistry +{% if (glb_privatereg_cacertpath is defined) and (glb_privatereg_cacertpath|length > 0) %} + caCertPath: "{{ glb_privatereg_cacertpath }}" +{% endif %} +{% endif %} +# (Required): The absolute or relative path to the GCP service account key for pulling +# GKE images +componentAccessServiceAccountKeyPath: "{{ job_sakeyfolder }}/{{ component_access_gcpsa_path }}" +# (Optional) Specify which GCP project to connect your GKE clusters to +gkeConnect: + projectID: "{{ ac_stackdriver_projectid }}" + registerServiceAccountKeyPath: "{{ job_sakeyfolder }}/{{ connect_register_gcpsa_path }}" + #registerServiceAccountKeyPath: "{{ job_sakeyfolder }}/connect-register.json" +stackdriver: + projectID: "{{ ac_stackdriver_projectid }}" + clusterLocation: "{{ ac_stackdriver_clusterlocation }}" + enableVPC: {{ ac_stackdriver_enablevpc }} + serviceAccountKeyPath: "{{ job_sakeyfolder }}/{{ logging_monitoring_gcpsa_path }}" + # serviceAccountKeyPath: "{{ job_sakeyfolder }}/logging-monitoring.json" + disableVsphereResourceMetrics: false +# # (Optional) Configure kubernetes apiserver audit logging +{% if (ac_cloudauditlogging_projectid is defined) and (ac_cloudauditlogging_projectid|length > 0) %} +cloudAuditLogging: + projectID: "{{ ac_cloudauditlogging_projectid }}" + # A GCP region where you would like to store audit logs for this cluster. + clusterLocation: "{{ ac_cloudauditlogging_clusterlocation }}" + # The absolute or relative path to the key file for a GCP service account used to + # send audit logs from the cluster + serviceAccountKeyPath: "{{ job_sakeyfolder }}/{{ audit_logging_gcpsa_path }}" + # serviceAccountKeyPath: "{{ job_sakeyfolder }}/audit-logging.json" +{% endif %} +# # (Optional/Preview) Configure backups for admin cluster. Backups will be stored under +# # /anthos-backups/ +clusterBackup: + datastore: "{{ ac_vc_datastore }}" +autoRepair: + enabled: {{ ac_autorepair }} +{% if (ac_secretsencryption_mode is defined) and ('GeneratedKey' == ac_secretsencryption_mode) %} +secretsEncryption: + # Secrets Encryption Mode. Possible values are: None GeneratedKey + mode: "{{ ac_secretsencryption_mode }}" + # GeneratedKey Secrets Encryption config + generatedKey: + # # key version + keyVersion: {{ ac_secretsencryption_keyversion }} +{% endif %} +# (Optional) Specify the type of OS image; available options can be set to "ubuntu_containerd" +# or "cos". Default is "ubuntu_containerd". +osImageType: "ubuntu_containerd" diff --git a/tools/anthosvmware-ansible-module/roles/admincluster/templates/admin-cluster-1.12.yaml.j2 b/tools/anthosvmware-ansible-module/roles/admincluster/templates/admin-cluster-1.12.yaml.j2 new file mode 100644 index 0000000000..3ddd4aaa3f --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/admincluster/templates/admin-cluster-1.12.yaml.j2 @@ -0,0 +1,128 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +apiVersion: v1 +kind: AdminCluster +# (Optional) A unique name for this admin cluster. This will default to a random name +# prefixed with 'gke-admin-' +name: "{{ ac_name }}" +# (Required) Absolute path to a GKE bundle on disk +#bundlePath: /var/lib/gke/bundles/gke-onprem-vsphere-{{ glb_anthos_version }}-full.tgz +bundlePath: /var/lib/gke/bundles/gke-onprem-vsphere-{{ glb_anthos_version }}.tgz +# (Required) vCenter configuration +vCenter: + address: {{ ac_vc_fqdn }} + datacenter: {{ ac_vc_datacenter }} + folder: {{ ac_vc_folder }} + cluster: {{ ac_vc_cluster }} + resourcePool: {{ ac_vc_respool }} + datastore: {{ ac_vc_datastore }} + caCertPath: {{ ac_vc_cacertpath }} + credentials: + fileRef: + path: {{ ac_vc_credfile }} + entry: {{ ac_vc_credentry }} + dataDisk: "{{ ac_vc_datadisk }}" +network: + hostConfig: + dnsServers: +{% for ip in ac_nw_dns %} + - "{{ ip }}" +{% endfor %} + ntpServers: +{% for ntp in ac_nw_ntp %} + - "{{ ntp }}" +{% endfor %} + searchDomainsForDNS: +{% for domain in ac_nw_searchdomains %} + - "{{ domain }}" +{% endfor %} + ipMode: + type: {{ ac_nw_ipallocmode }} +{% if (ac_nw_ipallocmode is defined) and ('static' == ac_nw_ipallocmode) %} + ipBlockFilePath: "{{ ac_nw_ipfile }}" +{% endif %} + serviceCIDR: {{ ac_nw_servicecidr }} + podCIDR: {{ ac_nw_podcidr }} + vCenter: + networkName: {{ ac_nw_vc_net }} +loadBalancer: + vips: + controlPlaneVIP: "{{ ac_lb_vips_cp }}" + addonsVIP: "{{ ac_lb_vips_addons }}" + kind: {{ ac_lb_kind }} +antiAffinityGroups: + enabled: {{ ac_antiaffinitygroups }} +connectivity: connected +# (Optional) Specify the proxy configuration +proxy: + url: "{{ glb_proxyurl }}" + noProxy: "{{ glb_noproxy }}" +{% if (glb_privatereg_url is defined) and (glb_privatereg_url|length > 0) %} +# # (Optional) Use a private Docker registry to host GKE images +privateRegistry: + address: "{{ glb_privatereg_url }}" + credentials: + fileRef: + path: credential.yaml + entry: privateRegistry +{% if (glb_privatereg_cacertpath is defined) and (glb_privatereg_cacertpath|length > 0) %} + caCertPath: "{{ glb_privatereg_cacertpath }}" +{% endif %} +{% endif %} +# (Required): The absolute or relative path to the GCP service account key for pulling +# GKE images +componentAccessServiceAccountKeyPath: "{{ job_sakeyfolder }}/{{ component_access_gcpsa_path }}" +# (Optional) Specify which GCP project to connect your GKE clusters to +gkeConnect: + projectID: "{{ ac_stackdriver_projectid }}" + registerServiceAccountKeyPath: "{{ job_sakeyfolder }}/{{ connect_register_gcpsa_path }}" + #registerServiceAccountKeyPath: "{{ job_sakeyfolder }}/connect-register.json" +stackdriver: + projectID: "{{ ac_stackdriver_projectid }}" + clusterLocation: "{{ ac_stackdriver_clusterlocation }}" + enableVPC: {{ ac_stackdriver_enablevpc }} + serviceAccountKeyPath: "{{ job_sakeyfolder }}/{{ logging_monitoring_gcpsa_path }}" + # serviceAccountKeyPath: "{{ job_sakeyfolder }}/logging-monitoring.json" + disableVsphereResourceMetrics: false +# # (Optional) Configure kubernetes apiserver audit logging +{% if (ac_cloudauditlogging_projectid is defined) and (ac_cloudauditlogging_projectid|length > 0) %} +cloudAuditLogging: + projectID: "{{ ac_cloudauditlogging_projectid }}" + # A GCP region where you would like to store audit logs for this cluster. + clusterLocation: "{{ ac_cloudauditlogging_clusterlocation }}" + # The absolute or relative path to the key file for a GCP service account used to + # send audit logs from the cluster + serviceAccountKeyPath: "{{ job_sakeyfolder }}/{{ audit_logging_gcpsa_path }}" + # serviceAccountKeyPath: "{{ job_sakeyfolder }}/audit-logging.json" +{% endif %} +# # (Optional/Preview) Configure backups for admin cluster. Backups will be stored under +# # /anthos-backups/ +clusterBackup: + datastore: "{{ ac_vc_datastore }}" +autoRepair: + enabled: {{ ac_autorepair }} +{% if (ac_secretsencryption_mode is defined) and ('GeneratedKey' == ac_secretsencryption_mode) %} +secretsEncryption: + # Secrets Encryption Mode. Possible values are: None GeneratedKey + mode: "{{ ac_secretsencryption_mode }}" + # GeneratedKey Secrets Encryption config + generatedKey: + # # key version + keyVersion: {{ ac_secretsencryption_keyversion }} +{% endif %} +# (Optional) Specify the type of OS image; available options can be set to "ubuntu_containerd" +# or "cos". Default is "ubuntu_containerd". +osImageType: "ubuntu_containerd" diff --git a/tools/anthosvmware-ansible-module/roles/admincluster/templates/cluster-ip-block.yaml.j2 b/tools/anthosvmware-ansible-module/roles/admincluster/templates/cluster-ip-block.yaml.j2 new file mode 100644 index 0000000000..bfaf8853a0 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/admincluster/templates/cluster-ip-block.yaml.j2 @@ -0,0 +1,22 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +blocks: + - netmask: {{ ac_nw_nm }} + gateway: {{ ac_nw_gw }} + ips: +{% for ip in ac_ipblock_ips %} + - ip: {{ ip }} +{% endfor %} diff --git a/tools/anthosvmware-ansible-module/roles/admincluster/templates/private-reg-ca.crt.j2 b/tools/anthosvmware-ansible-module/roles/admincluster/templates/private-reg-ca.crt.j2 new file mode 100644 index 0000000000..7686f7267c --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/admincluster/templates/private-reg-ca.crt.j2 @@ -0,0 +1 @@ +{{ glb_privateregistry_ca_content }} \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/adminws/README.md b/tools/anthosvmware-ansible-module/roles/adminws/README.md new file mode 100644 index 0000000000..3c2db4f58f --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/adminws/README.md @@ -0,0 +1,89 @@ +# Ansible Role: adminws + +Install and configure the Admin Workstation. Can also uninstall and upgrade the Admin Workstation. + +## Requirements + +No special requirements. + +## Role Variables + +Available variables are listed below, along with default values (see `defaults/main.yml`): + +```YAML +adminws_install: true +``` + +Whether to install `true` or uninstall `false`. + +```YAML +adminws_name: "" +``` + +Name of the Admin Workstation. This name shows as the VM-name in the vSphere web interface. + +```YAML +component_access_gcpsa_path: "component-access.json" +``` + +Destination file name of the Anthos component access Google Service Account on the build server / jump host from which the Admin Workstation is being build. + +```YAML +# vSphere/vCenter +adminws_vc_fqdn: '{{ lookup("env", "VMWARE_HOST") }}' +adminws_vc_validate_cert: true +adminws_vc_credfile: "credential.yaml" +adminws_vc_credentry: "vCenter" +adminws_vc_datacenter: "" +adminws_vc_datastore: "" +adminws_vc_cluster: "" +adminws_vc_network: "" # VM Network +adminws_vc_folder: "" # optional +adminws_vc_respool: "" # if default resourcePool use /Resources +adminws_vc_cacertpath: "{{ yamldestpath }}/vcenter.pem" # Location for automatically downloaded CA cert +adminws_datadiskname: "" # -gke-on-prem-admin-workstation-data-disk/-gke-admin-ws-data-disk.vmdk +``` + +The vSphere/vCenter-specific settings specify how to connect and with what credentials. +It includes the validation of the vSphere TLS certificate material, the vSphere object names, and +the file name of the Admin Workstation data disk. +Uses Ansible `lookup` to read sensitive information from injected environment variables. +This way, you can use AWX or Ansible Tower Vault to securely consume secrets. + +```YAML +# Networking +adminws_nw_ipallocmode: "" # dhcp or static +adminws_nw_ip: "" # IP address of VM +adminws_nw_gw: "" # gateway +adminws_nw_nm: "" # netmask 255.255.255.0 or similar +adminws_nw_dns: [""] +adminws_ntp: "" # only one can be set default ntp.ubuntu.com +``` + +Network settings. + +```YAML +adminws_skipvalidations: "" +# adminws_skipvalidations: "--skip-validation" +``` + +Skip validation checks for temporary workarounds in the deployment environment. + +## Dependencies + +None. + +## Example Playbook + +```YAML +- name: "[adminws] Installation" + hosts: all + gather_facts: False + roles: + - adminws +``` + +## **License** + +Copyright 2022 Google LLC. This software is provided as-is, without warranty or representation for any use or purpose. +Your use of it is subject to your agreement with Google. diff --git a/tools/anthosvmware-ansible-module/roles/adminws/defaults/main.yml b/tools/anthosvmware-ansible-module/roles/adminws/defaults/main.yml new file mode 100644 index 0000000000..b0c1e3fa34 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/adminws/defaults/main.yml @@ -0,0 +1,49 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# role adminws default values +sshprivkeypath: "/home/user/.ssh/x_id_ed25519" + +# admin workstation specific below until end of file +adminws_install: true +adminws_name: "" +component_access_gcpsa_path: "component-access.json" + +# vSphere/vCenter +adminws_vc_fqdn: '{{ lookup("env", "VMWARE_HOST") }}' +adminws_vc_validate_cert: true +adminws_vc_credfile: "credential.yaml" +adminws_vc_credentry: "vCenter" +adminws_vc_datacenter: "" +adminws_vc_datastore: "" +adminws_vc_cluster: "" +adminws_vc_network: "" # VM Network +adminws_vc_folder: "" # optional +adminws_vc_respool: "" # if default resourcePool use /Resources +adminws_vc_cacertpath: "{{ yamldestpath }}/vcenter.pem" # Location for automatically downloaded CA cert +adminws_datadiskname: "" # -gke-on-prem-admin-workstation-data-disk/-gke-admin-ws-data-disk.vmdk + +# Networking +adminws_nw_ipallocmode: "" # dhcp or static +adminws_nw_ip: "" # IP address of VM +adminws_nw_gw: "" # gateway +adminws_nw_nm: "" # netmask 255.255.255.0 or similar +adminws_nw_dns: [""] +adminws_ntp: "" # only one can be set default ntp.ubuntu.com + +adminws_skipvalidations: "" +# adminws_skipvalidations: "--skip-validation" + +# upgrade +adminws_upgrade: false diff --git a/tools/anthosvmware-ansible-module/roles/adminws/tasks/asserts.yml b/tools/anthosvmware-ansible-module/roles/adminws/tasks/asserts.yml new file mode 100644 index 0000000000..2b63937414 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/adminws/tasks/asserts.yml @@ -0,0 +1,44 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[adminws] Sanity Checks" + ansible.builtin.assert: + that: + - yamldestpath | length >= 6 + - sakeyfolder_base is defined + - adminws_name | length >= 4 + - adminws_vc_fqdn | length >= 4 + - adminws_vc_credentry | length >= 2 + - glb_vc_username | length >= 4 + - glb_vc_password | length >= 8 + - adminws_vc_datacenter | length >= 2 + - adminws_vc_datastore | length >= 2 + - adminws_vc_cluster | length >= 2 + - adminws_vc_network | length >= 2 + - adminws_vc_respool | length >= 4 + - adminws_vc_cacertpath is defined + - adminws_datadiskname | length >= 4 + - adminws_nw_ipallocmode is search("dhcp") or adminws_nw_ipallocmode is search("static") + - adminws_nw_ip | length >= 7 + - adminws_nw_gw | length >= 7 + - adminws_nw_nm | length >= 7 + - adminws_nw_dns + - adminws_ntp | length >= 7 + - adminws_create_gsa == adminws_create_gsa|bool + fail_msg: "Undefined variables." + success_msg: "All required variables are defined." + +- name: "[adminws] Note on optional values" + ansible.builtin.debug: + msg: "Optional values adminws_vc_folder are not checked. Please double check if required." diff --git a/tools/anthosvmware-ansible-module/roles/adminws/tasks/install.yml b/tools/anthosvmware-ansible-module/roles/adminws/tasks/install.yml new file mode 100644 index 0000000000..6cba1d1d64 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/adminws/tasks/install.yml @@ -0,0 +1,216 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[adminws] Create folder" + ansible.builtin.file: + path: "{{ yamldestpath }}" + state: directory + mode: 0770 + +- name: "[adminws] Check for private download" + ansible.builtin.set_fact: + is_private_download: "{{ true if private_download_base is defined and private_download_base|length > 8 and private_download_auth_header is defined and private_download_auth_header|length > 5 else false }}" + +- name: "[adminws] Block - Copy/delete GCP SA JSON key files, download vCenter certs, create workstation" + block: + + - name: "Include role for GCP SA JSON key files" + ansible.builtin.include_role: + name: copy_credentials + vars: + is_admin_workstation: true + + - name: "[adminws] Switch to Component Access Service Account" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gcloud + - auth + - activate-service-account + - --key-file + - "{{ job_sakeyfolder }}/{{ component_access_gcpsa_path }}" + + - name: "[adminws] Download gkeadm binary from private location" + ansible.builtin.get_url: + url: "{{ private_download_base }}/anthos/{{ glb_anthos_version }}/gkeadm-{{ glb_anthos_version }}" + headers: + Authorization: "{{ private_download_auth_header }}" + dest: "{{ yamldestpath }}/gkeadm-{{ glb_anthos_version }}" + mode: 0600 + when: is_private_download|bool + + - name: "[adminws] Download gkeadm binary" + ansible.builtin.command: # noqa 204 301 + chdir: "{{ yamldestpath }}" + argv: + - gsutil + - cp + - gs://gke-on-prem-release/gkeadm/{{ glb_anthos_version }}/linux/gkeadm + - "{{ yamldestpath }}/gkeadm-{{ glb_anthos_version }}" + when: not (is_private_download|bool) + + - name: "[adminws] Download gkeadm signature from private location" + ansible.builtin.get_url: + url: "{{ private_download_base }}/anthos/{{ glb_anthos_version }}/gkeadm-{{ glb_anthos_version }}.1.sig" + headers: + Authorization: "{{ private_download_auth_header }}" + dest: "{{ yamldestpath }}/gkeadm-{{ glb_anthos_version }}.1.sig" + mode: 0600 + when: is_private_download|bool + + - name: "[adminws] Download gkeadm signature" + ansible.builtin.command: # noqa 204 301 + chdir: "{{ yamldestpath }}" + argv: + - gsutil + - cp + - gs://gke-on-prem-release/gkeadm/{{ glb_anthos_version }}/linux/gkeadm.1.sig + - "{{ yamldestpath }}/gkeadm-{{ glb_anthos_version }}.1.sig" + when: not (is_private_download|bool) + + - name: "[adminws] Copy gkeadm signature key" + ansible.builtin.copy: + content: | + -----BEGIN PUBLIC KEY----- + MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZ + wQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw== + -----END PUBLIC KEY----- + dest: "{{ yamldestpath }}/gkeadm.1.key" + mode: 0600 + + - name: "[adminws] Verify gkeadm binary" + ansible.builtin.command: + argv: + - openssl + - dgst + - -verify + - "{{ yamldestpath }}/gkeadm.1.key" + - -signature + - "{{ yamldestpath }}/gkeadm-{{ glb_anthos_version }}.1.sig" + - -binary + - "{{ yamldestpath }}/gkeadm-{{ glb_anthos_version }}" + changed_when: false + + - name: "[adminws] Set file permissions on gkeadm binary" + ansible.builtin.file: + dest: "{{ yamldestpath }}/gkeadm-{{ glb_anthos_version }}" + mode: 0775 + + - name: "[adminws] Copy to generic gkeadm path" + ansible.builtin.copy: + src: "{{ yamldestpath }}/gkeadm-{{ glb_anthos_version }}" + dest: "{{ yamldestpath }}/gkeadm" + remote_src: true + mode: 0775 + + - name: "[adminws] Create vCenter CA cert directory" + ansible.builtin.file: + path: "{{ yamldestpath }}/vcenter" + mode: 0700 + state: directory + + - name: "[adminws] Download vCenter CA cert bundle" + ansible.builtin.command: # noqa 303 + chdir: "{{ yamldestpath }}" + argv: + - curl + - -k + - https://{{ adminws_vc_fqdn }}/certs/download.zip + - -o + - "{{ yamldestpath }}/vcenter/download.zip" + creates: "{{ yamldestpath }}/vcenter/download.zip" + + - name: "[adminws] Extract vCenter CA cert bundle" + ansible.builtin.unarchive: + src: "{{ yamldestpath }}/vcenter/download.zip" + dest: "{{ yamldestpath }}/vcenter" + creates: "{{ yamldestpath }}/vcenter/certs/lin" + remote_src: true + validate_certs: "{{ adminws_vc_validate_cert | bool }}" + + - name: "[adminws] Get vCenter CA cert file names" + ansible.builtin.find: + path: "{{ yamldestpath }}/vcenter/certs/lin" + pattern: '*.0' + register: cert_files + + - name: "[adminws] Get vCenter CA cert details" + ansible.builtin.command: + argv: + - openssl + - x509 + - -issuer + - -subject + - -nocert + - -in + - "{{ item.path }}" + loop: "{{ cert_files.files }}" + loop_control: + label: "{{ item.path }}" + register: cert_details + changed_when: false + + - name: "[adminws] Select the root vCenter CA cert" + ansible.builtin.copy: + src: "{{ item.cmd[6] }}" # References the file name in the openssl command used to generate the details + dest: "{{ yamldestpath }}/vcenter.pem" + remote_src: true + mode: 0600 + loop: "{{ cert_details.results }}" + loop_control: + label: "{{ item.cmd[6] }}" # References the file name in the openssl command used to generate the details + when: item.stdout_lines[0] | replace('issuer=', '') == item.stdout_lines[1] | replace('subject=', '') + + - name: "[adminws] Templating YAML files - config" + ansible.builtin.template: + src: admin-ws-config.yaml.j2 + dest: "{{ yamldestpath }}/admin-ws-config-{{ adminws_name }}.yaml" + mode: 0600 + + - name: "[adminws] Pre-download admin workstation OVA from private location" + ansible.builtin.get_url: + url: "{{ private_download_base }}/anthos/{{ glb_anthos_version }}/gke-on-prem-admin-appliance-vsphere-{{ glb_anthos_version }}.ova" + headers: + Authorization: "{{ private_download_auth_header }}" + dest: "{{ yamldestpath }}/gke-on-prem-admin-appliance-vsphere-{{ glb_anthos_version }}.ova" + mode: 0600 + when: is_private_download|bool + + - name: "[adminws] Create admin workstation" + ansible.builtin.command: # noqa 204 + chdir: "{{ yamldestpath }}" + argv: "{{ _argv }}" + creates: "{{ adminws_name }}" + vars: + _req_args: + - "gkeadm-{{ glb_anthos_version }}" + - create + - admin-workstation + - --config + - "admin-ws-config-{{ adminws_name }}.yaml" + - --ssh-key-path + - "{{ sshprivkeypath }}" + - "{{ '--auto-create-service-accounts' if adminws_create_gsa else '' }}" + _skips: "{{ adminws_skipvalidations if adminws_skipvalidations | type_debug == 'list' else (adminws_skipvalidations.split(' ')) }}" # noqa 204 + _argv: "{{ _req_args + _skips }}" + + environment: # noqa 204 + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" # noqa 204 + http_proxy: "{{ glb_proxyurl }}" + https_proxy: "{{ glb_proxyurl }}" + no_proxy: "{{ glb_noproxy }}" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup diff --git a/tools/anthosvmware-ansible-module/roles/adminws/tasks/main.yml b/tools/anthosvmware-ansible-module/roles/adminws/tasks/main.yml new file mode 100644 index 0000000000..46ee254eb0 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/adminws/tasks/main.yml @@ -0,0 +1,42 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[adminws] Include tasks - asserts.yml" + include_tasks: asserts.yml + +- name: "[adminws] check if file {{ adminws_name }} exists" + ansible.builtin.stat: + path: "{{ yamldestpath }}/{{ adminws_name }}" + register: adminws_file_exists + +- name: "[adminws] Include tasks - install.yml" + include_tasks: install.yml + when: + - adminws_install|default(false)|bool + - not adminws_upgrade|default(false)|bool + - not adminws_file_exists.stat.exists + +- name: "[adminws] Include tasks - uninstall.yml" + include_tasks: uninstall.yml + when: + - not adminws_install|default(false)|bool + - not adminws_upgrade|default(false)|bool + - adminws_file_exists.stat.exists + +- name: "[adminws] Include tasks - upgrade.yml" + include_tasks: upgrade.yml + when: + - adminws_upgrade|default(false)|bool + - adminws_file_exists.stat.exists diff --git a/tools/anthosvmware-ansible-module/roles/adminws/tasks/uninstall.yml b/tools/anthosvmware-ansible-module/roles/adminws/tasks/uninstall.yml new file mode 100644 index 0000000000..70870a38b6 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/adminws/tasks/uninstall.yml @@ -0,0 +1,55 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[ac] Clean up old SSH host keys from incomplete runs" + ansible.builtin.file: + path: "/home/ubuntu/.ssh/known_hosts" + state: absent + +- name: "Include role for GCP SA JSON key files" + ansible.builtin.include_role: + name: copy_credentials + vars: + is_admin_workstation: true + +# Ansible Lint skipping rules +# ruleID 301 Commands should not change things if nothing needs doing +# ruleID 204 Lines should be no longer than 160 chars +- name: "[adminws] Block - Delete workstation, delete credential file" + block: + - name: "[adminws] Delete admin workstation" + ansible.builtin.command: # noqa 305 204 301 + chdir: "{{ yamldestpath }}" + argv: + - "gkeadm-{{ glb_anthos_version }}" + - delete + - admin-workstation + - --config + - "admin-ws-config-{{ adminws_name }}.yaml" + - --force + - --skip-validation + environment: # noqa 204 + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" # noqa 204 + http_proxy: "{{ glb_proxyurl }}" + https_proxy: "{{ glb_proxyurl }}" + no_proxy: "{{ glb_noproxy }}" + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup + +- name: "[adminws] Delete admin workstation status file" + ansible.builtin.file: + path: "{{ yamldestpath }}/{{ adminws_name }}" + state: absent diff --git a/tools/anthosvmware-ansible-module/roles/adminws/tasks/upgrade.yml b/tools/anthosvmware-ansible-module/roles/adminws/tasks/upgrade.yml new file mode 100644 index 0000000000..04f8612a96 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/adminws/tasks/upgrade.yml @@ -0,0 +1,125 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[adminws] Block - Copy/delete GCP SA JSON key files, download gkeadm, upgrade workstation" + block: + + - name: "Include role for GCP SA JSON key files" + ansible.builtin.include_role: + name: copy_credentials + vars: + is_admin_workstation: true + + - name: "[adminws] Templating YAML files - config" + ansible.builtin.template: + src: admin-ws-config.yaml.j2 + dest: "{{ yamldestpath }}/admin-ws-config-{{ adminws_name }}.yaml" + mode: 0600 + + - name: "[adminws] Switch to Component Access Service Account" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gcloud + - auth + - activate-service-account + - --key-file + - "{{ job_sakeyfolder }}/{{ component_access_gcpsa_path }}" + + - name: "[adminws] gkeadm upgrade check status file" + ansible.builtin.stat: + path: "{{ gkeadm }}-{{ glb_anthos_version }}-download-status" + register: download_gkeadm_check_file + + - name: "[adminws] Create versioned gkeadm binary" + ansible.builtin.copy: + src: "{{ yamldestpath }}/gkeadm" + dest: "{{ yamldestpath }}/gkeadm-{{ glb_anthos_version }}" + remote_src: true + mode: 0775 + when: not download_gkeadm_check_file.stat.exists + + - name: "[adminws] Update gkeadm binary" + ansible.builtin.command: # noqa 204 + chdir: "{{ yamldestpath }}" + argv: "{{ _argv }}" + vars: + _req_args: + - "{{ yamldestpath }}/gkeadm-{{ glb_anthos_version }}" + - upgrade + - gkeadm + - --target-version + - "{{ glb_anthos_version }}" + - --config + - "admin-ws-config-{{ adminws_name }}.yaml" + _skips: "{{ adminws_skipvalidations if adminws_skipvalidations | type_debug == 'list' else (adminws_skipvalidations.split(' ')) }}" # noqa 204 + _argv: "{{ _req_args + _skips }}" + register: gkeadm_upgrade_result + when: not download_gkeadm_check_file.stat.exists + + - name: "[adminws] Copy to generic gkeadm path" + ansible.builtin.copy: + src: "{{ yamldestpath }}/gkeadm-{{ glb_anthos_version }}" + dest: "{{ yamldestpath }}/gkeadm" + remote_src: true + mode: 0775 + when: not download_gkeadm_check_file.stat.exists + + - name: "[adminws] gkeadm upgrade results" + ansible.builtin.file: + path: "{{ gkeadm }}-{{ glb_anthos_version }}-download-status" + state: touch + mode: 0600 + when: gkeadm_upgrade_result.rc | default('') == 0 + + - name: "[adminws] adminws upgrade check status file" + ansible.builtin.stat: + path: "{{ yamldestpath }}/{{ adminws_name }}-{{ glb_anthos_version }}-upgrade-status" + register: upgrade_adminws_check_file + + - name: "[adminws] Upgrade AdminWS" + ansible.builtin.command: # noqa 204 + chdir: "{{ yamldestpath }}" + argv: "{{ _argv }}" + creates: "{{ yamldestpath }}/gke-on-prem-admin-appliance-vsphere-{{ glb_anthos_version }}.ova" + vars: + _req_args: + - "{{ yamldestpath }}/gkeadm-{{ glb_anthos_version }}" + - upgrade + - admin-workstation + - --config + - "admin-ws-config-{{ adminws_name }}.yaml" + - --info-file + - "{{ adminws_name }}" + _skips: "{{ adminws_skipvalidations if adminws_skipvalidations | type_debug == 'list' else (adminws_skipvalidations.split(' ')) }}" # noqa 204 + _argv: "{{ _req_args + _skips }}" + register: adminws_upgrade_result + when: not upgrade_adminws_check_file.stat.exists + + - name: "[adminws] adminws upgrade results" + ansible.builtin.file: + path: "{{ yamldestpath }}/{{ adminws_name }}-{{ glb_anthos_version }}-upgrade-status" + state: touch + mode: 0600 + when: adminws_upgrade_result.rc | default('') == 0 + + environment: # noqa 204 + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" # noqa 204 + http_proxy: "{{ glb_proxyurl }}" + https_proxy: "{{ glb_proxyurl }}" + no_proxy: "{{ glb_noproxy }}" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup diff --git a/tools/anthosvmware-ansible-module/roles/adminws/templates/admin-ws-config.yaml.j2 b/tools/anthosvmware-ansible-module/roles/adminws/templates/admin-ws-config.yaml.j2 new file mode 100644 index 0000000000..9bea028412 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/adminws/templates/admin-ws-config.yaml.j2 @@ -0,0 +1,52 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +gcp: + componentAccessServiceAccountKeyPath: "{{ job_sakeyfolder }}/{{ component_access_gcpsa_path }}" +vCenter: + credentials: + address: "{{ adminws_vc_fqdn }}" + fileRef: + path: {{ adminws_vc_credfile }} + entry: {{ adminws_vc_credentry }} + datacenter: "{{ adminws_vc_datacenter }}" + datastore: "{{ adminws_vc_datastore }}" + cluster: "{{ adminws_vc_cluster }}" + network: "{{ adminws_vc_network }}" +{% if adminws_vc_folder | length > 2 %} + folder: "{{ adminws_vc_folder }}" +{% endif %} + resourcePool: "{{ adminws_vc_respool }}" + caCertPath: "{{ adminws_vc_cacertpath }}" +proxyUrl: "{{ glb_proxyurl }}" +adminWorkstation: + name: {{ adminws_name }} + cpus: 4 + memoryMB: 8192 + diskGB: 50 + dataDiskName: {{ adminws_datadiskname }} + dataDiskMB: 512 + network: + ipAllocationMode: "{{ adminws_nw_ipallocmode }}" + hostConfig: + ip: "{{ adminws_nw_ip }}" + gateway: "{{ adminws_nw_gw }}" + netmask: "{{ adminws_nw_nm }}" + dns: +{% for ip in adminws_nw_dns %} + - "{{ ip }}" +{% endfor %} + proxyUrl: "{{ glb_proxyurl }}" + ntpServer: "{{ adminws_ntp }}" diff --git a/tools/anthosvmware-ansible-module/roles/ais/README.md b/tools/anthosvmware-ansible-module/roles/ais/README.md new file mode 100644 index 0000000000..02e5931770 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/ais/README.md @@ -0,0 +1,80 @@ +# Ansible Role: ais + +Installs and configures the Anthos Identity Service (AIS). + +## Requirements + +No special requirements. + +## Role Variables + +Available variables are listed below, along with default values (see `defaults/main.yml`): + +```yaml +ais_kubeconfig: "{{ uc_name + '-kubeconfig' if uc_name is defined else 'kubeconfig' }}" +ais_patch_file: "{{ uc_name + '-ais-patch.yaml' if uc_name is defined else 'admin-ais-patch.yaml' }}" +ais_login_config_file: "{{ uc_name + '-login-config.yaml' if uc_name is defined else 'admin-login-config.yaml' }}" +``` + +Admin or User cluster `kubeconfig` file name. +AIS patch file for its Kubernetes resource. +File name for the cluster login config file. + +```yaml +# OIDC auth values +ais_oidc_clientid: '{{ lookup("env", "AIS_OIDC_CLIENTID") }}' +ais_oidc_clientsecret: '{{ lookup("env", "AIS_OIDC_CLIENTSECRET") }}' +``` + +OIDC Client ID and Client secret values to be used in AIS configuration. + +```yaml +# AIS configuration +ais_authentication: +- name: oidc-ad + oidc: + clientID: '{{ ais_oidc_clientid }}' + clientSecret: '{{ ais_oidc_clientsecret }}' + cloudConsoleRedirectURI: "" + extraParams: prompt=consent,access_type=offline + issuerURI: "" + kubectlRedirectURI: "" + scopes: "" + userClaim: "" +``` + +AIS configuration block that will be added to the cluster ClientConfig. + +```yaml +# GCP related +ais_gcpsa: '{{ lookup("env", "GCPSA_AIS_FILE") }}' +ais_gcpsa_path: "ais-to-gcs.json" +ais_gcsbucket: 'gs://{{ lookup("env", "AIS_GCS_BUCKET") }}' +``` + +Optionally, reference a Google Service Account with privileges to write to an existing Google Cloud Storage bucket +to store the AIS cluster login config file. + +## Dependencies + +- `admincluster` role, or +- `usercluster` role + +## Example Playbook + +This role typically runs after the Admin or User Cluster installation. +The example below shows the `ais` role after the `admincluster` role. + +```yaml +# Configure the Anthos Identity Service +- name: "[ais] Configuration" + hosts: all + gather_facts: false + roles: + - ais +``` + +## **License** + +Copyright 2022 Google LLC. This software is provided as-is, without warranty or representation for any use or purpose. +Your use of it is subject to your agreement with Google. diff --git a/tools/anthosvmware-ansible-module/roles/ais/defaults/main.yml b/tools/anthosvmware-ansible-module/roles/ais/defaults/main.yml new file mode 100644 index 0000000000..59fc85d3e2 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/ais/defaults/main.yml @@ -0,0 +1,40 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Role default values +ais_kubeconfig: "{{ uc_name + '-kubeconfig' if uc_name is defined else 'kubeconfig' }}" +ais_patch_file: "{{ uc_name + '-ais-patch.yaml' if uc_name is defined else 'admin-ais-patch.yaml' }}" +ais_login_config_file: "{{ uc_name + '-login-config.yaml' if uc_name is defined else 'admin-login-config.yaml' }}" + +# OIDC auth values +ais_oidc_clientid: '{{ lookup("env", "AIS_OIDC_CLIENTID") }}' +ais_oidc_clientsecret: '{{ lookup("env", "AIS_OIDC_CLIENTSECRET") }}' + +# AIS configuration +ais_authentication: +- name: oidc-ad + oidc: + clientID: "{{ ais_oidc_clientid if ais_oidc_clientid is defined else '' }}" + clientSecret: "{{ ais_oidc_clientsecret if ais_oidc_clientsecret is defined else '' }}" + cloudConsoleRedirectURI: "" + extraParams: prompt=consent,access_type=offline + issuerURI: "" + kubectlRedirectURI: "" + scopes: "" + userClaim: "" + +# GCP related (Google Service Account, GCS bucket name) +ais_gcpsa: '{{ lookup("env", "GCPSA_AIS_FILE") }}' +ais_gcpsa_path: "ais-to-gcs.json" +ais_gcsbucket: 'gs://{{ lookup("env", "AIS_GCS_BUCKET") }}' diff --git a/tools/anthosvmware-ansible-module/roles/ais/tasks/asserts.yml b/tools/anthosvmware-ansible-module/roles/ais/tasks/asserts.yml new file mode 100644 index 0000000000..103c5d9475 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/ais/tasks/asserts.yml @@ -0,0 +1,38 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[ais] Check ais_authentication block" + ansible.builtin.assert: + that: + - yamldestpath | length >= 6 + - ais_authentication is defined + - ais_authentication | type_debug == 'list' + - ais_authentication | length > 0 + - ais_oidc_clientid | length > 5 + - ais_oidc_clientsecret | length > 5 + fail_msg: "ais_authentication block not defined or formatted correctly, it should be a list" + success_msg: "ais_authentication block formatted correctly" + when: + - ais_authentication is defined + +- name: "[ais] Check ais_ldap_secrets block" + ansible.builtin.assert: + that: + - ais_ldap_secrets is defined + - ais_ldap_secrets | type_debug == 'list' + - ais_ldap_secrets | length > 0 + fail_msg: "ais_ldap_secrets block not defined or formatted correctly, it should be a list" + success_msg: "ais_ldap_secrets block formatted correctly" + when: + - ais_ldap_secrets is defined diff --git a/tools/anthosvmware-ansible-module/roles/ais/tasks/install.yml b/tools/anthosvmware-ansible-module/roles/ais/tasks/install.yml new file mode 100644 index 0000000000..bee0cbfbda --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/ais/tasks/install.yml @@ -0,0 +1,171 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[ais] Set kubeconfig path and patch file name" + ansible.builtin.set_fact: + ais_kubeconfig: "{{ uc_name + '-kubeconfig' if uc_name is defined else 'kubeconfig' }}" + ais_patch_file: "{{ uc_name + '-ais-patch.yaml' if uc_name is defined else 'admin-ais-patch.yaml' }}" + ais_login_config_file: "{{ uc_name + '-login-config.yaml' if uc_name is defined else 'admin-login-config.yaml' }}" + ais_ldap_tmp_dir: /tmp/{{ uc_name if uc_name is defined else 'admin' }}-ldap-secrets + ais_dir: yamldestpath/{{ uc_name if uc_name is defined else ac_name }} + +- name: "[ais] Manage LDAP service account secrets" + block: + - name: "[ais] Create LDAP secret template directory" + ansible.builtin.file: + path: "{{ ais_ldap_tmp_dir }}" + state: directory + when: + - ais_authentication is defined + - ais_ldap_secrets is defined + + - name: "[ais] Template LDAP secrets" + ansible.builtin.template: + src: ais-ldap-basic-auth-secret.yaml.j2 + dest: "{{ [ais_ldap_tmp_dir, item.authentication_name + '-ldap-secret.yaml'] | join('/') }}" + mode: 0600 + loop: "{{ ais_ldap_secrets }}" + loop_control: + label: "{{ item.authentication_name }}" + no_log: true + when: + - ais_authentication is defined + - ais_ldap_secrets is defined + + - name: "[ais] Refresh LDAP secrets (cleanup)" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ ais_kubeconfig }}" + - delete + - secret + - "{{ (ais_authentication | selectattr('name', 'search', item.authentication_name) | list | first).ldap.serviceAccountSecret.name }}" + - -n + - "{{ (ais_authentication | selectattr('name', 'search', item.authentication_name) | list | first).ldap.serviceAccountSecret.namespace }}" + - --ignore-not-found + loop: "{{ ais_ldap_secrets }}" + loop_control: + label: "{{ item.authentication_name }}" + no_log: true + when: + - ais_authentication is defined + - ais_ldap_secrets is defined + + - name: "[ais] Refresh LDAP secrets (create)" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ ais_kubeconfig }}" + - apply + - -f + - "{{ [ais_ldap_tmp_dir, item.authentication_name + '-ldap-secret.yaml'] | join('/') }}" + loop: "{{ ais_ldap_secrets }}" + loop_control: + label: "{{ item.authentication_name }}" + no_log: true + register: apply_result + when: + - ais_authentication is defined + - ais_ldap_secrets is defined + + - name: "[ais] Output Apply results Masked" + ansible.builtin.debug: + msg: "{{ item | to_json | replace(item.item.password, '****') }}" + loop: "{{ apply_result.results }}" + loop_control: + label: "{{ item.item.authentication_name }}" + when: + - ais_authentication is defined + - ais_ldap_secrets is defined + + always: + - name: "[ais] Cleanup templated LDAP secrets" + ansible.builtin.file: + path: "{{ ais_ldap_tmp_dir }}" + state: absent + when: + - ais_authentication is defined + - ais_ldap_secrets is defined + +- name: "[ais] Create patch file" + ansible.builtin.template: + src: ais-patch-file.yaml.j2 + dest: "{{ [ais_dir, ais_patch_file] | join('/') }}" + mode: 0600 + when: + - ais_authentication is defined + +- name: "[ais] Apply patch to ClientConfigs" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ ais_kubeconfig }}" + - patch + - ClientConfigs + - default + - -n + - kube-public + - --patch-file + - "{{ [ais_dir, ais_patch_file] | join('/') }}" + - --type + - merge + when: + - ais_authentication is defined + +- name: "[ais] Create proxy config for apt-get" + ansible.builtin.template: + src: apt-proxy.conf.j2 + dest: /etc/apt/apt.conf.d/proxy.conf + mode: 0644 + become: true + when: + - ( glb_proxyurl is defined and glb_proxyurl | length > 7 ) + - ais_authentication is defined + +- name: "[ais] Install anthos-auth component" + ansible.builtin.apt: + name: + - google-cloud-sdk-anthos-auth + state: latest + become: true + when: + - ais_authentication is defined + +- name: "[ais] Delete old kubectl login config" + ansible.builtin.file: + path: "{{ [ais_dir, ais_login_config_file] | join('/') }}" + state: absent + +- name: "[ais] Export kubectl login config" + ansible.builtin.command: + argv: + - gcloud + - anthos + - create-login-config + - --kubeconfig + - "{{ ais_kubeconfig }}" + - --output + - "{{ [ais_dir, ais_login_config_file] | join('/') }}" + when: + - ais_authentication is defined + +- name: "[ais] Delete patch file" + ansible.builtin.file: + path: "{{ [ais_dir, ais_patch_file] | join('/') }}" + state: absent + when: + - ais_authentication is defined \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/ais/tasks/main.yml b/tools/anthosvmware-ansible-module/roles/ais/tasks/main.yml new file mode 100644 index 0000000000..ad23a955ab --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/ais/tasks/main.yml @@ -0,0 +1,30 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[ais] Include tasks - asserts.yml" + include_tasks: asserts.yml + +- name: "[ais] Include tasks - install.yml" + include_tasks: install.yml + +- name: "[ais] Include tasks - upload.yml" + include_tasks: upload.yml + when: + - ais_gcpsa is defined + or ais_gcpsa_content is defined + - ais_gcpsa | length > 0 + or ais_gcpsa_content | length > 0 + - ais_gcsbucket is defined + - ais_gcsbucket | length > 0 diff --git a/tools/anthosvmware-ansible-module/roles/ais/tasks/upload.yml b/tools/anthosvmware-ansible-module/roles/ais/tasks/upload.yml new file mode 100644 index 0000000000..122e53995e --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/ais/tasks/upload.yml @@ -0,0 +1,49 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[ais] Block - Copy GCP SA files, upload login config to GCS, clean up" + block: + - name: "[ais] Copy sensitive files" + ansible.builtin.include_role: + name: copy_credentials + vars: + is_ais: true + + - name: "[ais] Switch Google Service Account" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gcloud + - auth + - activate-service-account + - --key-file + - "{{ job_sakeyfolder }}/{{ ais_gcpsa_path }}" + + - name: "[ais] Upload login config file" + ansible.builtin.command: + argv: + - gsutil + - cp + - "{{ [yamldestpath, ais_login_config_file] | join('/') }}" + - "{{ ais_gcsbucket }}/{{ uc_name if uc_name is defined else ac_name }}/{{ ais_login_config_file }}" + + environment: + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" + http_proxy: "{{ glb_proxyurl }}" + https_proxy: "{{ glb_proxyurl }}" + no_proxy: "{{ glb_noproxy }}" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup diff --git a/tools/anthosvmware-ansible-module/roles/ais/templates/ais-ldap-basic-auth-secret.yaml.j2 b/tools/anthosvmware-ansible-module/roles/ais/templates/ais-ldap-basic-auth-secret.yaml.j2 new file mode 100644 index 0000000000..a1f7f37891 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/ais/templates/ais-ldap-basic-auth-secret.yaml.j2 @@ -0,0 +1,24 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +apiVersion: v1 +kind: Secret +metadata: + name: {{ (ais_authentication | selectattr('name', 'search', item.authentication_name) | list | first).ldap.serviceAccountSecret.name }} + namespace: {{ (ais_authentication | selectattr('name', 'search', item.authentication_name) | list | first).ldap.serviceAccountSecret.namespace }} +type: kubernetes.io/basic-auth +stringData: + username: {{ item.username }} + password: {{ item.password }} \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/ais/templates/ais-patch-file.yaml.j2 b/tools/anthosvmware-ansible-module/roles/ais/templates/ais-patch-file.yaml.j2 new file mode 100644 index 0000000000..b0b129c8cb --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/ais/templates/ais-patch-file.yaml.j2 @@ -0,0 +1,3 @@ +spec: + authentication: +{{ ais_authentication | to_nice_yaml(indent=0) | indent(width=4, indentfirst=true) }} diff --git a/tools/anthosvmware-ansible-module/roles/ais/templates/apt-proxy.conf.j2 b/tools/anthosvmware-ansible-module/roles/ais/templates/apt-proxy.conf.j2 new file mode 100644 index 0000000000..553d9ccba1 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/ais/templates/apt-proxy.conf.j2 @@ -0,0 +1,2 @@ +Acquire::http::Proxy "{{ glb_proxyurl }}"; +Acquire::https::Proxy "{{ glb_proxyurl }}"; diff --git a/tools/anthosvmware-ansible-module/roles/asm/README.md b/tools/anthosvmware-ansible-module/roles/asm/README.md new file mode 100644 index 0000000000..b06c633ecc --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/README.md @@ -0,0 +1,129 @@ +# Ansible Role: asm + +Installs and configures Anthos Service Mesh (ASM). Also allows version upgrades. + +## Requirements + +No special requirements. + +## Role Variables + +Available variables are listed below, along with default values (see `defaults/main.yml`): + +``` +asm_install: True +asm_version: "1.13" +asm_minor_version: "1.13.7-asm.3" +asm_revision: "asm-1137-3" # dynamically determined during Ansible execution +asm_asmcli: "asmcli" # installation script name +asm_asmcli_url: "https://storage.googleapis.com/csm-artifacts/asm" +asm_asmcli_version: "1.13.7-asm.3-config1" # full version including the revision +asm_cluster_name: "{{ uc_name }}" +asm_temp: "/tmp/{{ asm_cluster_name }}" +asm_out_dir: "{{ asm_temp }}/asm-{{ asm_version }}" # local directory where ASM artifacts are downloaded and hydrated +``` + +Install/uninstall toggle, version, script name, source and destination. + +``` +# GCP related +asm_gcp_project: "" # GCP project where user cluster is registered +asm_gcp_project_number: "" +asm_mesh_id: "proj-{{ asm_gcp_project_number }}" # consists of proj-[GCP Project Number] +asm_gcpsa: '{{ lookup("env", "GCPSA_ASMCFG_FILE") }}' +asm_gcpsa_path: "asm-meshconfig.json" +asm_user_email: "" +``` + +Google Cloud Project ID and number, ASM Mesh ID, Google Service Account source and destination file name, and admin user email. + +``` +# Mesh config related +asm_kubeconfig: "{{ yamldestpath }}/{{ asm_cluster_name }}-kubeconfig" # absolute path to user cluster kubeconfig +asm_ingress_namespace: "asm-ingress" # Namespace the Ingress Gateways will get deployed in +asm_network_id: "" # Mesh network ID. Must be unique and non-numeric for multi-cluster mesh to work +``` + +Path to User Cluster `kubeconfig` file. +Name of ASM Ingress Gateway Kubernetes namespace. +ASM network ID, especially relevant for multi-cluster meshes with East-West gateways. + +``` +# Offline mode related +asm_offline_mode: False +asm_offline_bundle_file: "asm-1.13.tar.gz" +``` + +Required when using an ASM offline bundle. + +During ASM installation, the`asmcli` install script downloads `kpt` and `asm` packages from the below endpoints: +* `github.com/GoogleCloudPlatform/anthos-service-mesh-packages` +* `github.com/GoogleContainerTools/kpt/releases/download` + +However, these requests are redirected to a different domain - namely, `githubusercontent.com`. + +An offline bundle can be used if the Admin Workstation has restricted access to any of these endpoints (or if a user simply has the desire to use a predownloaded bundle). With an offline bundle, the asmcli installation packages must be downloaded, bundled in a .tar file, and made available to the jumphost where the Ansible playbook run. Ideally, the .tar file is uploaded to an accessible git repo to ensure there is a singular and centralized location for its content. + +Once available in an accessible and centralized location, the jumphost running the automation scripts can download the file for use. Currently, the automation code expects the .tar file to be downloaded into the `/roles/asm/files/` directory. Then, during ASM installation, the above fields would be used to configure the installation to use the bundled package instead of reaching out to Github - therefore, bypassing the requests to the Github domains. + +``` +# OPTIONAL: CERTIFICATE RELATED - Set the flag to True and define the following cert variables to install cert +asm_install_cert: False +# define cert destination paths +asm_ca_cert_path: "{{ asm_temp }}/ca-cert.pem" +asm_ca_key_path: "{{ asm_temp }}/ca-key.pem" +asm_root_cert_path: "{{ asm_temp }}/root-cert.pem" +asm_cert_chain_path: "{{ asm_temp }}/cert-chain.pem" +# define cert source files +asm_cacert_file: '{{ lookup("env", "ASM_CACERT_FILE") }}' +asm_cakey_file: '{{ lookup("env", "ASM_CAKEY_FILE") }}' +asm_rootcert_file: '{{ lookup("env", "ASM_ROOTCERT_FILE") }}' +asm_certchain_file: '{{ lookup("env", "ASM_CERTCHAIN_FILE") }}' +``` + +Optionally, use Istio CA (citadel) and provide certificate material. +Assumes the same Root CA is used for a multi-cluster mesh setup. + +``` +# Optional: Upgrade completion task related +asm_upgrade_completion: False # pass this as command-line argument to playbook asm_upgrade.yml +asm_old_revision: "" # set this to previous revision of ASM installed in the cluster +``` + +Upgrade toggle. + +``` +# Optional: private registry +asm_registry_url: "{{ glb_privatereg_url if glb_privatereg_url is defined else '' }}" # set to private registry used for container images + +# Optional: private registry, ASM hardcoded image paths in configs +asm_csc_tag: "1.12.2-asm.0" # tag used for canonical-service-controller image +asm_kubebuilder_tag: "v0.5.0" # tag used for kubebuilder/kube-rbac-proxy image +``` + +Private container registry parameters. + +> **Private Container Registry Disclaimer:** The custom overlay file entry (`spec.hub` in `roles/asm/templates/control-plane-config.yaml.j2`) is configured to replace most of the public image references used to install ASM with images from the configured private registry. However, there are two images that are hardcoded in the ASM implementation at time of this automation's release (v1.14.0). These images are replaced with specific tasks in the automation. These images are: +>* `gcr.io/gke-release/asm/canonical-service-controller:1.10.3-asm.16` +>* `gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0` +> +>A feature request has been opened with the ASM product team to parameterize these images (like the other images used) so that they can also be replaced with the custom overlay. + +## Dependencies + +- `usercluster` role + +## Example Playbook + +``` +- name: "[asm] Install Anthos Service Mesh" + hosts: all + gather_facts: false + roles: + - asm +``` + +## **License** + +Copyright 2022 Google LLC. This software is provided as-is, without warranty or representation for any use or purpose. +Your use of it is subject to your agreement with Google. diff --git a/tools/anthosvmware-ansible-module/roles/asm/defaults/main.yml b/tools/anthosvmware-ansible-module/roles/asm/defaults/main.yml new file mode 100644 index 0000000000..1eb4a52f32 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/defaults/main.yml @@ -0,0 +1,96 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Role default values +asm_install: true +asm_version: "1.13" +asm_revision: "asm-1137-3" # dynamically determined during Ansible execution +asm_asmcli: "asmcli" # installation script name +asm_asmcli_url: "https://storage.googleapis.com/csm-artifacts/asm" +asm_asmcli_version: "1.13.7-asm.3-config1" # full version including the revision +asm_cluster_name: "{{ uc_name }}" +asm_temp: "/tmp/{{ asm_cluster_name }}" +asm_out_dir: "{{ asm_temp }}/asm-{{ asm_version }}" # local directory where ASM artifacts are downloaded and hydrated +asm_copy_istioctl: true # set to true if istioctl is required to be copied to a different location +istioctl_dest_path: "/usr/bin" # location where istioctl should be copied +asm_debug_mode: false # set to true to keep the asm_temp directory in adminWS for troubleshooting + + +# GCP related +asm_gcp_project: "" # GCP project where user cluster is registered +asm_gcp_project_number: "" +asm_mesh_id: "proj-{{ asm_gcp_project_number }}" # consists of proj-[GCP Project Number] +asm_gcpsa: '{{ lookup("env", "GCPSA_ASMCFG_FILE") }}' +asm_gcpsa_path: "asm-meshconfig.json" +asm_user_email: "" + +# Mesh config related +asm_kubeconfig: "{{ yamldestpath }}/{{ asm_cluster_name }}-kubeconfig" # absolute path to user cluster kubeconfig +asm_ingress_namespace: "asm-ingress" # Namespace the Ingress Gateways will get deployed in +asm_network_id: "" # Mesh network ID. Must be unique and non-numeric for multi-cluster mesh to work + +# Gateway related +asm_tls_cred_name: "ingress-wildcard" +asm_tlscert_file: '{{ lookup("env", "ASM_TLSCERT_FILE") }}' # file holding the server-side TLS certificate +asm_tlskey_file: '{{ lookup("env", "ASM_TLSKEY_FILE") }}' # file holding the server’s private key +asm_tls_cert_path: "{{ asm_temp }}/tls-cert.pem" # path to the file holding the server-side TLS certificate to use +asm_tls_key_path: "{{ asm_temp }}/tls-key.pem" # path to the file holding the server’s private key +asm_ingress_hosts: [] # list + +# Offline mode related +asm_offline_mode: false +asm_offline_bundle_file: "asm-1.13.7.tar.gz" + +# OPTIONAL: CERTIFICATE RELATED - Set the flag to True and define the following cert variables to install cert +asm_install_cert: false +# define cert destination paths +asm_ca_cert_path: "{{ asm_temp }}/ca-cert.pem" +asm_ca_key_path: "{{ asm_temp }}/ca-key.pem" +asm_root_cert_path: "{{ asm_temp }}/root-cert.pem" +asm_cert_chain_path: "{{ asm_temp }}/cert-chain.pem" +# define cert source files +asm_cacert_file: '{{ lookup("env", "ASM_CACERT_FILE") }}' +asm_cakey_file: '{{ lookup("env", "ASM_CAKEY_FILE") }}' +asm_rootcert_file: '{{ lookup("env", "ASM_ROOTCERT_FILE") }}' +asm_certchain_file: '{{ lookup("env", "ASM_CERTCHAIN_FILE") }}' + +# Optional: Upgrade completion task related +asm_upgrade_completion: false # pass this as command-line argument to playbook asm_upgrade.yml +asm_old_revision: "" # set this to previous revision of ASM installed in the cluster + +# Optional: private registry +asm_registry_url: "{{ glb_privatereg_url if glb_privatereg_url is defined else '' }}" # set to private registry used for container images +asm_minor_version: "1.13.7-asm.3" # required when using private registry + +# Optional: private registry, ASM hardcoded image paths in configs +asm_csc_tag: "1.10.3-asm.16" # tag used for canonical-service-controller image +asm_kubebuilder_tag: "v0.5.0" # tag used for kubebuilder/kube-rbac-proxy image + +## Optional: private registry images, uploaded to artifactory if glb_privatereg_url is defined +asm_container_images: + "1.13.7-asm.3": + - gcr.io/gke-release/asm/pilot:1.13.7-asm.3 + - gcr.io/gke-release/asm/proxyv2:1.13.7-asm.3 + - gcr.io/gke-release/asm/mdp:1.13.7-asm.3 + - gcr.io/gke-release/asm/install-cni:1.13.7-asm.3 + - gcr.io/gke-release/asm/canonical-service-controller:1.10.3-asm.16 + - gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0 + "1.14.1-asm.3": + - gcr.io/gke-release/asm/pilot:1.14.1-asm.3 + - gcr.io/gke-release/asm/proxyv2:1.14.1-asm.3 + - gcr.io/gke-release/asm/mdp:1.14.1-asm.3 + - gcr.io/gke-release/asm/install-cni:1.14.1-asm.3 + - gcr.io/gke-release/asm/canonical-service-controller:1.10.3-asm.16 + - gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0 \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/asm/files/asm-1.13.7.tar.gz b/tools/anthosvmware-ansible-module/roles/asm/files/asm-1.13.7.tar.gz new file mode 100644 index 0000000000..8e762f7418 Binary files /dev/null and b/tools/anthosvmware-ansible-module/roles/asm/files/asm-1.13.7.tar.gz differ diff --git a/tools/anthosvmware-ansible-module/roles/asm/files/asm-1.14.tar.gz b/tools/anthosvmware-ansible-module/roles/asm/files/asm-1.14.tar.gz new file mode 100644 index 0000000000..476a827700 Binary files /dev/null and b/tools/anthosvmware-ansible-module/roles/asm/files/asm-1.14.tar.gz differ diff --git a/tools/anthosvmware-ansible-module/roles/asm/tasks/asserts.yml b/tools/anthosvmware-ansible-module/roles/asm/tasks/asserts.yml new file mode 100644 index 0000000000..5344e6efe8 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/tasks/asserts.yml @@ -0,0 +1,72 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[ASM] Sanity Checks" + ansible.builtin.assert: + that: + - yamldestpath | length >= 6 + - sakeyfolder_base is defined + - asm_version | length >= 3 + - asm_revision | length >= 3 + - asm_asmcli | length >= 3 + - asm_asmcli_url | length >= 20 + - asm_asmcli_version | length >= 10 + - asm_kubeconfig | length >= 10 + - asm_gcp_project | length >= 10 + - asm_gcp_project_number | length >= 6 + - asm_out_dir | length >= 3 + - asm_temp | length >= 3 + - asm_ingress_namespace | length >= 4 + - asm_network_id | length >= 4 + - asm_cluster_name | length >= 2 + fail_msg: "Undefined variables." + success_msg: "All required variables are defined." + when: not mesh_install|default(false)|bool + +- name: "[mesh] Sanity Checks" + ansible.builtin.assert: + that: + - yamldestpath | length >= 6 + - sakeyfolder_base is defined + - asm_version | length >= 3 + - asm_revision | length >= 3 + - asm_asmcli | length >= 3 + - asm_asmcli_url | length >= 20 + - asm_asmcli_version | length >= 10 + - asm_out_dir | length >= 3 + - uc_name | length >= 2 + - asm_temp | length >= 3 + - mesh_project_id | length >= 10 + - mesh_unique_id | length >= 2 + - asm_network_id | length >= 4 + fail_msg: "Undefined variables." + success_msg: "All required variables are defined." + when: mesh_install|default(false)|bool + +- name: "[asm] Ensure container images are available for '{{ asm_version }}' when private registry" + ansible.builtin.assert: + that: + - asm_container_images is defined + - asm_minor_version is defined + - asm_minor_version in asm_container_images + - asm_container_images[asm_minor_version] | type_debug == 'list' + - asm_container_images[asm_minor_version] | length > 0 + fail_msg: "Please add ASM version '{{ asm_version }}' container images to asm_container_images map" + success_msg: "ASM version '{{ asm_minor_version }}' found in asm_container_images map" + when: + - asm_install|default(false)|bool + - not mesh_install|default(false)|bool + - glb_privatereg_url is defined and glb_privatereg_url | length > 3 + - glb_privatereg_username is defined and glb_privatereg_username | length > 5 + - glb_privatereg_password is defined and glb_privatereg_password | length > 5 \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/asm/tasks/cert.yml b/tools/anthosvmware-ansible-module/roles/asm/tasks/cert.yml new file mode 100644 index 0000000000..1dbf59269e --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/tasks/cert.yml @@ -0,0 +1,31 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[asm] Copy cert file from Tower Vault" + ansible.builtin.copy: + src: "{{ cert_file }}" + dest: "{{ cert_dest_path }}" + mode: 0600 + when: + - cert_file is defined and (cert_file | length > 1) + - cert_dest_path is defined and (cert_dest_path | length > 1) + +- name: "[asm] Create cert from Inventory" + ansible.builtin.template: + src: cert.j2 + dest: "{{ cert_dest_path }}" + mode: 0600 + when: + - cert_string is defined and (cert_string | length > 1) + - cert_dest_path is defined and (cert_dest_path | length > 1) \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/asm/tasks/gateway-config.yml b/tools/anthosvmware-ansible-module/roles/asm/tasks/gateway-config.yml new file mode 100644 index 0000000000..6858a2da5b --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/tasks/gateway-config.yml @@ -0,0 +1,114 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[asm] Check for tls cert and key" + ansible.builtin.set_fact: + tls_cert_provided: true + when: + - asm_tlskey_file is defined and asm_tlskey_file|length > 3 + or asm_tlskey_content is defined and asm_tlskey_content|length > 3 + - asm_tlscert_file is defined and asm_tlscert_file|length > 3 + or asm_tlscert_content is defined and asm_tlscert_content|length > 3 + - asm_tls_key_path is defined and asm_tls_key_path|length > 3 + - asm_tls_cert_path is defined and asm_tls_cert_path|length > 3 + +# Uses lookup plugin to get the ASM TLS CERT filename on Ansible controller +# With OSS Ansible on the command-line, you could use --extra-vars +- name: "[asm] Copy TLS server-side certificate" + include_tasks: + file: cert.yml + vars: + cert_file: "{{ asm_tlscert_file | default('') }}" + cert_string: "{{ asm_tlscert_content | default('') }}" + cert_dest_path: "{{ asm_tls_cert_path }}" + when: + - tls_cert_provided | default(false) | bool + +# Uses lookup plugin to get the ASM TLS key filename on Ansible controller +# With OSS Ansible on the command-line, you could use --extra-vars +- name: "[asm] Copy TLS privatekey" + include_tasks: + file: cert.yml + vars: + cert_file: "{{ asm_tlskey_file | default('') }}" + cert_string: "{{ asm_tlskey_content | default('') }}" + cert_dest_path: "{{ asm_tls_key_path }}" + when: + - tls_cert_provided | default(false) | bool + +- name: "[asm] Generate secret for https/tls cert in asm ingress namespace" + ansible.builtin.command: + argv: + - kubectl + - create + - secret + - generic + - "{{ asm_tls_cred_name }}" + - --save-config + - --dry-run=client + - --from-file=key={{ asm_tls_key_path }} + - --from-file=cert={{ asm_tls_cert_path }} + - -n + - "{{ asm_ingress_namespace }}" + - -o + - yaml + register: tls_secret + when: + - tls_cert_provided|default(false)|bool + +- name: "[asm] Create file for TLS secret" + ansible.builtin.copy: + content: "{{ tls_secret.stdout }}" + dest: "{{ asm_temp }}/tls-secret.yaml" + mode: 0640 + when: + - tls_cert_provided|default(false)|bool + - (tls_secret.stdout is defined) and (tls_secret.stdout|length > 0) + +- name: "[asm] Create TLS Secret in asm ingress namespace" + ansible.builtin.command: + argv: + - kubectl + - apply + - -f + - "{{ asm_temp }}/tls-secret.yaml" + when: + - tls_cert_provided|default(false)|bool + - (tls_secret.stdout is defined) and (tls_secret.stdout|length > 0) + +- name: "[asm] Template Gateway object in asm ingress namespace" + ansible.builtin.template: + src: "gateway.yaml.j2" + dest: "{{ asm_temp }}/gateway-asm.yaml" + owner: "{{ ansible_user }}" + group: "{{ ansible_user }}" + mode: '0600' + when: + - tls_cert_provided|default(false)|bool + - asm_ingress_hosts is defined + - asm_ingress_hosts | type_debug == 'list' + - asm_ingress_hosts | length > 0 + +- name: "[asm] Create Gateway object in asm ingress namespace" + ansible.builtin.command: + argv: + - kubectl + - apply + - -f + - "{{ asm_temp }}/gateway-asm.yaml" + when: + - tls_cert_provided|default(false)|bool + - asm_ingress_hosts is defined + - asm_ingress_hosts | type_debug == 'list' + - asm_ingress_hosts | length > 0 diff --git a/tools/anthosvmware-ansible-module/roles/asm/tasks/install.yml b/tools/anthosvmware-ansible-module/roles/asm/tasks/install.yml new file mode 100644 index 0000000000..eac909bbe3 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/tasks/install.yml @@ -0,0 +1,226 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[asm] Block - Create NS, hydrate tempaltes, and configure asm" + block: + - name: "[asm] Setup cluster config" + include_tasks: setup-config.yml + + - name: "[asm] Switch to Connect Register Service Account Name" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gcloud + - auth + - activate-service-account + - --key-file + - "{{ job_sakeyfolder }}/{{ connect_register_gcpsa_path }}" + + - name: "[asm] Add mesh_id label to cluster" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gcloud + - container + - hub + - memberships + - update + - "{{ asm_cluster_name }}" + - --project + - "{{ asm_gcp_project }}" + - "--update-labels" + - "mesh_id={{ asm_mesh_id }}" + + - name: "[asm] Switch to asm Service Account Name" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gcloud + - auth + - activate-service-account + - --key-file + - "{{ job_sakeyfolder }}/{{ asm_gcpsa_path }}" + + - name: "[asm] Template RBAC rule required for asm" + ansible.builtin.template: + src: cluster-role-binding.yaml.j2 + dest: "{{ asm_temp }}/cluster-role-binding-asm.yaml" + owner: "{{ ansible_user }}" + group: "{{ ansible_user }}" + mode: '0600' + + - name: "[asm] Delete old RBAC rule required for asm (if exists)" + ansible.builtin.command: + argv: + - kubectl + - delete + - -f + - "{{ asm_temp }}/cluster-role-binding-asm.yaml" + - --ignore-not-found + + - name: "[asm] Create RBAC rule required for asm" + ansible.builtin.command: + argv: + - kubectl + - create + - -f + - "{{ asm_temp }}/cluster-role-binding-asm.yaml" + + - name: "[asm] Template Istio system namespace" + ansible.builtin.template: + src: namespace.yaml.j2 + dest: "{{ asm_temp }}/namespace-asm.yaml" + owner: "{{ ansible_user }}" + group: "{{ ansible_user }}" + mode: '0600' + when: not asm_upgrade|default(false)|bool + + - name: "[asm] Create istio-system namespace" + ansible.builtin.command: + argv: + - kubectl + - apply + - -f + - "{{ asm_temp }}/namespace-asm.yaml" + when: not asm_upgrade|default(false)|bool + + - name: "[asm] Template the ASM control plane config file" + ansible.builtin.template: + src: control-plane-config.yaml.j2 + dest: "{{ asm_temp }}/control-plane-config-asm.yaml" + owner: "{{ ansible_user }}" + group: "{{ ansible_user }}" + mode: '0600' + + # Install ASM + - name: "[asm] Install Anthos Service Mesh" + ansible.builtin.command: # noqa 305 204 + argv: "{{ _req_args | reject('equalto', omit) | list }}" + vars: + _req_args: + - "{{ asm_temp }}/asmcli" + - install + - --output_dir + - "{{ asm_out_dir }}" + - --kubeconfig + - "{{ asm_kubeconfig }}" + - --fleet-id + - "{{ asm_gcp_project }}" + - --platform + - multicloud + - --network_id + - "{{ asm_network_id }}" + - --ca + - citadel + - "{{ omit if not asm_install_cert else '--ca_cert' }}" + - "{{ omit if not asm_install_cert else asm_ca_cert_path }}" + - "{{ omit if not asm_install_cert else '--ca_key' }}" + - "{{ omit if not asm_install_cert else asm_ca_key_path }}" + - "{{ omit if not asm_install_cert else '--root_cert' }}" + - "{{ omit if not asm_install_cert else asm_root_cert_path }}" + - "{{ omit if not asm_install_cert else '--cert_chain' }}" + - "{{ omit if not asm_install_cert else asm_cert_chain_path }}" + - --enable_meshconfig_init + - --enable_cluster_labels + - --option + - stackdriver + - --custom-overlay + - "{{ asm_temp }}/control-plane-config-asm.yaml" + - --revision_name + - "{{ asm_revision }}" + - "{{ omit if not ( (asm_registry_url is defined) and (asm_registry_url| length > 0) ) else '--disable_canonical_service' }}" + - --verbose + - "{{ omit if not asm_offline_mode else '--offline' }}" + + # Find and replace kubebuilder hardcoded images in controller.yaml + - name: "[asm] Replace hardcoded kube-rbac-proxy image" + ansible.builtin.replace: + path: "{{ asm_out_dir }}/asm/canonical-service/controller.yaml" + regexp: '(image:)(.*?)\/(kubebuilder)(.*?):(.*)' + replace: '\1 {{ asm_registry_url }}\4:{{ asm_kubebuilder_tag }}' + when: (asm_registry_url is defined) and (asm_registry_url| length > 0) + + # Find and replace CSC hardcoded images in controller.yaml + - name: "[asm] Replace hardcoded canonical services image" + ansible.builtin.replace: + path: "{{ asm_out_dir }}/asm/canonical-service/controller.yaml" + regexp: '(image:)(.*?)\/(canonical-service-controller.*?):(.*)' + replace: '\1 {{ asm_registry_url }}/\3:{{ asm_csc_tag }}' + when: (asm_registry_url is defined) and (asm_registry_url| length > 0) + + # Deploy the Canonical Services Controller + - name: "[asm] Deploy Canonical Services Controller with private registry images" + ansible.builtin.command: + argv: + - kubectl + - apply + - -f + - "{{ asm_out_dir }}/asm/canonical-service/controller.yaml" + when: (asm_registry_url is defined) and (asm_registry_url| length > 0) + + - name: "[asm] Template asm-ingress namespace" + ansible.builtin.template: + src: namespace-ingress.yaml.j2 + dest: "{{ asm_temp }}/namespace-ingress-asm.yaml" + owner: "{{ ansible_user }}" + group: "{{ ansible_user }}" + mode: '0600' + + - name: "[asm] Create asm-ingress namespace" + ansible.builtin.command: + argv: + - kubectl + - apply + - -f + - "{{ asm_temp }}/namespace-ingress-asm.yaml" + + - name: "[asm] Template the Ingress Gateway manifest" + ansible.builtin.template: + src: "igw-deployment.yaml.j2" + dest: "{{ asm_temp }}/igw-asm.yaml" + owner: "{{ ansible_user }}" + group: "{{ ansible_user }}" + mode: '0600' + when: not asm_upgrade|default(false)|bool + + - name: "[asm] Create the Ingress Gateway" + ansible.builtin.command: + argv: + - kubectl + - apply + - -f + - "{{ asm_temp }}/igw-asm.yaml" + when: not asm_upgrade|default(false)|bool + + - name: "[asm] Setup TLS secret and Gateway configuration" + include_tasks: gateway-config.yml + + - name: "[asm] Delete RBAC rule created above" + ansible.builtin.command: + argv: + - kubectl + - delete + - -f + - "{{ asm_temp }}/cluster-role-binding-asm.yaml" + - --ignore-not-found + + environment: + KUBECONFIG: "{{ asm_kubeconfig }}" + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" + http_proxy: "{{ glb_proxyurl }}" + https_proxy: "{{ glb_proxyurl }}" + no_proxy: "{{ glb_noproxy }}" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup diff --git a/tools/anthosvmware-ansible-module/roles/asm/tasks/main.yml b/tools/anthosvmware-ansible-module/roles/asm/tasks/main.yml new file mode 100644 index 0000000000..ed8df54dcf --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/tasks/main.yml @@ -0,0 +1,104 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[ASM] Include tasks - asserts.yml" + include_tasks: asserts.yml + +- name: "[ASM] Detect if ASM already installed and get current revision" + ansible.builtin.command: + chdir: "{{ yamldestpath }}" + argv: + - kubectl + - --kubeconfig + - "{{ asm_kubeconfig }}" + - get + - deploy + - -n + - istio-system + - -l + - app=istiod + - -o + - jsonpath={.items[*].metadata.name} + register: r_asm_revision + when: not mesh_install|default(false)|bool + +- name: "[ASM] Update ASM config if current revision is same as requested revision" + ansible.builtin.set_fact: + asm_update: true + when: + - (r_asm_revision.stdout is defined) and (r_asm_revision.stdout|length > 0) + - r_asm_revision.stdout is search(asm_revision) + - asm_install|default(false)|bool + - not mesh_install|default(false)|bool + - not asm_upgrade_completion|default(false)|bool + +- name: "[ASM] if installed revision is different then set upgrade flag to true" + ansible.builtin.set_fact: + asm_upgrade: true + when: + - (r_asm_revision.stdout is defined) and (r_asm_revision.stdout|length > 0) + - r_asm_revision.stdout is not search(asm_revision) + - asm_install|default(false)|bool + - not mesh_install|default(false)|bool + - not asm_upgrade_completion|default(false)|bool + - not asm_update|default(false)|bool + +- name: "[ASM] Include tasks - install.yml" + include_tasks: install.yml + when: + - asm_install|default(false)|bool + - not mesh_install|default(false)|bool + - not asm_upgrade|default(false)|bool + - not asm_upgrade_completion|default(false)|bool + - not asm_update|default(false)|bool + +- name: "[ASM] Include tasks - upgrade.yml" + include_tasks: upgrade.yml + when: + - asm_upgrade|default(false)|bool + - not asm_update|default(false)|bool + +- name: "[ASM] Include tasks - update.yml" + include_tasks: update.yml + when: + - asm_update|default(false)|bool + - not asm_upgrade|default(false)|bool + +- name: "[ASM] Check if request is for uninstall" + ansible.builtin.set_fact: + asm_uninstall: true + when: + - not asm_install|default(false)|bool + - not mesh_install|default(false)|bool + - not asm_upgrade|default(false)|bool + - not asm_upgrade_completion|default(false)|bool + - (r_asm_revision.stdout is defined) and (r_asm_revision.stdout|length > 0) + +- name: "[ASM] Include tasks - uninstall.yml" + include_tasks: uninstall.yml + when: + - asm_uninstall|default(false)|bool + +- name: "[ASM] Include tasks - upgrade-completion.yml" + include_tasks: upgrade-completion.yml + when: + - asm_upgrade_completion|default(false)|bool + - (asm_old_revision is defined) and (asm_old_revision|length > 0) + - (r_asm_revision.stdout is defined) and (r_asm_revision.stdout|length > 0) + - r_asm_revision.stdout is search(asm_old_revision) + +- name: "[mesh] Include tasks - multi-cluster-mesh.yml" + include_tasks: multi-cluster-mesh.yml + when: mesh_install|default(false)|bool diff --git a/tools/anthosvmware-ansible-module/roles/asm/tasks/multi-cluster-mesh.yml b/tools/anthosvmware-ansible-module/roles/asm/tasks/multi-cluster-mesh.yml new file mode 100644 index 0000000000..2a23a819f4 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/tasks/multi-cluster-mesh.yml @@ -0,0 +1,148 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[mesh] Setup cluster config" + include_tasks: setup-config.yml + +- name: "[mesh] Block - East-West Gateway, Working Dir, Mesh Clusters" + block: + + - name: "[mesh] Switch to asm Service Account Name" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gcloud + - auth + - activate-service-account + - --key-file + - "{{ job_sakeyfolder }}/{{ asm_gcpsa_path }}" + run_once: true + delegate_to: "{{ play_hosts[0] }}" + + - name: "[mesh] Get Project Number" + ansible.builtin.command: + argv: + - gcloud + - projects + - describe + - "{{ mesh_project_id }}" + - --format=value(projectNumber) + changed_when: false + register: mesh_proj_number + run_once: true + delegate_to: "{{ play_hosts[0] }}" + + - name: "[mesh] Set Facts" + ansible.builtin.set_fact: + mesh_id: "proj-{{ mesh_proj_number.stdout | trim }}" + cluster_name: "cn-{{ mesh_project_id }}-global-{{ uc_name }}" + + - name: "[mesh] Generate east-west gateway configuration" + ansible.builtin.command: + argv: + - "{{ asm_out_dir }}/asm/istio/expansion/gen-eastwest-gateway.sh" + - --mesh + - "{{ mesh_id }}" + - --cluster + - "{{ cluster_name }}" + - --network + - "{{ asm_network_id }}" + - --revision + - "{{ asm_revision }}" + register: mesh_gateway + + - name: "[mesh] Create file for east-west gateway" + ansible.builtin.copy: + content: "{{ mesh_gateway.stdout }}" + dest: "{{ asm_temp }}/{{ uc_name }}-eastwest-gateway.yml" + mode: 0640 + + - name: "[mesh] Install east-west gateway to cluster" + ansible.builtin.command: + argv: + - "{{ asm_out_dir }}/istioctl" + - --kubeconfig + - "{{ yamldestpath }}/{{ uc_name }}-kubeconfig" + - install + - -y + - -f + - "{{ asm_temp }}/{{ uc_name }}-eastwest-gateway.yml" + + - name: "[mesh] Expose services via the east-west gateway" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ yamldestpath }}/{{ uc_name }}-kubeconfig" + - apply + - -n + - istio-system + - -f + - "{{ asm_out_dir }}/asm/istio/expansion/expose-services.yaml" + + - name: "[mesh] Create working directory" + ansible.builtin.file: + path: "/tmp/{{ mesh_unique_id }}" + mode: 0700 + state: directory + run_once: true + delegate_to: "{{ play_hosts[0] }}" + + - name: "[mesh] Get cluster kubeconfig files" + ansible.builtin.slurp: + src: "{{ yamldestpath }}/{{ uc_name }}-kubeconfig" + register: kubeconfig + changed_when: false + + - name: "[mesh] Drop kubeconfigs into working directory" + ansible.builtin.copy: + content: "{{ kubeconfig.content | b64decode }}" + dest: "/tmp/{{ mesh_unique_id }}/{{ uc_name }}-kubeconfig" + mode: 0600 + delegate_to: "{{ play_hosts[0] }}" + + - name: "[mesh] Get list of all kubeconfigs" + ansible.builtin.find: + path: "/tmp/{{ mesh_unique_id }}/" + pattern: "*-kubeconfig" + register: cluster_kubeconfigs + run_once: true + delegate_to: "{{ play_hosts[0] }}" + + - name: "[mesh] Create mesh between clusters" + ansible.builtin.command: + argv: + - "{{ asm_temp }}/asmcli" + - create-mesh + - "{{ mesh_project_id }}" + - --output_dir + - "{{ asm_out_dir }}" + - "{{ item[0].path }}" + - "{{ item[1].path }}" + loop: "{{ cluster_kubeconfigs.files | combinations(2) | list }}" + loop_control: + label: "{{ item[0].path }} with {{ item[1].path }}" + run_once: true + delegate_to: "{{ play_hosts[0] }}" + + always: + + - name: "[mesh] Remove kubeconfig working directory" + ansible.builtin.file: + path: "/tmp/{{ mesh_unique_id }}/" + state: absent + + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup diff --git a/tools/anthosvmware-ansible-module/roles/asm/tasks/private-upload.yml b/tools/anthosvmware-ansible-module/roles/asm/tasks/private-upload.yml new file mode 100644 index 0000000000..ded03477ec --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/tasks/private-upload.yml @@ -0,0 +1,72 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[asm] Login to private container registry" + ansible.builtin.command: + argv: + - docker + - login + - -u + - '{{ glb_privatereg_username }}' + - -p + - '{{ glb_privatereg_password }}' + - '{{ glb_privatereg_url }}' + no_log: true + +- name: "[asm] Check if images are already pushed to private container registry" + ansible.builtin.command: + argv: + - docker + - manifest + - inspect + - "{{ glb_privatereg_url }}/{{ item.split('/')[-1] }}" + loop: "{{ asm_container_images[asm_minor_version] }}" + loop_control: + label: "{{ glb_privatereg_url }}/{{ item.item.split('/')[-1] }}" + register: r_image_check + failed_when: false + +- name: "[asm] Download new images from gcr" + ansible.builtin.command: + argv: + - docker + - pull + - "{{ item.item }}" + loop: "{{ r_image_check.results }}" + loop_control: + label: "{{ item.item }}" + when: item.rc != 0 + +- name: "[asm] Change new image tags to private registry" + ansible.builtin.command: + argv: + - docker + - tag + - "{{ item.item }}" + - "{{ glb_privatereg_url }}/{{ item.item.split('/')[-1] }}" + loop: "{{ r_image_check.results }}" + loop_control: + label: "{{ glb_privatereg_url }}/{{ item.item.split('/')[-1] }}" + when: item.rc != 0 + +- name: "[asm] Push new images to private registry" + ansible.builtin.command: + argv: + - docker + - push + - "{{ glb_privatereg_url }}/{{ item.item.split('/')[-1] }}" + loop: "{{ r_image_check.results }}" + loop_control: + label: "{{ glb_privatereg_url }}/{{ item.item.split('/')[-1] }}" + when: item.rc != 0 \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/asm/tasks/setup-config.yml b/tools/anthosvmware-ansible-module/roles/asm/tasks/setup-config.yml new file mode 100644 index 0000000000..8794712ad1 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/tasks/setup-config.yml @@ -0,0 +1,159 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "Include role for GCP SA JSON key files" + ansible.builtin.include_role: + name: copy_credentials + vars: + is_asm: true + when: not asm_uninstall|default(false)|bool + +- name: "[asm] Create folder for ASM files on admin workstation" + ansible.builtin.file: + path: "{{ asm_temp }}" + mode: 0700 + state: directory + when: + - (asm_temp is defined) and (asm_temp | length > 0) + +- name: "[asm] Check if request is for upgrade, uninstall or mesh setup" + ansible.builtin.set_fact: + asm_install_cert: false + when: + - (asm_upgrade|default(false)|bool) or (mesh_install|default(false)|bool) or (asm_uninstall|default(false)|bool) + +# Uses lookup plugin to get the ASM ROOTCERT filename on Ansible controller +# With OSS Ansible on the command-line, you could use --extra-vars +- name: "[asm] Copy ROOT certificate" + include_tasks: + file: cert.yml + vars: + cert_file: "{{ asm_rootcert_file | default('') }}" + cert_string: "{{ asm_rootcert_content | default('') }}" + cert_dest_path: "{{ asm_root_cert_path | default('') }}" + when: + - asm_install_cert | default(false) | bool + +# Uses lookup plugin to get the ASM CACERT filename on Ansible controller +# With OSS Ansible on the command-line, you could use --extra-vars +- name: "[asm] Copy CA certificate" + include_tasks: + file: cert.yml + vars: + cert_file: "{{ asm_cacert_file | default('') }}" + cert_string: "{{ asm_cacert_content | default('') }}" + cert_dest_path: "{{ asm_ca_cert_path | default('') }}" + when: + - asm_install_cert | default(false) | bool + +# Uses lookup plugin to get the ASM CA key filename on Ansible controller +# With OSS Ansible on the command-line, you could use --extra-vars +- name: "[asm] Copy CA key" + include_tasks: + file: cert.yml + vars: + cert_file: "{{ asm_cakey_file | default('') }}" + cert_string: "{{ asm_cakey_content | default('') }}" + cert_dest_path: "{{ asm_ca_key_path | default('') }}" + when: + - asm_install_cert | default(false) | bool + +# Uses lookup plugin to get the ASM CERTCHAIN filename on Ansible controller +# With OSS Ansible on the command-line, you could use --extra-vars +- name: "[asm] Copy certificate chain file" + include_tasks: + file: cert.yml + vars: + cert_file: "{{ asm_certchain_file | default('') }}" + cert_string: "{{ asm_certchain_content | default('') }}" + cert_dest_path: "{{ asm_cert_chain_path | default('') }}" + when: + - asm_install_cert | default(false) | bool + +- name: "[asm] Check for private artifactory binary repo" + ansible.builtin.set_fact: + is_private_artifactory: true + when: + - private_download_base is defined + - private_download_base|length > 8 + - private_download_auth_header is defined + - private_download_auth_header|length > 10 + +- name: "[asm] Download asmcli from private location" + ansible.builtin.get_url: + url: "{{ private_download_base }}/asm/{{ asm_asmcli_version }}/asmcli_{{ asm_asmcli_version }}" + headers: + Authorization: "{{ private_download_auth_header }}" + dest: "{{ asm_temp }}/asmcli" + mode: 0700 + when: + - is_private_artifactory|default(false)|bool + +- name: "[asm] Download asmcli" + ansible.builtin.get_url: + url: "{{ asm_asmcli_url }}/asmcli_{{ asm_asmcli_version }}" + dest: "{{ asm_temp }}/asmcli" + checksum: sha256:{{ asm_asmcli_url }}/asmcli_{{ asm_asmcli_version }}.sha256 + mode: 0700 + when: + - not (is_private_artifactory|default(false)|bool) + +- name: "[asm] Make asmcli executable" + ansible.builtin.file: + path: "{{ asm_temp }}/asmcli" + owner: "{{ ansible_user }}" + group: "{{ ansible_user }}" + mode: '0700' + +- name: "[asm] Extract ASM offline package bundle" + ansible.builtin.unarchive: + src: "{{ asm_offline_bundle_file }}" + dest: "{{ asm_temp }}" + when: + - asm_offline_mode|default(false)|bool + - asm_offline_bundle_file is defined + - asm_offline_bundle_file | length > 0 + +- name: "[asm] Create ASM offline package bundle" + ansible.builtin.command: + argv: + - "{{ asm_temp }}/asmcli" + - build-offline-package + - --output_dir + - "{{ asm_out_dir }}" + when: + - not asm_offline_mode|default(false)|bool + +- name: "[asm] Copy istioctl to istioctl_dest_path" + ansible.builtin.shell: + cmd: 'sudo cp {{ asm_out_dir }}/istioctl {{ istioctl_dest_path }}; sudo chmod 755 {{ istioctl_dest_path }}/istioctl' + when: + - asm_copy_istioctl|default(false)|bool + - not mesh_install|default(false)|bool + - not asm_uninstall|default(false)|bool + +- name: "[asm] Check for private registry" + ansible.builtin.set_fact: + is_private_registry: true + when: + - not mesh_install|default(false)|bool + - not asm_uninstall|default(false)|bool + - glb_privatereg_url is defined and glb_privatereg_url | length > 3 + - glb_privatereg_username is defined and glb_privatereg_username | length > 5 + - glb_privatereg_password is defined and glb_privatereg_password | length > 5 + +- name: "[asm] Upload ASM Images to Private Registry" + include_tasks: private-upload.yml + when: is_private_registry|default(false)|bool \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/asm/tasks/uninstall.yml b/tools/anthosvmware-ansible-module/roles/asm/tasks/uninstall.yml new file mode 100644 index 0000000000..f0f643e908 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/tasks/uninstall.yml @@ -0,0 +1,59 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[mesh] Setup cluster config" + include_tasks: setup-config.yml + +- name: "[asm] Block - Uninstall Anthos Service Mesh" + block: + - name: "[asm] Delete webhooks" + ansible.builtin.command: + argv: + - kubectl + - delete + - validatingwebhookconfiguration,mutatingwebhookconfiguration + - -l + - operator.istio.io/component=Pilot + ignore_errors: true + + - name: "[asm] Remove the in-cluster control plane" + ansible.builtin.command: + chdir: "{{ asm_out_dir }}" + argv: + - ./istioctl + - x + - uninstall + - --purge + - --skip-confirmation + ignore_errors: true + + - name: "[asm] Delete istio-system, asm-ingress and asm-system namespaces" + ansible.builtin.command: + argv: + - kubectl + - delete + - namespace + - istio-system + - asm-system + - "{{ asm_ingress_namespace }}" + - --ignore-not-found + ignore_errors: true + + environment: + KUBECONFIG: "{{ asm_kubeconfig }}" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup diff --git a/tools/anthosvmware-ansible-module/roles/asm/tasks/update.yml b/tools/anthosvmware-ansible-module/roles/asm/tasks/update.yml new file mode 100644 index 0000000000..2085f38bf2 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/tasks/update.yml @@ -0,0 +1,34 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[asm] Block - Update TLS secret and Gateway configuration" + block: + - name: "[asm] Create folder for ASM files on admin workstation" + ansible.builtin.file: + path: "{{ asm_temp }}" + mode: 0700 + state: directory + when: + - (asm_temp is defined) and (asm_temp | length > 0) + + - name: "[asm] Setup TLS secret and Gateway configuration" + include_tasks: gateway-config.yml + + environment: + KUBECONFIG: "{{ asm_kubeconfig }}" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/asm/tasks/upgrade-completion.yml b/tools/anthosvmware-ansible-module/roles/asm/tasks/upgrade-completion.yml new file mode 100644 index 0000000000..40d4d1142c --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/tasks/upgrade-completion.yml @@ -0,0 +1,94 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[asm] Create folder for ASM files on admin workstation" + ansible.builtin.file: + path: "{{ asm_temp }}" + mode: 0700 + state: directory + when: + - (asm_temp is defined) and (asm_temp | length > 0) + +- name: "[asm] Block - Complete the transition of Anthos Service Mesh Upgrade" + block: + - name: "[asm] Template validating webhook" + ansible.builtin.template: + src: validating-webhook.yaml.j2 + dest: "{{ asm_temp }}/validating-webhook-asm.yaml" + owner: "{{ ansible_user }}" + group: "{{ ansible_user }}" + mode: '0600' + + - name: "[asm] Delete the old version of validatingwebhook configuration" + ansible.builtin.command: + argv: + - kubectl + - delete + - validatingwebhookconfiguration + - istio-validator-{{ asm_old_revision }}-istio-system + - -n + - istio-system + - --ignore-not-found + + - name: "[asm] Update validating webhook to new revision" + ansible.builtin.command: + argv: + - kubectl + - apply + - -f + - "{{ asm_temp }}/validating-webhook-asm.yaml" + + - name: "[asm] Move the default tag to new revision" + ansible.builtin.command: + argv: + - "{{ istioctl_dest_path }}/istioctl" + - tag + - set + - default + - --revision + - "{{ asm_revision }}" + - --overwrite + ignore_errors: true + + - name: "[asm] Delete the old version of istiod" + ansible.builtin.command: + argv: + - kubectl + - delete + - "Service,Deployment,HorizontalPodAutoscaler,PodDisruptionBudget" + - istiod-{{ asm_old_revision }} + - -n + - istio-system + - --ignore-not-found + + - name: "[asm] Delete the old version of IstioOperator configuration" + ansible.builtin.command: + argv: + - kubectl + - delete + - IstioOperator + - installed-state-{{ asm_old_revision }} + - -n + - istio-system + - --ignore-not-found + + environment: + KUBECONFIG: "{{ asm_kubeconfig }}" + + always: + - name: "[cleanup] Delete temp directory created for ASM files" + ansible.builtin.file: + path: "{{ asm_temp }}" + state: absent + when: (asm_temp is defined) and (asm_temp | length > 0) diff --git a/tools/anthosvmware-ansible-module/roles/asm/tasks/upgrade.yml b/tools/anthosvmware-ansible-module/roles/asm/tasks/upgrade.yml new file mode 100644 index 0000000000..0d7f37c9df --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/tasks/upgrade.yml @@ -0,0 +1,47 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[asm] Install new version" + include_tasks: install.yml + +- name: "[asm] Block - Restart ingress gateway to use new revision" + block: + - name: "[asm] Restart the Pods to trigger re-injection" + ansible.builtin.command: + argv: + - kubectl + - rollout + - restart + - deployment + - -n + - "{{ asm_ingress_namespace }}" + + - name: "[asm] Wait until the ingress gateway pods are ready" + async: 610 + poll: 15 + ansible.builtin.command: + argv: + - kubectl + - wait + - --for=condition=Ready + - --timeout=600s + - pod + - -l + - "app=istio-ingressgateway,istio.io/rev={{ asm_revision }}" + - -n + - "{{ asm_ingress_namespace }}" + + environment: + KUBECONFIG: "{{ asm_kubeconfig }}" diff --git a/tools/anthosvmware-ansible-module/roles/asm/templates/cert.j2 b/tools/anthosvmware-ansible-module/roles/asm/templates/cert.j2 new file mode 100644 index 0000000000..59f1474934 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/templates/cert.j2 @@ -0,0 +1 @@ +{{ cert_string }} \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/asm/templates/cluster-role-binding.yaml.j2 b/tools/anthosvmware-ansible-module/roles/asm/templates/cluster-role-binding.yaml.j2 new file mode 100644 index 0000000000..85163f38da --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/templates/cluster-role-binding.yaml.j2 @@ -0,0 +1,27 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: cluster-admin-asm +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: cluster-admin +subjects: +- apiGroup: rbac.authorization.k8s.io + kind: User + name: "{{ asm_user_email }}" diff --git a/tools/anthosvmware-ansible-module/roles/asm/templates/control-plane-config.yaml.j2 b/tools/anthosvmware-ansible-module/roles/asm/templates/control-plane-config.yaml.j2 new file mode 100644 index 0000000000..535226d6bf --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/templates/control-plane-config.yaml.j2 @@ -0,0 +1,35 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +apiVersion: install.istio.io/v1alpha1 +kind: IstioOperator +spec: + values: + global: + multiCluster: + clusterName: "{{ asm_cluster_name }}" + network: "{{ asm_network_id }}" + components: + egressGateways: + - name: istio-egressgateway + enabled: false + ingressGateways: + - name: istio-ingressgateway + enabled: false +{% if (asm_registry_url is defined) and (asm_registry_url| length > 0) %} + hub: "{{ asm_registry_url }}" +{% endif %} + meshConfig: + accessLogFile: "/dev/stdout" diff --git a/tools/anthosvmware-ansible-module/roles/asm/templates/gateway.yaml.j2 b/tools/anthosvmware-ansible-module/roles/asm/templates/gateway.yaml.j2 new file mode 100644 index 0000000000..a774fd6753 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/templates/gateway.yaml.j2 @@ -0,0 +1,55 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +apiVersion: networking.istio.io/v1beta1 +kind: Gateway +metadata: + name: istio-gateway + namespace: "{{ asm_ingress_namespace }}" +spec: + selector: + istio: ingressgateway + servers: + - port: + name: http + number: 80 + protocol: HTTP2 + hosts: +{% for host in asm_ingress_hosts %} + - "{{ host }}" +{% endfor %} + tls: + httpsRedirect: true + - port: + name: https + number: 443 + protocol: HTTPS + hosts: +{% for host in asm_ingress_hosts %} + - "{{ host }}" +{% endfor %} + tls: + credentialName: "{{ asm_tls_cred_name }}" + mode: SIMPLE + - port: + name: tls + number: 15443 + protocol: TLS + hosts: +{% for host in asm_ingress_hosts %} + - "{{ host }}" +{% endfor %} + tls: + mode: PASSTHROUGH diff --git a/tools/anthosvmware-ansible-module/roles/asm/templates/igw-deployment.yaml.j2 b/tools/anthosvmware-ansible-module/roles/asm/templates/igw-deployment.yaml.j2 new file mode 100644 index 0000000000..6055aae8c1 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/templates/igw-deployment.yaml.j2 @@ -0,0 +1,136 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +apiVersion: apps/v1 +kind: Deployment +metadata: + name: asm-ingressgateway + namespace: {{ asm_ingress_namespace }} +spec: + replicas: 3 + selector: + matchLabels: + app: istio-ingressgateway + istio: ingressgateway + template: + metadata: + annotations: + # This is required to tell Anthos Service Mesh to inject the gateway with the + # required configuration. + inject.istio.io/templates: gateway + labels: + app: istio-ingressgateway + istio: ingressgateway + spec: + containers: + - name: istio-proxy + image: auto # The image will automatically update each time the pod starts. + resources: + limits: + cpu: 2000m + memory: 1024Mi + requests: + cpu: 100m + memory: 128Mi + serviceAccountName: asm-ingressgateway + +--- +# Optional: PodDisruptionBudget will instruct Kubernetes to ensure at least 1 replica +# is available when possible +apiVersion: policy/v1beta1 +kind: PodDisruptionBudget +metadata: + name: asm-ingressgateway + namespace: {{ asm_ingress_namespace }} +spec: + maxUnavailable: 1 + selector: + matchLabels: + istio: ingressgateway + app: istio-ingressgateway +--- +# Optional: HorizontalPodAutoscaler will automatically scale the gateway replica count based on +# CPU utilization +apiVersion: autoscaling/v2beta1 +kind: HorizontalPodAutoscaler +metadata: + name: asm-ingressgateway + namespace: {{ asm_ingress_namespace }} +spec: + maxReplicas: 5 + metrics: + - resource: + name: cpu + targetAverageUtilization: 80 + type: Resource + minReplicas: 3 + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: asm-ingressgateway +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: asm-ingressgateway + namespace: {{ asm_ingress_namespace }} +rules: +- apiGroups: [""] + resources: ["secrets"] + verbs: ["get", "watch", "list"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: asm-ingressgateway + namespace: {{ asm_ingress_namespace }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: asm-ingressgateway +subjects: +- kind: ServiceAccount + name: asm-ingressgateway +--- +apiVersion: v1 +kind: Service +metadata: + name: asm-ingressgateway + namespace: {{ asm_ingress_namespace }} + labels: + app: istio-ingressgateway + istio: ingressgateway +spec: + ports: + # status-port exposes a /healthz/ready endpoint that can be used with GKE Ingress health checks + - name: status-port + port: 15021 + protocol: TCP + targetPort: 15021 + # Any ports exposed in Gateway resources should be exposed here. + - name: http2 + port: 80 + - name: https + port: 443 + selector: + istio: ingressgateway + app: istio-ingressgateway + type: LoadBalancer +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: asm-ingressgateway + namespace: {{ asm_ingress_namespace }} diff --git a/tools/anthosvmware-ansible-module/roles/asm/templates/namespace-ingress.yaml.j2 b/tools/anthosvmware-ansible-module/roles/asm/templates/namespace-ingress.yaml.j2 new file mode 100644 index 0000000000..19c93dc683 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/templates/namespace-ingress.yaml.j2 @@ -0,0 +1,22 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +apiVersion: v1 +kind: Namespace +metadata: + name: "{{ asm_ingress_namespace }}" + labels: + istio.io/rev: "{{ asm_revision }}" + name: "{{ asm_ingress_namespace }}" diff --git a/tools/anthosvmware-ansible-module/roles/asm/templates/namespace.yaml.j2 b/tools/anthosvmware-ansible-module/roles/asm/templates/namespace.yaml.j2 new file mode 100644 index 0000000000..0bffa81aad --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/templates/namespace.yaml.j2 @@ -0,0 +1,21 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +apiVersion: v1 +kind: Namespace +metadata: + name: "istio-system" + labels: + topology.istio.io/network: "{{ asm_network_id }}" diff --git a/tools/anthosvmware-ansible-module/roles/asm/templates/validating-webhook.yaml.j2 b/tools/anthosvmware-ansible-module/roles/asm/templates/validating-webhook.yaml.j2 new file mode 100644 index 0000000000..4421b7687e --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/asm/templates/validating-webhook.yaml.j2 @@ -0,0 +1,43 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +apiVersion: v1 +kind: Service +metadata: + name: istiod + namespace: istio-system + labels: + istio.io/rev: "{{ asm_revision }}" + app: istiod + istio: pilot + release: istio +spec: + ports: + - port: 15010 + name: grpc-xds # plaintext + protocol: TCP + - port: 15012 + name: https-dns # mTLS with k8s-signed cert + protocol: TCP + - port: 443 + name: https-webhook # validation and injection + targetPort: 15017 + protocol: TCP + - port: 15014 + name: http-monitoring # prometheus stats + protocol: TCP + selector: + app: istiod + istio.io/rev: "{{ asm_revision }}" diff --git a/tools/anthosvmware-ansible-module/roles/cleanup/README.md b/tools/anthosvmware-ansible-module/roles/cleanup/README.md new file mode 100644 index 0000000000..51f555f79d --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/cleanup/README.md @@ -0,0 +1,26 @@ +# Ansible Role: cleanup + +Cleans up sensitive files from the Admin Workstation. +This role evaluates variables passed in from inventory files and their corresponding `group_vars/`. +This role is called with an `include_role` statement from within other roles. + +## Requirements + +No special requirements. + +## Role Variables + +Not applicable. Variables are passed in from inventory files and their corresponding `group_vars/`. + +## Dependencies + +None. + +## Example Playbook + +Not applicable. Called from other roles with an `include_role` statement. + +## **License** + +Copyright 2022 Google LLC. This software is provided as-is, without warranty or representation for any use or purpose. +Your use of it is subject to your agreement with Google. diff --git a/tools/anthosvmware-ansible-module/roles/cleanup/tasks/adminws.yml b/tools/anthosvmware-ansible-module/roles/cleanup/tasks/adminws.yml new file mode 100644 index 0000000000..4862c688d0 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/cleanup/tasks/adminws.yml @@ -0,0 +1,58 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[cleanup] Find GCP SA JSON files on jumphost" + find: + paths: ["{{ yamldestpath }}"] + file_type: file + pattern: '*.json' + register: jsonfiles + +# only runs when JSON files are present +- name: "[cleanup] Clean up GCP SA JSON files on jumphost" + file: + path: "{{ item.path }}" + state: absent + loop: "{{ jsonfiles.files }}" + when: jsonfiles.files is defined + +- name: "[cleanup] Find *credential*.yaml files on jumphost" + find: + paths: ["{{ yamldestpath }}"] + file_type: file + pattern: '*credential*.yaml' + register: credyamlfiles + +# only runs when *credential*.yaml files are present +- name: "[cleanup] Clean up *credential*.yaml on jumphost" + file: + path: "{{ item.path }}" + state: absent + loop: "{{ credyamlfiles.files }}" + when: credyamlfiles.files is defined + +- name: "[cleanup] check if file {{ adminws_name }} exists" + ansible.builtin.stat: + path: "{{ yamldestpath }}/{{ adminws_name }}" + register: adminws_file_exists + +# SSH into new AdminWS to clean up sensitive files +- name: "[cleanup] Clean up credential.yaml and GCP SA files on newly created AdminwWS" + shell: + cmd: "eval '$(cat {{ yamldestpath }}/{{ adminws_name }} | grep @) -oStrictHostKeyChecking=no rm -f {{ adminws_vc_credfile }} {{ adminws_gcpsa }}'" + ignore_errors: true + when: + - adminws_install|default(false)|bool + - adminws_file_exists.stat.exists \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/cleanup/tasks/asserts.yml b/tools/anthosvmware-ansible-module/roles/cleanup/tasks/asserts.yml new file mode 100644 index 0000000000..83a804a759 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/cleanup/tasks/asserts.yml @@ -0,0 +1,20 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[cleanup] Sanity Checks" + ansible.builtin.assert: + that: + - yamldestpath | length > 5 + fail_msg: "Undefined variables." + success_msg: "All required variables are defined." \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/cleanup/tasks/clusters.yml b/tools/anthosvmware-ansible-module/roles/cleanup/tasks/clusters.yml new file mode 100644 index 0000000000..027c510bb0 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/cleanup/tasks/clusters.yml @@ -0,0 +1,91 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[cleanup] Find GCP SA JSON files on admin workstation" + find: + paths: ["{{ yamldestpath }}"] + file_type: file + pattern: '*.json' + register: jsonfiles + +# only runs when JSON files are present +- name: "[cleanup] Clean up GCP SA JSON files on admin workstation" + file: + path: "{{ item.path }}" + state: absent + loop: "{{ jsonfiles.files }}" + when: jsonfiles.files is defined + +- name: "[cleanup] Find *credential*.yaml files on admin workstation" + find: + path: "{{ yamldestpath }}" + file_type: file + pattern: '*credential*.yaml' + register: credyamlfiles + +# only runs when *credential*.yaml files are present +- name: "[cleanup] Clean up *credential*.yaml on admin workstation" + file: + path: "{{ item.path }}" + state: absent + loop: "{{ credyamlfiles.files }}" + when: credyamlfiles.files is defined + +- name: "[cleanup] Find *credential*.yaml files on admin workstation in admin cluster subfolder" + find: + path: "{{ yamldestpath }}/{{ ac_name }}" + file_type: file + pattern: '*credential*.yaml' + register: credyamlfiles + when: + - ac_name is defined + - ac_name | length > 0 + +# only runs when *credential*.yaml files are present in admin cluster subfolder +- name: "[cleanup] Clean up *credential*.yaml on admin workstation in admin cluster subfolder" + file: + path: "{{ item.path }}" + state: absent + loop: "{{ credyamlfiles.files }}" + when: + - credyamlfiles.files is defined + +- name: "[cleanup] Find *credential*.yaml files on admin workstation in user cluster subfolder" + find: + path: "{{ yamldestpath }}/{{ uc_name }}" + file_type: file + pattern: '*credential*.yaml' + register: credyamlfiles + when: + - uc_name is defined + - uc_name | length > 0 + +# only runs when *credential*.yaml files are present in user cluster subfolder +- name: "[cleanup] Clean up *credential*.yaml on admin workstation in user cluster subfolder" + file: + path: "{{ item.path }}" + state: absent + loop: "{{ credyamlfiles.files }}" + when: + - credyamlfiles.files is defined + +- name: "[cleanup] Delete temp directory created for ASM files" + file: + path: "{{ asm_temp }}" + state: absent + when: + - (asm_temp is defined) and (asm_temp | length > 0) + - not (asm_debug_mode|default(false)|bool) + diff --git a/tools/anthosvmware-ansible-module/roles/cleanup/tasks/main.yml b/tools/anthosvmware-ansible-module/roles/cleanup/tasks/main.yml new file mode 100644 index 0000000000..dc0f5298a1 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/cleanup/tasks/main.yml @@ -0,0 +1,37 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[cleanup] Include tasks - asserts.yml" + include_tasks: asserts.yml + when: ac_name is defined or uc_name is defined or adminws_name is defined + +- name: "[cleanup] Set folder name for GCP SA JSON key files" + ansible.builtin.set_fact: + job_sakeyfolder: "{{ sakeyfolder_base }}-{{ ac_name if ac_name is defined else uc_name if uc_name is defined else adminws_name if adminws_name is defined else 'upload' }}" + +- name: "[cleanup] Clean up GCP SA JSON files on jumphost" + file: + path: "{{ job_sakeyfolder }}" + state: absent + +- name: "[cleanup] Clean up sensitive files on jumphost" + include_tasks: adminws.yml + when: adminws_name is defined + +- name: "[cleanup] Clean up sensitive file on admin workstation" + include_tasks: clusters.yml + when: + - (ac_name is defined and ac_name | length > 0) or + (uc_name is defined and uc_name | length > 0) or + (acm_membership is defined and acm_membership | length > 0) diff --git a/tools/anthosvmware-ansible-module/roles/copy_credentials/README.md b/tools/anthosvmware-ansible-module/roles/copy_credentials/README.md new file mode 100644 index 0000000000..f225c5c481 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/copy_credentials/README.md @@ -0,0 +1,26 @@ +# Ansible Role: copy_credentials + +Conditionally copies sensitive files onto the Admin Workstation. +This role evaluates variables passed in from inventory files and their corresponding `group_vars/`. +This role is called with an `include_role` statement from within other roles. + +## Requirements + +No special requirements. + +## Role Variables + +Not applicable. Variables are passed in from inventory files and their corresponding `group_vars/`. + +## Dependencies + +None. + +## Example Playbook + +Not applicable. Called from other roles with an `include_role` statement. + +## **License** + +Copyright 2022 Google LLC. This software is provided as-is, without warranty or representation for any use or purpose. +Your use of it is subject to your agreement with Google. diff --git a/tools/anthosvmware-ansible-module/roles/copy_credentials/tasks/asserts.yml b/tools/anthosvmware-ansible-module/roles/copy_credentials/tasks/asserts.yml new file mode 100644 index 0000000000..ad73766f39 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/copy_credentials/tasks/asserts.yml @@ -0,0 +1,21 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[copy_creds] Sanity Checks" + ansible.builtin.assert: + that: + - yamldestpath | length > 5 + - sakeyfolder_base | length > 5 + fail_msg: "Undefined variables." + success_msg: "All required variables are defined." \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/copy_credentials/tasks/gcpsa.yml b/tools/anthosvmware-ansible-module/roles/copy_credentials/tasks/gcpsa.yml new file mode 100644 index 0000000000..fc26f69846 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/copy_credentials/tasks/gcpsa.yml @@ -0,0 +1,31 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[copy_creds] Copy GCP SA JSON Key file from Tower Vault" + ansible.builtin.copy: + src: "{{ gcpsa_key_file }}" + dest: "{{ job_sakeyfolder }}/{{ gcpsa_dest_file }}" + mode: 0600 + when: + - gcpsa_key_file is defined + - gcpsa_key_file | length > 1 + +- name: "[copy_creds] Create GCP SA JSON Key file from Inventory" + ansible.builtin.template: + src: gcpsa.json.j2 + dest: "{{ job_sakeyfolder }}/{{ gcpsa_dest_file }}" + mode: 0600 + when: + - gcpsa_key_string is defined + - gcpsa_key_string | length > 1 \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/copy_credentials/tasks/main.yml b/tools/anthosvmware-ansible-module/roles/copy_credentials/tasks/main.yml new file mode 100644 index 0000000000..3b1537e47c --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/copy_credentials/tasks/main.yml @@ -0,0 +1,159 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[copy_creds] Set folder name for GCP SA JSON key files" + ansible.builtin.set_fact: + job_sakeyfolder: "{{ sakeyfolder_base }}-{{ ac_name if ac_name is defined else uc_name if uc_name is defined else adminws_name if adminws_name is defined else 'upload' }}" + +- name: "[copy_creds] Create folder for GCP SA JSON key files" + ansible.builtin.file: + path: "{{ job_sakeyfolder }}" + mode: 0700 + state: directory + +# Not applicable when using Ansible Tower/AWX +# The playbooks/sakeys/ folder can be used to store your GCP SA JSON key files when using OSS Ansible on the command-line +- name: "[copy_creds] Copy local GCP SA JSON key files" + ansible.builtin.copy: + src: "{{ item }}" + dest: "{{ job_sakeyfolder }}" + mode: 0600 + # with_fileglob assumes JSON files are located in playbooks/sakeys/ on the Ansible Controller + with_fileglob: + - sakeys/*.json + +- name: "[copy_creds] GCA SA Component Access Account Key" + include_tasks: + file: gcpsa.yml + vars: + gcpsa_key_file: "{{ component_access_gcpsa | default('') }}" + gcpsa_key_string: "{{ component_access_gcpsa_content | default('') }}" + gcpsa_dest_file: "{{ component_access_gcpsa_path }}" + when: + - is_admin_workstation | default(false) | bool + or is_admin_cluster | default(false) | bool + or is_build_server | default(false) | bool + or is_upload | default(false) | bool + or is_standalone | default(false) | bool + +- name: "[copy_creds] GCA SA Connect Register Account Key" + include_tasks: + file: gcpsa.yml + vars: + gcpsa_key_file: "{{ connect_register_gcpsa | default('') }}" + gcpsa_key_string: "{{ connect_register_gcpsa_content | default('') }}" + gcpsa_dest_file: "{{ connect_register_gcpsa_path }}" + when: + - is_admin_cluster | default(false) | bool + or is_user_cluster | default(false) | bool + or is_asm | default(false) | bool + or is_label | default(false) | bool + or is_standalone | default(false) | bool + +- name: "[copy_creds] GCA SA Logging and Monitoring Account Key" + include_tasks: + file: gcpsa.yml + vars: + gcpsa_key_file: "{{ logging_monitoring_gcpsa | default('') }}" + gcpsa_key_string: "{{ logging_monitoring_gcpsa_content | default('') }}" + gcpsa_dest_file: "{{ logging_monitoring_gcpsa_path }}" + when: + - is_admin_cluster | default(false) | bool + or is_user_cluster | default(false) | bool + or is_standalone | default(false) | bool + +- name: "[copy_creds] GCA SA Audit Logging Account Key" + include_tasks: + file: gcpsa.yml + vars: + gcpsa_key_file: "{{ audit_logging_gcpsa | default('') }}" + gcpsa_key_string: "{{ audit_logging_gcpsa_content | default('') }}" + gcpsa_dest_file: "{{ audit_logging_gcpsa_path }}" + when: + - is_admin_cluster | default(false) | bool + or is_user_cluster | default(false) | bool + or is_standalone | default(false) | bool + +- name: "[copy_creds] GCA SA AIS Config Upload Account Key" + include_tasks: + file: gcpsa.yml + vars: + gcpsa_key_file: "{{ ais_gcpsa | default('') }}" + gcpsa_key_string: "{{ ais_gcpsa_content | default('') }}" + gcpsa_dest_file: "{{ ais_gcpsa_path }}" + when: + - is_ais | default(false) | bool + or is_standalone | default(false) | bool + +- name: "[copy_creds] GCA SA ASM Management Account Key" + include_tasks: + file: gcpsa.yml + vars: + gcpsa_key_file: "{{ asm_gcpsa | default('') }}" + gcpsa_key_string: "{{ asm_gcpsa_content | default('') }}" + gcpsa_dest_file: "{{ asm_gcpsa_path }}" + when: + - is_asm | default(false) | bool + or is_standalone | default(false) | bool + +- name: "[copy_creds] GCA SA ACM Management Account Key" + include_tasks: + file: gcpsa.yml + vars: + gcpsa_key_file: "{{ acm_gcpsa | default('') }}" + gcpsa_key_string: "{{ acm_gcpsa_content | default('') }}" + gcpsa_dest_file: "{{ acm_gcpsa_path }}" + when: + - is_acm | default(false) | bool + or is_standalone | default(false) | bool + +- name: "[copy_creds] Set credential.yaml directory (Admin WS)" + ansible.builtin.set_fact: + cred_path: "{{ yamldestpath }}" + when: + - is_admin_workstation | default(false) | bool + or is_standalone | default(false) | bool + +- name: "[copy_creds] Set credential.yaml directory (Cluster)" + ansible.builtin.set_fact: + cred_path: "{{ yamldestpath }}/{{ ac_name if ac_name is defined else uc_name }}" + when: + - is_admin_cluster | default(false) | bool + or is_user_cluster | default(false) | bool + or is_standalone | default(false) | bool + +- name: "[copy_creds] Ensure credentials directory exists" + ansible.builtin.file: + path: "{{ cred_path }}" + state: directory + mode: 0700 + when: + - is_admin_workstation | default(false) | bool + or is_admin_cluster | default(false) | bool + or is_user_cluster | default(false) | bool + or is_standalone | default(false) | bool + +- name: "[copy_creds] Templating YAML files - vCenter credentials" + ansible.builtin.template: + src: credential.yaml.j2 + dest: "{{ cred_path }}/credential.yaml" + mode: 0600 + when: + - is_admin_workstation | default(false) | bool + or is_admin_cluster | default(false) | bool + or is_user_cluster | default(false) | bool + or is_standalone | default(false) | bool + - glb_vc_username is defined + - glb_vc_password is defined \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/copy_credentials/templates/credential.yaml.j2 b/tools/anthosvmware-ansible-module/roles/copy_credentials/templates/credential.yaml.j2 new file mode 100644 index 0000000000..e262e1e910 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/copy_credentials/templates/credential.yaml.j2 @@ -0,0 +1,28 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +apiVersion: v1 +kind: CredentialFile +# list of credentials +items: +# reference name for this credential entry +- name: vCenter + username: "{{ glb_vc_username }}" + password: "{{ glb_vc_password }}" +{% if (glb_privatereg_url is defined) and (glb_privatereg_url|length > 0) %} +- name: "privateRegistry" + username: "{{ glb_privatereg_username }}" + password: "{{ glb_privatereg_password }}" +{% endif %} diff --git a/tools/anthosvmware-ansible-module/roles/copy_credentials/templates/gcpsa.json.j2 b/tools/anthosvmware-ansible-module/roles/copy_credentials/templates/gcpsa.json.j2 new file mode 100644 index 0000000000..d3a94345a8 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/copy_credentials/templates/gcpsa.json.j2 @@ -0,0 +1,16 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +{{ gcpsa_key_string | to_nice_json }} \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/label_membership/README.md b/tools/anthosvmware-ansible-module/roles/label_membership/README.md new file mode 100644 index 0000000000..87138fc4fb --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/label_membership/README.md @@ -0,0 +1,54 @@ +# Ansible Role: label_membership + +Labels the Anthos membership in GCP with custom labels. This role works for both Admin clusters and User clusters. + +## Requirements + +This role requires the connect register account key and project id where the cluster is registered. +The key is copied to the admin workstation, and project id configured with the following variables: + +```YAML +# For Admin Cluster +ac_gkeconnect_projectid: "PROJECT_ID" +connect_register_gcpsa: '{{ lookup("env", "GCPSA_CONREG_FILE") }}' +connect_register_gcpsa_path: "connect-register.json" + +# For User Cluster +uc_gkeconnect_projectid: "PROJECT_ID" +connect_register_gcpsa: '{{ lookup("env", "GCPSA_CONREG_FILE") }}' +connect_register_gcpsa_path: "connect-register.json" +``` + +## Role Variables + +Available variables are listed below: + +```YAML +gcp_labels: + example: value + example2: value2 +``` + +The labels to be configured for the environment in map format. + +## Dependencies + +None. + +## Example Playbook + +This role should generally run after the admin cluster or user cluster install, and not during uninstallation. + +```YAML +- name: "[ac] Cluster management" + hosts: all + gather_facts: False + roles: + - admincluster + - { role: label_membership, when: gcp_labels is defined and (gcp_labels|length > 0) and ac_install|default(true)|bool } +``` + +## **License** + +Copyright 2022 Google LLC. This software is provided as-is, without warranty or representation for any use or purpose. +Your use of it is subject to your agreement with Google. diff --git a/tools/anthosvmware-ansible-module/roles/label_membership/tasks/main.yml b/tools/anthosvmware-ansible-module/roles/label_membership/tasks/main.yml new file mode 100644 index 0000000000..e23a8e6855 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/label_membership/tasks/main.yml @@ -0,0 +1,64 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[labels] End play if gcp_labels is not defined" + ansible.builtin.meta: end_play + when: not (gcp_labels is defined and (gcp_labels|length > 0)) + +- name: "Include role for GCP SA JSON key files" + ansible.builtin.include_role: + name: copy_credentials + vars: + is_label: true + +- name: "[labels] Block - Manage GCP labels" # noqa var-spacing + block: + - name: "[labels] Set membership details" + ansible.builtin.set_fact: + target_membership: "{{ ac_name if ac_name is defined else uc_name }}" + target_project: "{{ ac_gkeconnect_projectid if ac_gkeconnect_projectid is defined else uc_gkeconnect_projectid }}" + + - name: "[labels] Activate Connect Register SA" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gcloud + - auth + - activate-service-account + - --key-file + - "{{ job_sakeyfolder }}/{{ connect_register_gcpsa_path }}" + + - name: "[labels] Update labels on membership" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gcloud + - container + - hub + - memberships + - update + - --location=global + - --project={{ target_project }} + - "{{ target_membership }}" + - --update-labels + - "{{ gcp_labels.items() | map('join', '=') | join(',')}}" + + environment: + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" + http_proxy: "{{ glb_proxyurl | default('') }}" + https_proxy: "{{ glb_proxyurl | default('') }}" + no_proxy: "{{ glb_noproxy | default('') }}" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/repave/README.md b/tools/anthosvmware-ansible-module/roles/repave/README.md new file mode 100644 index 0000000000..4c0bd02f33 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/repave/README.md @@ -0,0 +1,34 @@ +# Ansible Role: repave + +Triggers creation of new VMs and replaces old VMs. +This recycles all VMs of the Admin and User Cluster with the exception of the +Admin Cluster control plane node. + +## Requirements + +No special requirements. + +## Role Variables + +Available variables are listed below, along with default values (see `defaults/main.yml`): + +``` +oneOrMultipleVars +``` + +Explanation. + +## Dependencies + +None. + +## Example Playbook + +``` + +``` + +## **License** + +Copyright 2022 Google LLC. This software is provided as-is, without warranty or representation for any use or purpose. +Your use of it is subject to your agreement with Google. diff --git a/tools/anthosvmware-ansible-module/roles/repave/tasks/asserts.yml b/tools/anthosvmware-ansible-module/roles/repave/tasks/asserts.yml new file mode 100644 index 0000000000..9285b8cd4b --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/repave/tasks/asserts.yml @@ -0,0 +1,21 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[repave] Sanity Checks" + assert: + that: + - uc_name is defined or ac_name is defined + fail_msg: "Undefined variables." + success_msg: "All required variables are defined." \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/repave/tasks/machine.yml b/tools/anthosvmware-ansible-module/roles/repave/tasks/machine.yml new file mode 100644 index 0000000000..76f767692c --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/repave/tasks/machine.yml @@ -0,0 +1,116 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[repave] Get current list of machines" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ repave_kubeconfig }}" + - get + - machines + - --selector=set!=master + - --no-headers + register: curr_machines + +- name: "[repave] Annotate machine '{{ machine }}' for replacement" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ repave_kubeconfig }}" + - annotate + - machine + - "{{ machine }}" + - onprem.cluster.gke.io/repair-machine=true + +- name: "[repave] Remove machine '{{ machine }}' to trigger replacement" + async: 1800 + poll: 15 + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ repave_kubeconfig }}" + - delete + - machine + - "{{ machine }}" + +- name: "[repave] Wait for replacement machine name" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ repave_kubeconfig }}" + - get + - machines + - --selector=set!=master + - --no-headers + register: new_machines + retries: 10 + delay: 5 + until: new_machines.stdout_lines | length >= curr_machines.stdout_lines | length + +- name: "[repave] Set new machine name" + ansible.builtin.set_fact: + new_machine_name: "{{ new_machines.stdout_lines | difference(curr_machines.stdout_lines) }}" + +- name: "[repave] Wait for replacement machine '{{ new_machine_name | join(' ') }}' to be provisioned" + async: 610 + poll: 15 + ansible.builtin.command: + argv: "{{ _base_argv + new_machine_name }}" + vars: + _base_argv: + - kubectl + - --kubeconfig + - "{{ repave_kubeconfig }}" + - get + - machine + - -o + - jsonpath={.status.phase} + register: _machine + retries: 10 + delay: 5 + until: _machine.stdout=="Ready" + +- name: "[repave] Wait for replacement machine node" + ansible.builtin.command: + argv: "{{ _base_argv + new_machine_name }}" + vars: + _base_argv: + - kubectl + - --kubeconfig + - "{{ repave_kubeconfig }}" + - get + - node + register: _node + retries: 15 + delay: 5 + until: not 'not found' in _node.stdout + +- name: "[repave] Wait for replacement machine '{{ new_machine_name | join(' ') }}' to be ready" + async: 610 + poll: 15 + ansible.builtin.command: + argv: "{{ _base_argv + new_machine_name }}" + vars: + _base_argv: + - kubectl + - --kubeconfig + - "{{ repave_kubeconfig }}" + - wait + - --for=condition=Ready + - --timeout=600s + - node diff --git a/tools/anthosvmware-ansible-module/roles/repave/tasks/machinedeployment.yml b/tools/anthosvmware-ansible-module/roles/repave/tasks/machinedeployment.yml new file mode 100644 index 0000000000..636eb402b5 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/repave/tasks/machinedeployment.yml @@ -0,0 +1,78 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[repave] Set repave id" + ansible.builtin.set_fact: + repave_id: '{{ lookup("pipe", "date +%s") }}' + +- name: "[repave] Annotate '{{ machinedeployment }}'' MachineDeployment with 'lastrepave: {{ repave_id }}' to trigger repave" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ repave_kubeconfig }}" + - patch + - machinedeployment + - "{{ machinedeployment }}" + - --patch + - "spec:\n template:\n metadata:\n annotations:\n lastrepave: '{{ repave_id }}'" + - --type + - merge + +- name: "[repave] Wait for new '{{ machinedeployment }}' names with 'lastrepave: {{ repave_id }}'" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ repave_kubeconfig }}" + - get + - machine + - "-o=jsonpath='{.items[?(@.metadata.annotations.lastrepave==\"{{ repave_id }}\")].metadata.name}'" + - --no-headers + register: new_machines + retries: 240 + delay: 15 + until: new_machines.stdout | split(' ') | length >= (machine_count | int) + +- name: "[repave] Wait for '{{ machinedeployment }}' machines to be provisioned" + async: 610 + poll: 15 + ansible.builtin.command: + argv: "{{ _base_argv + (new_machines.stdout | replace(\"'\",'') | split(' ')) }}" + vars: + _base_argv: + - kubectl + - --kubeconfig + - "{{ repave_kubeconfig }}" + - wait + - --for=jsonpath=.status.phase=Ready + - --timeout=600s + - machine + +- name: "[repave] Wait for '{{ machinedeployment }}' nodes to be ready" + async: 610 + poll: 15 + ansible.builtin.command: + argv: "{{ _base_argv + (new_machines.stdout | replace(\"'\",'') | split(' ')) }}" + vars: + _base_argv: + - kubectl + - --kubeconfig + - "{{ repave_kubeconfig }}" + - wait + - --for=condition=Ready + - --timeout=600s + - node + diff --git a/tools/anthosvmware-ansible-module/roles/repave/tasks/main.yml b/tools/anthosvmware-ansible-module/roles/repave/tasks/main.yml new file mode 100644 index 0000000000..5997287f6d --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/repave/tasks/main.yml @@ -0,0 +1,20 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[repave] Include tasks - asserts.yml" + include_tasks: asserts.yml + +- name: "[repave] Include tasks - repave.yml" + include_tasks: repave.yml \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/repave/tasks/repave.yml b/tools/anthosvmware-ansible-module/roles/repave/tasks/repave.yml new file mode 100644 index 0000000000..e8b6079f88 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/repave/tasks/repave.yml @@ -0,0 +1,35 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[repave] Set kubeconfig file" + ansible.builtin.set_fact: + repave_kubeconfig: "{{ uc_name + '-kubeconfig' if uc_name is defined else 'kubeconfig' }}" + +- name: "[repave] Query machines to repave" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ repave_kubeconfig }}" + - get + - machines + - --selector=set!=master + - --no-headers + register: repave_machines + +- name: "[repave] Repave machines" + include_tasks: machine.yml + loop: "{{ repave_machines.stdout_lines | default('') }}" + loop_control: + loop_var: machine diff --git a/tools/anthosvmware-ansible-module/roles/upload_artifactory/defaults/main.yml b/tools/anthosvmware-ansible-module/roles/upload_artifactory/defaults/main.yml new file mode 100644 index 0000000000..dba99c1c0c --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/upload_artifactory/defaults/main.yml @@ -0,0 +1,37 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# role upload_artifactory default values +workdir: "/tmp/upload_artifacts" # specify the directory to store all the downloaded files +upload_artifactory_install: true +component_access_gcpsa: "" # content of access gcp sa key +component_access_gcpsa_path: "" # file name to write out the contents of the gcp sa to +sakeyfolder_base: "{{ workdir }}/sakeys" # folder where the sa keys are copied to + +# artifactory specific +upload_artifactory_fqdn: "" # artifactory url (i.e artifactory.me.com) +upload_artifactory_validate_cert: false +upload_artifactory_username: "" +upload_artifactory_password: "" +upload_artifactory_ca_cert: "" +upload_artifactory_repo_name: "" # generic repo to upload artifacts to (i.e anthos-generic) +upload_artifactory_url: "https://{{ upload_artifactory_username }}:{{ upload_artifactory_password }}@{{ upload_artifactory_fqdn }}/artifactory" +upload_artifactory_url_with_repo: "https://{{ upload_artifactory_username }}:{{ upload_artifactory_password }}@{{ upload_artifactory_fqdn }}/artifactory/{{ upload_artifactory_repo_name }}" # noqa 204 + +# Files +govc_version: "" # version of anthos (ie. 0.29.0) +asm_asmcli_version: "" # version of asmcli (ie. 1.13.7-asm.3-config1) +upload_artifactory_http_artifacts: "" # list of files to download from an http endpoint +upload_artifactory_gs_artifacts: "" # list of files to download from gcs diff --git a/tools/anthosvmware-ansible-module/roles/upload_artifactory/tasks/asserts.yml b/tools/anthosvmware-ansible-module/roles/upload_artifactory/tasks/asserts.yml new file mode 100644 index 0000000000..cb6b992153 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/upload_artifactory/tasks/asserts.yml @@ -0,0 +1,30 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[upload_artifactory] Sanity Checks" + ansible.builtin.assert: + that: + - sakeyfolder_base is defined + - upload_artifactory_fqdn | length >= 4 + - upload_artifactory_username | length >= 4 + - upload_artifactory_password | length >= 8 + - upload_artifactory_repo_name | length >= 2 + - govc_version | length >= 2 + - asm_asmcli_version | length >= 2 + - gcloud_version | length >= 2 + - upload_artifactory_http_artifacts + - upload_artifactory_gs_artifacts + fail_msg: "Undefined variables." + success_msg: "All required variables are defined." + diff --git a/tools/anthosvmware-ansible-module/roles/upload_artifactory/tasks/install.yml b/tools/anthosvmware-ansible-module/roles/upload_artifactory/tasks/install.yml new file mode 100644 index 0000000000..4dfbfcfa5b --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/upload_artifactory/tasks/install.yml @@ -0,0 +1,205 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[upload_artifactory] Create folder" + ansible.builtin.file: + path: "{{ workdir }}" + state: directory + mode: 0770 + +- name: "[upload_artifactory] Block - Copy/delete GCP SA JSON key files, Upload artifacts" + block: + - name: "Include role for GCP SA JSON key files" + ansible.builtin.include_role: + name: copy_credentials + vars: + is_upload: true + + - name: "[upload_artifactory] Check Existence of http files in artifactory" + ansible.builtin.command: + argv: + - curl + - -k + - -s + - -o + - /dev/null + - -w + - "%{http_code}" + - "{{ upload_artifactory_url }}/api/storage/{{ upload_artifactory_repo_name }}/{{ item.dst }}" + loop: "{{ upload_artifactory_http_artifacts }}" + loop_control: + label: "{{ item.file }}" + no_log: true + ignore_errors: true + register: http_check_result + + - name: "[upload_artifactory] Output Check results Masked" + ansible.builtin.debug: + msg: "{{ item | to_json | replace(upload_artifactory_password, '****') }}" + loop: "{{ http_check_result.results }}" + loop_control: + label: "{{ item.item.file }}" + + - name: "[upload_artifactory] Download files from remote urls" + ansible.builtin.command: + argv: + - curl + - -k + - -L + - "{{ item.item.src }}" + - -o + - "{{ workdir }}/{{ item.item.file }}" + loop: "{{ http_check_result.results }}" + loop_control: + label: "{{ item.item.file }}" + no_log: true + ignore_errors: true + register: http_file_download_result + when: item.stdout != "200" + + - name: "[upload_artifactory] Output Download results Masked" + ansible.builtin.debug: + msg: "{{ item | to_json | replace(upload_artifactory_password, '****') }}" + loop: "{{ http_file_download_result.results }}" + loop_control: + label: "{{ item.item.item.file }}" + + - name: "[upload_artifactory] Upload HTTP Files to Artifactory" + ansible.builtin.command: + argv: + - curl + - -k + - -X + - PUT + - -T + - "{{ workdir }}/{{ item.item.file }}" + - "{{ upload_artifactory_url_with_repo }}/{{ item.item.dst }}" + loop: "{{ http_check_result.results }}" + loop_control: + label: "{{ item.item.file }}" + no_log: true + ignore_errors: true + register: http_file_upload_result + when: item.stdout != "200" + + - name: "[upload_artifactory] Output Upload results Masked" + ansible.builtin.debug: + msg: "{{ item | to_json | replace(upload_artifactory_password, '****') }}" + loop: "{{ http_file_upload_result.results }}" + loop_control: + label: "{{ item.item.item.file }}" + failed_when: + - item.changed | bool + - "'created' not in item.stdout" + - item.item.item.file not in item.stdout + + - name: "[upload_artifactory] Switch to Access Service Account Name" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gcloud + - auth + - activate-service-account + - --key-file + - "{{ job_sakeyfolder }}/{{ component_access_gcpsa_path }}" + + - name: "[upload_artifactory] Check Existence of gcs files in artifactory" + ansible.builtin.command: + argv: + - curl + - -k + - -s + - -o + - /dev/null + - -w + - "%{http_code}" + - "{{ upload_artifactory_url }}/api/storage/{{ upload_artifactory_repo_name }}/{{ item.dst }}" + loop: "{{ upload_artifactory_gs_artifacts }}" + loop_control: + label: "{{ item.file }}" + no_log: true + ignore_errors: true + register: gcs_check_result + + - name: "[upload_artifactory] Output Check results Masked" + ansible.builtin.debug: + msg: "{{ item | to_json | replace(upload_artifactory_password, '****') }}" + loop: "{{ gcs_check_result.results }}" + loop_control: + label: "{{ item.item.file }}" + + - name: "[upload_artifactory] Download files from GCS" + ansible.builtin.command: # noqa 305 301 no-changed-when + argv: + - gsutil + - cp + - "{{ item.item.src }}" + - "{{ workdir }}/{{ item.item.file }}" + creates: "{{ workdir }}/{{ item.item.file }}" + loop: "{{ gcs_check_result.results }}" + loop_control: + label: "{{ item.item.file }}" + no_log: true + ignore_errors: true + register: gcs_file_download_result + when: item.stdout != "200" + + - name: "[upload_artifactory] Output Download results Masked" + ansible.builtin.debug: + msg: "{{ item | to_json | replace(upload_artifactory_password, '****') }}" + loop: "{{ gcs_file_download_result.results }}" + loop_control: + label: "{{ item.item.item.file }}" + failed_when: + - item.changed | bool + - item.failed | bool + + - name: "[upload_artifactory] Upload GCS Files to Artifactory" + ansible.builtin.command: + argv: + - curl + - -k + - -X + - PUT + - -T + - "{{ workdir }}/{{ item.item.file }}" + - "{{ upload_artifactory_url_with_repo }}/{{ item.item.dst }}" + loop: "{{ gcs_check_result.results }}" + loop_control: + label: "{{ item.item.file }}" + no_log: true + ignore_errors: true + register: gcs_file_upload_result + when: item.stdout != "200" + + - name: "[upload_artifactory] Output Upload results Masked" + ansible.builtin.debug: + msg: "{{ item | to_json | replace(upload_artifactory_password, '****') }}" + loop: "{{ gcs_file_upload_result.results }}" + loop_control: + label: "{{ item.item.item.file }}" + failed_when: + - item.changed | bool + - "'created' not in item.stdout" + - item.item.item.file not in item.stdout + + environment: # noqa 204 + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" # noqa 204 + http_proxy: "{{ glb_proxyurl }}" + https_proxy: "{{ glb_proxyurl }}" + no_proxy: "{{ glb_noproxy }}" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup diff --git a/tools/anthosvmware-ansible-module/roles/upload_artifactory/tasks/main.yml b/tools/anthosvmware-ansible-module/roles/upload_artifactory/tasks/main.yml new file mode 100644 index 0000000000..6be0732156 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/upload_artifactory/tasks/main.yml @@ -0,0 +1,25 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[upload_artifactory] Include tasks - asserts.yml" + include_tasks: asserts.yml + +- name: "[upload_artifactory] Include tasks - install.yml" + include_tasks: install.yml + when: upload_artifactory_install|default(false)|bool + +- name: "[upload_artifactory] Include tasks - uninstall.yml" + include_tasks: uninstall.yml + when: not upload_artifactory_install|default(false)|bool diff --git a/tools/anthosvmware-ansible-module/roles/upload_artifactory/tasks/uninstall.yml b/tools/anthosvmware-ansible-module/roles/upload_artifactory/tasks/uninstall.yml new file mode 100644 index 0000000000..c342d74039 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/upload_artifactory/tasks/uninstall.yml @@ -0,0 +1,76 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Ansible Lint skipping rules +# ruleID 301 Commands should not change things if nothing needs doing +# ruleID 204 Lines should be no longer than 160 chars +- name: "[upload_artifactory] Block - Delete artifacts from Artificatory" + block: + # - name: Get Results + # ansible.builtin.debug: + # msg: "{{ item }}" + # loop: + # - "{{ upload_artifactory_http_artifacts }}" + # - "{{ upload_artifactory_gs_artifacts }}" + + - name: "[upload_artifactory] Check Existence of files in artifactory" + ansible.builtin.command: + argv: + - curl + - -k + - -s + - -o + - /dev/null + - -w + - "%{http_code}" + - "{{ upload_artifactory_url }}/api/storage/{{ upload_artifactory_repo_name }}/{{ item.dst }}" + loop: "{{ upload_artifactory_http_artifacts + upload_artifactory_gs_artifacts }}" + no_log: true + ignore_errors: true + register: check_result + + - name: "[upload_artifactory] Output Check results Masked" + ansible.builtin.debug: + msg: "{{ item | to_json | replace(upload_artifactory_password, '****') }}" + loop: "{{ check_result.results }}" + loop_control: + label: "{{ item.item.file }}" + + - name: "[upload_artifactory] Delete HTTP Files from Artifactory" + ansible.builtin.command: + argv: + - curl + - -k + - -X + - DELETE + - "{{ upload_artifactory_url_with_repo }}/{{ item.item.dst }}" + loop: "{{ check_result.results }}" + no_log: true + when: item.stdout == "200" + + - name: "[upload_artifactory] Delete work directory" + ansible.builtin.file: + path: "{{ workdir }}" + state: absent + + environment: + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" # noqa 204 + http_proxy: "{{ glb_proxyurl }}" + https_proxy: "{{ glb_proxyurl }}" + no_proxy: "{{ glb_noproxy }}" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup diff --git a/tools/anthosvmware-ansible-module/roles/usercluster/README.md b/tools/anthosvmware-ansible-module/roles/usercluster/README.md new file mode 100644 index 0000000000..98bdb84494 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/usercluster/README.md @@ -0,0 +1,259 @@ +# Ansible Role: usercluster + +Installs and configures the User Cluster. + +## Requirements + +Optionally include the `ais` role to set up the Anthos Identity Service (AIS) with OIDC for user login. + +## Role Variables + +### User Cluster + +Available variables are listed below, along with default values (see `defaults/main.yml`): + +``` +uc_name: "uc1" +``` + +Name of the user cluster. This will be the name that shows up in the Google Cloud Console. It will +also be the name that prepends all of the node names created for the user cluster along with a +unique hash (ex: `uc1-p01-5d66697477-rzvj6`). + +``` +component_access_gcpsa: "" +component_access_gcpsa_path: "" +uc_kubeconfig: "{{ uc_name }}-kubeconfig" +``` + +Source and destination of component access Google Service Account ("GSA"). +Name of user cluster kubeconfig file. + +``` +# vSphere/vCenter +# uc_vc_fqdn: "" +# uc_vc_username: "" +# uc_vc_password: "" +# uc_vc_credfile: "credential.yaml" +# uc_vc_credentry: "vCenter" +# uc_vc_datacenter: "" +# uc_vc_datastore: "" +# uc_vc_cluster: "" +# uc_vc_network: "" # VM Network +# uc_vc_folder: "" # optional +# uc_vc_respool: "" # if default resourcePool use /Resources +# uc_vc_cacertpath: "/home/ubuntu/vcenter.pem" # Default location with automatically downloaded cert file name +``` + +Settings to access vSphere/vCenter, including object names. +These settings are inherited from the admin cluster by default and can be omitted. + +``` +# Networking +uc_nw_ipallocmode: "" # dhcp or static +uc_nw_ipfile: "{{ uc_name }}-ip-block.yaml" +uc_nw_gw: "" # gateway +uc_nw_nm: "" # netmask 255.255.255.0 or similar +uc_nw_dns: [""] +uc_nw_ntp: ["ntp.ubuntu.com"] +uc_nw_searchdomains: [""] +uc_nw_servicecidr: "10.96.0.0/20" +uc_nw_podcidr: "" +uc_nw_vc_net: "VM Network" + +# values for the file content of network.ipMode.ipBlockFilePath +uc_ipblock_netmask: "255.255.255.0" +uc_ipblock_gateway: "10.20.0.255" +uc_ipblock_ips: ["10.20.0.51","10.20.0.52","10.20.0.53","10.20.0.54"] +``` + +Network settings. + +``` +# Load balancing +uc_lb_kind: MetalLB +uc_lb_vips_cp: "" +uc_lb_vips_ingress: "" +uc_lb_metallb_ips: "" +``` + +Load balancing and VIPs with MetalLB. + +``` +# masternode sizing +uc_masternode_cpus: 4 +uc_masternode_mem: 8192 +uc_masternode_replicas: 1 +uc_masternode_datastore: "" +``` + +Master node sizing and number of replicas with default values. +Set replicas to `3` for a highly-available (HA) User Cluster control plane. +Name of vSphere datastore if different from the Admin Cluster. + +``` +uc_antiaffinitygroups: false +``` + +Toggle distribution of VMs onto different virtualization hosts. + +``` +# nodepools +uc_nodepools_name: "{{ uc_name }}-pool-1" +uc_nodepools_cpus: 4 +uc_nodepools_mem: 8192 +uc_nodepools_replicas: 3 +uc_nodepools_osimagetype: ubuntu_containerd +uc_nodepools_vsphere_tags_category: "" # optional and must be precreated in vSphere +uc_nodepools_vsphere_tags_name: "" # optional and must be precreated in vSphere +``` + +Node pool name, resources, replicas, OS type, and optional vSphere tags, which must be created in advance to be assigned. + +``` +# GCP project IDs +uc_stackdriver_projectid: "" +uc_stackdriver_clusterlocation: "" +uc_stackdriver_enablevpc: false +logging_monitoring_gcpsa_path: "" +uc_stackdriver_disablevsphereresourcemetrics: false +uc_gkeconnect_projectid: "" +connect_register_gcpsa_path: "" +uc_cloudauditlogging_projectid: "" +uc_cloudauditlogging_clusterlocation: "" +audit_logging_gcpsa_path: "" +``` + +Google Cloud Project IDs, region, and GSA paths. + +``` +uc_autorepair: true +``` + +Toogle Autorepair feature. + +``` +# Kubernetes Secrets at-rest encryption +uc_secretsencryption_mode: "GeneratedKey" +uc_secretsencryption_keyversion: 1 +``` + +Enable encryption at-rest of Kubernetes Secrets. + +``` +# Optional --skip-validation for gkectl +uc_skipvalidations: "" +uc_verbosity: 8 +``` + +Verbosity and option to skip all or specific `gkectl` preflight checks. + +### Anthos Service Mesh + +Optionally define the below variables to be used with the ASM install, upgrade, and uninstall playbooks + +``` +# Optional: ASM +asm_version: "1.13" +asm_revision: "asm-1137-3" +asm_asmcli_version: "1.13.7-asm.3-config1" +asm_network_id: "uc1-6789" +``` +ASM Service Mesh version, revision, and network ID information. + +The available `asmcli` versions for use can be found by using the below command: +``` +gsutil ls gs://csm-artifacts/asm/ +``` + +You can filter for a specific revision with `grep`. For example: +``` +gsutil ls gs://csm-artifacts/asm/ | grep 1.14 +``` + +> **Note:** `asm_network_id` is used for configuring a multi-cluster mesh. It *must be unique* for proper +service discovery within the mesh. + +``` +# ASM - GCP related +asm_gcp_project: "asm-test" # GCP project where user cluster is registered +asm_gcp_project_number: "123456789" +asm_gcpsa_path: "asm-meshconfig.json" +asm_user_email: "asm-sa@asm-test.iam.gserviceaccount.com" +``` + +Google Cloud Project IDs and GSA information. + + +### Anthos Config Management + +Optionally define the below variables to be used with the ACM install, upgrade, and uninstall playbooks + +``` +# Optional: ACM +acm_membership: "{{ uc_name }}" # user cluster name +acm_version: "1.12.1" # version in format #.#.# +``` + +Anthos Config Management version and membership details. + +``` +# ACM - GCP related +acm_gcpproject: "acm-test" +acm_gcpsa_path: "acm-gcpsa.json" +``` + +Google Cloud Project IDs and GSA information. + +``` +# ACM - Config Sync +acm_configsync: + # Set to true to install and enable Config Sync + enabled: false + # FORMAT - unstructured, hierarchy + sourceFormat: unstructured + # REPO - git repository URL + syncRepo: "" + syncBranch: "" + # TYPE - none, ssh, cookiefile, token, gcpserviceaccount, gcenode + secretType: gcpserviceaccount + # If TYPE == gcpserviceaccount, then add its email below + gcpServiceAccountEmail: acm-sa@acm-test.iam.gserviceaccount.com + # DIRECTORY - folder in git repository for ACM + policyDir: "" + # PREVENT_DRIFT - true, false + preventDrift: true +acm_root_repo_sshkey: "" # ssh file key if using acm_configsync.secretType: ssh +``` +Configuration for enabling ACM Config Sync, including git repo and authentication details. + +``` +# ACM - PolicyController +acm_policycontroller: + # Set to true to install and enable Policy Controller + enabled: false +``` +Configuration for enabling ACM Policy Controller. + +## Dependencies + +None. + +## Example Playbook +Each user cluster lifecycle operation (ex: installation, upgrade, uninstallation) has its own playbook. + +``` +# Install User Cluster +- name: "[User Cluster] Installation" + hosts: all + gather_facts: false + roles: + - usercluster + - { role: label_membership, when: gcp_labels is defined and (gcp_labels|length > 0) and uc_install|default(true)|bool } + - { role: ais, when: ais_authentication is defined and (ais_authentication|length > 0) and uc_install|default(true)|bool } +``` + +## **License** + +Copyright 2022 Google LLC. This software is provided as-is, without warranty or representation for any use or purpose. +Your use of it is subject to your agreement with Google. diff --git a/tools/anthosvmware-ansible-module/roles/usercluster/defaults/main.yml b/tools/anthosvmware-ansible-module/roles/usercluster/defaults/main.yml new file mode 100644 index 0000000000..d239d45199 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/usercluster/defaults/main.yml @@ -0,0 +1,102 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# role uc default values +uc_install: true +uc_name: "uc1" +component_access_gcpsa: "" +component_access_gcpsa_path: "" +uc_kubeconfig: "{{ uc_name }}-kubeconfig" + +# vSphere/vCenter +# uc_vc_fqdn: "" +# uc_vc_credfile: "credential.yaml" +# uc_vc_credentry: "vCenter" +# uc_vc_username: "" +# uc_vc_password: "" +# uc_vc_datacenter: "" +# uc_vc_datastore: "" +# uc_vc_cluster: "" +# uc_vc_network: "" # VM Network +# uc_vc_folder: "" # optional +# uc_vc_respool: "" # if default resourcePool use /Resources +# uc_vc_cacertpath: "/home/ubuntu/vcenter.pem" # Default location with automatically downloaded cert file name + +# Networking +uc_nw_ipallocmode: "" # dhcp or static +uc_nw_ipfile: "{{ uc_name }}-ip-block.yaml" +uc_nw_gw: "" # gateway +uc_nw_nm: "" # netmask 255.255.255.0 or similar +uc_nw_dns: [""] +uc_nw_ntp: ["ntp.ubuntu.com"] +uc_nw_searchdomains: [""] +uc_nw_servicecidr: "10.96.0.0/20" +uc_nw_podcidr: "" +uc_nw_vc_net: "VM Network" + +# values for the file content of network.ipMode.ipBlockFilePath +uc_ipblock_netmask: "255.255.255.0" +uc_ipblock_gateway: "10.20.0.255" +uc_ipblock_ips: ["10.20.0.51","10.20.0.52","10.20.0.53","10.20.0.54"] + +# Load balancing +uc_lb_kind: MetalLB +uc_lb_vips_cp: "" +uc_lb_vips_ingress: "" +uc_lb_metallb_ips: "" + +# masternode sizing +uc_masternode_cpus: 4 +uc_masternode_mem: 8192 +uc_masternode_replicas: 1 +uc_antiaffinitygroups: false +uc_masternode_datastore: "" + +# nodepools +uc_nodepools_name: "{{ uc_name }}-pool-1" +uc_nodepools_cpus: 4 +uc_nodepools_mem: 8192 +uc_nodepools_replicas: 3 +uc_nodepools_osimagetype: ubuntu_containerd +uc_nodepools_vsphere_tags_category: "" # optional and must be precreated in vSphere +uc_nodepools_vsphere_tags_name: "" # optional and must be precreated in vSphere + +# GCP project IDs +uc_stackdriver_projectid: "" +uc_stackdriver_clusterlocation: "" +uc_stackdriver_enablevpc: false +logging_monitoring_gcpsa_path: "" +uc_stackdriver_disablevsphereresourcemetrics: false +uc_gkeconnect_projectid: "" +connect_register_gcpsa_path: "" +uc_cloudauditlogging_projectid: "" +uc_cloudauditlogging_clusterlocation: "" +audit_logging_gcpsa_path: "" + +uc_autorepair: true + +# Application logging and monitoring +uc_stackdriver_enableapplicationmetrics: false + +# Kubernetes Secrets at-rest encryption +uc_secretsencryption_mode: "GeneratedKey" +uc_secretsencryption_keyversion: 1 + +# optional --skip-validation for gkectl +uc_skipvalidations: "" +uc_verbosity: 8 + +# upgrades +uc_upgrade: false diff --git a/tools/anthosvmware-ansible-module/roles/usercluster/tasks/asserts.yml b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/asserts.yml new file mode 100644 index 0000000000..c7096f4f41 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/asserts.yml @@ -0,0 +1,54 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[uc] Sanity Checks" + ansible.builtin.assert: + that: + - yamldestpath | length >= 6 + - sakeyfolder_base is defined + - uc_name | length >= 3 + - uc_nw_ipallocmode is search("dhcp") or uc_nw_ipallocmode is search("static") + - uc_nw_gw | length >= 7 + - uc_nw_nm | length >= 7 + - uc_nw_dns is defined and uc_nw_dns | type_debug == "list" + - uc_nw_servicecidr | length >= 7 + - uc_nw_podcidr | length >= 7 + - uc_nw_ntp is defined and uc_nw_ntp | type_debug == "list" + - uc_nw_searchdomains is defined and uc_nw_searchdomains | type_debug == "list" + - uc_ipblock_netmask | length >= 7 + - uc_ipblock_gateway | length >= 7 + - uc_lb_kind is search("MetalLB") + - uc_ipblock_ips is defined and uc_ipblock_ips | type_debug == "list" + - uc_lb_vips_cp | length >= 7 + - uc_masternode_cpus | int + - uc_masternode_mem | int + - uc_antiaffinitygroups == uc_antiaffinitygroups|bool + - uc_stackdriver_projectid | length >= 6 + - uc_stackdriver_clusterlocation | length >= 6 + - uc_stackdriver_enablevpc == uc_stackdriver_enablevpc|bool + - logging_monitoring_gcpsa_path | length >= 6 + - uc_stackdriver_disablevsphereresourcemetrics == uc_stackdriver_disablevsphereresourcemetrics|bool + - uc_gkeconnect_projectid | length >= 6 + - connect_register_gcpsa_path | length >= 6 + # - uc_cloudauditlogging_projectid | length >= 6 + # - uc_cloudauditlogging_clusterlocation | length >= 6 + # - audit_logging_gcpsa_path | length >= 6 + - uc_autorepair == uc_autorepair|bool + fail_msg: "Undefined variables." + success_msg: "All required variables are defined." + +- name: "[uc] Note on optional values" + ansible.builtin.debug: + msg: "Optional values uc_vc_folder, uc_nodepools_vsphere_tags_category, uc_nodepools_vsphere_tags_name are not checked. Please double check if required." # noqa 204 diff --git a/tools/anthosvmware-ansible-module/roles/usercluster/tasks/install.yml b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/install.yml new file mode 100644 index 0000000000..3b5ca8f3a0 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/install.yml @@ -0,0 +1,116 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[uc] Setup cluster config" + include_tasks: setup-config.yml + +- name: "[uc] Block - Preflight checks, deploy cluster and delete GCP SA JSON key files" + block: + + - name: "[uc] Preflight check status file" + ansible.builtin.stat: + path: "{{ yamldestpath }}/{{ uc_name }}/{{ uc_name }}-preflight" + register: preflight_file + + - name: "[uc] Preflight check" + async: 3600 + poll: 5 + ansible.builtin.command: + chdir: "{{ yamldestpath }}/{{ uc_name }}" + argv: "{{ _argv }}" + vars: + _req_args: + - gkectl + - check-config + - --config + - "{{ uc_name }}-cluster.yaml" + - --kubeconfig + - ../kubeconfig + - --alsologtostderr + - -v + - "{{ uc_verbosity }}" + _skips: "{{ uc_skipvalidations if uc_skipvalidations | type_debug == 'list' else (uc_skipvalidations.split(' ')) }}" + _argv: "{{ _req_args + _skips }}" + register: check_config_result + tags: preflight + when: not preflight_file.stat.exists + + - name: "[uc] Preflight results" + ansible.builtin.file: + path: "{{ yamldestpath }}/{{ uc_name }}/{{ uc_name }}-preflight" + state: touch + mode: 0600 + when: check_config_result.rc | default('') == 0 + tags: preflight + + # async: 5400 poll: 5 results in task polling for approximately 5015s + - name: "[uc] Create user cluster" + async: 5400 + poll: 5 + ansible.builtin.command: # noqa 305 no-changed-when + chdir: "{{ yamldestpath }}/{{ uc_name }}" + argv: "{{ _argv }}" + creates: "~/{{ uc_name }}-kubeconfig" + vars: + _req_args: + - gkectl + - create + - cluster + - --config + - "{{ uc_name }}-cluster.yaml" + - --kubeconfig + - ../kubeconfig + - --alsologtostderr + - -v + - "{{ uc_verbosity }}" + _skips: "{{ uc_skipvalidations if uc_skipvalidations | type_debug == 'list' else (uc_skipvalidations.split(' ')) }}" + _argv: "{{ _req_args + _skips }}" + + - name: "[uc] Template patch for stackdriver configuration" + ansible.builtin.template: + src: patch-stackdriver.yaml.j2 + dest: "{{ yamldestpath }}/{{ uc_name }}/patch-stackdriver.yaml" + mode: 0600 + when: + - uc_stackdriver_enableapplicationmetrics|default(false)|bool + + - name: "[uc] Patch stackdriver with enableStackdriverForApplications flag" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ yamldestpath }}/{{ uc_name }}-kubeconfig" + - patch + - stackdriver + - stackdriver + - -n + - kube-system + - --patch-file + - "{{ yamldestpath }}/{{ uc_name }}/patch-stackdriver.yaml" + - --type + - merge + when: + - uc_stackdriver_enableapplicationmetrics|default(false)|bool + + environment: + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" + http_proxy: "{{ glb_proxyurl }}" + https_proxy: "{{ glb_proxyurl }}" + no_proxy: "{{ glb_noproxy }}" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup diff --git a/tools/anthosvmware-ansible-module/roles/usercluster/tasks/main.yml b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/main.yml new file mode 100644 index 0000000000..55ec3fba33 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/main.yml @@ -0,0 +1,67 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[uc] Clean up old SSH host keys from incomplete runs" + ansible.builtin.file: + path: "/home/ubuntu/.ssh/known_hosts" + state: absent + +- name: "[uc] Include tasks - asserts.yml" + include_tasks: asserts.yml + +- name: "[uc] Check for existing cluster kubeconfig" + ansible.builtin.stat: + path: "~/{{ uc_kubeconfig }}" + register: uc_kubeconfig_file + +- name: "[uc] Check if existing api server is up" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ uc_kubeconfig }}" + - cluster-info + when: + - uc_kubeconfig_file.stat.exists + - uc_install|default(false)|bool + changed_when: false + +- name: "[uc] Include tasks - install.yml" + include_tasks: install.yml + when: + - uc_install|default(false)|bool + - not uc_kubeconfig_file.stat.exists + +# updating cluster settings +- name: "[uc] Include tasks - update.yml" + include_tasks: update.yml + when: + - uc_install|default(false)|bool + - uc_kubeconfig_file.stat.exists + - not uc_upgrade|default(false)|bool + +# version updates and upgrades +- name: "[uc] Include tasks - upgrade.yml" + include_tasks: upgrade.yml + when: + - uc_install|default(false)|bool + - uc_kubeconfig_file.stat.exists + - uc_upgrade|default(false)|bool + +- name: "[uc] Include tasks - uninstall.yml" + include_tasks: uninstall.yml + when: + - not uc_install|default(false)|bool + - uc_kubeconfig_file.stat.exists diff --git a/tools/anthosvmware-ansible-module/roles/usercluster/tasks/setup-config.yml b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/setup-config.yml new file mode 100644 index 0000000000..05a945c73b --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/setup-config.yml @@ -0,0 +1,36 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "Include role for GCP SA JSON key files" + ansible.builtin.include_role: + name: copy_credentials + vars: + is_user_cluster: true + +- name: "[uc] Create folder on Admin Workstation for YAML files" + ansible.builtin.file: + path: "/home/ubuntu/{{ uc_name }}" + state: directory + mode: 0700 + +- name: "[uc] Templating YAML files" + ansible.builtin.template: + src: user-cluster-{{ glb_major_version }}.yaml.j2 + dest: "{{ yamldestpath }}/{{ uc_name }}/{{ uc_name }}-cluster.yaml" + mode: 0600 + register: uc_template + +- name: "[uc] Templating YAML files - if using static IPs" + include_tasks: static-ips.yml + when: uc_nw_ipallocmode == "static" diff --git a/tools/anthosvmware-ansible-module/roles/usercluster/tasks/static-ips.yml b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/static-ips.yml new file mode 100644 index 0000000000..572a5263a1 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/static-ips.yml @@ -0,0 +1,20 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +- name: "[uc] Templating YAML files - IP block" + ansible.builtin.template: + src: cluster-ip-block.yaml.j2 + dest: "{{ yamldestpath }}/{{ uc_name }}/{{ uc_name }}-ip-block.yaml" + mode: 0600 \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/usercluster/tasks/uninstall.yml b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/uninstall.yml new file mode 100644 index 0000000000..b6ff76939c --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/uninstall.yml @@ -0,0 +1,58 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "Include role for GCP SA JSON key files" + ansible.builtin.include_role: + name: copy_credentials + vars: + is_user_cluster: true + +- name: "[uc] Block - Delete cluster and GCP SA JSON key files" + block: + + # async: 3600 poll: 5 results in task polling for approximately 5015s + - name: "[uc] Delete user cluster" + async: 3600 + poll: 5 + ansible.builtin.command: # noqa 305 no-changed-when + chdir: "{{ yamldestpath }}/{{ uc_name }}" + argv: + - gkectl + - delete + - cluster + - --cluster + - "{{ uc_name }}" + - --kubeconfig + - ../kubeconfig + - --alsologtostderr + - -v + - "{{ uc_verbosity }}" + register: uc_delete_status + + - name: "[uc] Delete User Cluster Kubeconfig" + ansible.builtin.file: + path: "~/{{ uc_name }}-kubeconfig" + state: absent + when: uc_delete_status.rc | default('') == 0 + + environment: + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" + http_proxy: "{{ glb_proxyurl }}" + https_proxy: "{{ glb_proxyurl }}" + no_proxy: "{{ glb_noproxy }}" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup diff --git a/tools/anthosvmware-ansible-module/roles/usercluster/tasks/update.yml b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/update.yml new file mode 100644 index 0000000000..32dd604f7b --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/update.yml @@ -0,0 +1,79 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[uc] Setup cluster config" + include_tasks: setup-config.yml + +- name: "[uc] Block - Update cluster, then cleanup sensitive files" + block: + + # async: 3600 poll: 5 results in task polling for approximately 5015s + - name: "[uc] Update user cluster" + async: 3600 + poll: 5 + ansible.builtin.command: # noqa 305 no-changed-when + chdir: "{{ yamldestpath }}/{{ uc_name }}" + argv: "{{ _argv }}" + vars: + _req_args: + - gkectl + - update + - cluster + - --config + - "{{ uc_name }}-cluster.yaml" + - --kubeconfig + - ../kubeconfig + - --alsologtostderr + - --yes + - -v + - "{{ uc_verbosity }}" + _skips: "{{ uc_skipvalidations if uc_skipvalidations | type_debug == 'list' else (uc_skipvalidations.split(' ')) }}" + _argv: "{{ _req_args + _skips }}" + + - name: "[uc] Template patch for stackdriver configuration" + ansible.builtin.template: + src: patch-stackdriver.yaml.j2 + dest: "{{ yamldestpath }}/{{ uc_name }}/patch-stackdriver.yaml" + mode: 0600 + when: + - uc_stackdriver_enableapplicationmetrics|default(false)|bool + + - name: "[uc] Patch stackdriver with enableStackdriverForApplications flag" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - "{{ yamldestpath }}/{{ uc_name }}-kubeconfig" + - patch + - stackdriver + - stackdriver + - -n + - kube-system + - --patch-file + - "{{ yamldestpath }}/{{ uc_name }}/patch-stackdriver.yaml" + - --type + - merge + when: + - uc_stackdriver_enableapplicationmetrics|default(false)|bool + + environment: + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" + http_proxy: "{{ glb_proxyurl }}" + https_proxy: "{{ glb_proxyurl }}" + no_proxy: "{{ glb_noproxy }}" + + always: + - name: "[uc] Delete sensitive files" + ansible.builtin.include_role: + name: cleanup \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/usercluster/tasks/upgrade.yml b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/upgrade.yml new file mode 100644 index 0000000000..20f2472bea --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/usercluster/tasks/upgrade.yml @@ -0,0 +1,109 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: "[uc] Clean up old SSH host keys from incomplete runs" + ansible.builtin.file: + path: "/home/ubuntu/.ssh/known_hosts" + state: absent + +- name: "[uc] Setup cluster config" + include_tasks: setup-config.yml + +- name: "[uc] Block - Upload new bundle, upgrade cluster and delete GCP SA JSON key files" + block: + - name: "[uc] Get current user cluster version" + ansible.builtin.command: + argv: + - kubectl + - --kubeconfig + - kubeconfig + - get + - onpremusercluster + - "{{ uc_name }}" + - -n + - "{{ uc_name }}-gke-onprem-mgmt" + - "-o=jsonpath={.spec.gkeOnPremVersion}" + register: live_version + + - name: "[uc] Verify inventory version not equal to current version" + ansible.builtin.assert: + that: glb_anthos_version != live_version.stdout + fail_msg: "Requested User cluster version {{ glb_anthos_version }} equals current live version {{ live_version.stdout }}." + success_msg: "Requested User cluster version different from live." + + - name: "[uc] Get gkectl version from Admin workstation" + ansible.builtin.shell: "gkectl version | head -n1 | awk '{print $2;}'" + register: gkectl_version + + - name: "[uc] Verify inventory version matches Admin workstation version" + ansible.builtin.assert: + that: glb_anthos_version == gkectl_version.stdout + fail_msg: "Requested User cluster version {{ glb_anthos_version }} does not match Admin workstation version {{ gkectl_version.stdout }}." + success_msg: "Requested User cluster version matched Admin workstation version" + + - name: "[uc] Replace cluster version in its config YAML file" + ansible.builtin.lineinfile: + path: "{{ uc_name }}/{{ uc_name }}-cluster.yaml" + regexp: "^gkeOnPremVersion:.*" + line: "gkeOnPremVersion: {{ glb_anthos_version }}" + + - name: "[uc] Upload new bundle to vSphere" + ansible.builtin.command: + chdir: "{{ yamldestpath }}/{{ uc_name }}" + argv: "{{ _argv }}" + vars: + _req_args: + - gkectl + - prepare + - --bundle-path + - "/var/lib/gke/bundles/gke-onprem-vsphere-{{ glb_anthos_version }}-full.tgz" + - --kubeconfig + - ../kubeconfig + - --alsologtostderr + - -v + - "{{ uc_verbosity }}" + _skips: "{{ uc_skipvalidations if uc_skipvalidations | type_debug == 'list' else (uc_skipvalidations.split(' ')) }}" + _argv: "{{ _req_args + _skips }}" + + - name: "[uc] Upgrade user cluster" + async: 5400 + poll: 5 + ansible.builtin.command: # noqa 305 no-changed-when + chdir: "{{ yamldestpath }}/{{ uc_name }}" + argv: "{{ _argv }}" + vars: + _req_args: + - gkectl + - upgrade + - cluster + - --config + - "{{ uc_name }}-cluster.yaml" + - --kubeconfig + - ../kubeconfig + - --alsologtostderr + - -v + - "{{ uc_verbosity }}" + _skips: "{{ uc_skipvalidations if uc_skipvalidations | type_debug == 'list' else (uc_skipvalidations.split(' ')) }}" + _argv: "{{ _req_args + _skips }}" + + environment: + PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" + http_proxy: "{{ glb_proxyurl }}" + https_proxy: "{{ glb_proxyurl }}" + no_proxy: "{{ glb_noproxy }}" + + always: + - name: "Include role to delete sensitive files" + ansible.builtin.include_role: + name: cleanup diff --git a/tools/anthosvmware-ansible-module/roles/usercluster/templates/cluster-ip-block.yaml.j2 b/tools/anthosvmware-ansible-module/roles/usercluster/templates/cluster-ip-block.yaml.j2 new file mode 100644 index 0000000000..404c75b911 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/usercluster/templates/cluster-ip-block.yaml.j2 @@ -0,0 +1,22 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +blocks: + - netmask: {{ uc_nw_nm }} + gateway: {{ uc_nw_gw }} + ips: +{% for ip in uc_ipblock_ips %} + - ip: {{ ip }} +{% endfor %} diff --git a/tools/anthosvmware-ansible-module/roles/usercluster/templates/patch-stackdriver.yaml.j2 b/tools/anthosvmware-ansible-module/roles/usercluster/templates/patch-stackdriver.yaml.j2 new file mode 100644 index 0000000000..f7d98556be --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/usercluster/templates/patch-stackdriver.yaml.j2 @@ -0,0 +1,2 @@ +spec: + enableStackdriverForApplications: true \ No newline at end of file diff --git a/tools/anthosvmware-ansible-module/roles/usercluster/templates/user-cluster-1.11.yaml.j2 b/tools/anthosvmware-ansible-module/roles/usercluster/templates/user-cluster-1.11.yaml.j2 new file mode 100644 index 0000000000..2a35603f06 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/usercluster/templates/user-cluster-1.11.yaml.j2 @@ -0,0 +1,249 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +apiVersion: v1 +kind: UserCluster +# (Required) A unique name for this cluster +name: "{{ uc_name }}" +# (Required) GKE on-prem version (example: 1.3.0-gke.16) +gkeOnPremVersion: {{ glb_anthos_version }} +# # (Optional) vCenter configuration (default: inherit from the admin cluster) +vCenter: +{% if (uc_vc_datacenter is defined) and (uc_vc_datacenter | length > 0) %} + datacenter: "{{ uc_vc_datacenter }}" +{% endif %} +{% if (uc_vc_cluster is defined) and (uc_vc_cluster | length > 0) %} + cluster: "{{ uc_vc_cluster }}" +{% endif %} +{% if (uc_vc_respool is defined) and (uc_vc_respool | length > 0) %} + resourcePool: "{{ uc_vc_respool }}" +{% endif %} +{% if (uc_vc_datastore is defined) and (uc_vc_datastore | length > 0) %} + datastore: "{{ uc_vc_datastore }}" +{% endif %} +{% if (uc_vc_cacertpath is defined) and (uc_vc_cacertpath | length > 0) %} + caCertPath: "{{ uc_vc_cacertpath }}" +{% endif %} +{% if (uc_vc_fqdn is defined) and (uc_vc_fqdn | length > 0) and (uc_vc_username is defined) and (uc_vc_username | length > 0) and (uc_vc_password is defined) and (uc_vc_password | length > 0) %} + credentials: + fileRef: + path: "{{ uc_vc_credfile }}" + entry: "{{ uc_vc_credentry }}" +{% endif %} +{% if (uc_vc_folder is defined) and (uc_vc_folder | length > 0) %} + folder: "{{ uc_vc_folder }}" +{% endif %} +# (Required) Network configuration; vCenter section is optional and inherits from +# the admin cluster if not specified +network: + # # (Optional) This section overrides ipBlockFile values. Use with ipType "static" mode. + hostConfig: + # # List of DNS servers + dnsServers: +{% for ip in uc_nw_dns %} + - "{{ ip }}" +{% endfor %} + # # List of NTP servers + ntpServers: +{% for ntp in uc_nw_ntp %} + - "{{ ntp }}" +{% endfor %} + searchDomainsForDNS: +{% for domain in uc_nw_searchdomains %} + - "{{ domain }}" +{% endfor %} + ipMode: + # (Required) Define what IP mode to use ("dhcp" "static" or "none"(multinic only)) + type: {{ uc_nw_ipallocmode }} + # # (Required when using "static" mode) The absolute or relative path to the yaml file + # # to use for static IP allocation. Hostconfig part will be overwritten by network.hostconfig + # # if specified + ipBlockFilePath: "{{ uc_nw_ipfile }}" + # (Required) The Kubernetes service CIDR range for the cluster. Must not overlap + # with the pod CIDR range + serviceCIDR: {{ uc_nw_servicecidr }} + # (Required) The Kubernetes pod CIDR range for the cluster. Must not overlap with + # the service CIDR range + podCIDR: {{ uc_nw_podcidr }} + vCenter: + # vSphere network name + networkName: {{ uc_nw_vc_net }} + # # (Optional) List of additional node network interfaces feature enabled by multipleNetworkInterfaces + # additionalNodeInterfaces: + # # vSphere network name + # - networkName: "" + # # (Required) Define what IP mode to use ("dhcp" "static" or "none"(multinic only)) + # type: dhcp + # # # (Required when using "static" mode) The absolute or relative path to the yaml file + # # # to use for static IP allocation. Hostconfig part will be overwritten by network.hostconfig + # # # if specified + # # ipBlockFilePath: "" +# (Required) Load balancer configuration +loadBalancer: + # (Required) The VIPs to use for load balancing + vips: + # Used to connect to the Kubernetes API + controlPlaneVIP: "{{ uc_lb_vips_cp }}" + # Shared by all services for ingress traffic + ingressVIP: "{{ uc_lb_vips_ingress }}" + # (Required) Which load balancer to use "F5BigIP" "Seesaw" "ManualLB" or "MetalLB". + # The automation script is configured to use MetalLB. + # Uncomment the corresponding field below to provide the detailed spec + kind: {{ uc_lb_kind }} + # # (Required when using "ManualLB" kind) Specify pre-defined nodeports + # manualLB: + # # NodePort for ingress service's http (only needed for user cluster) + # ingressHTTPNodePort: 30243 + # # NodePort for ingress service's https (only needed for user cluster) + # ingressHTTPSNodePort: 30879 + # # NodePort for konnectivity server service (only needed for user cluster) + # konnectivityServerNodePort: 30563 + # # NodePort for control plane service + # controlPlaneNodePort: 30562 + # # NodePort for addon service (only needed for admin cluster) + # addonsNodePort: 0 + # # (Required when using "F5BigIP" kind) Specify the already-existing partition and + # # credentials + # f5BigIP: + # address: "" + # credentials: + # # reference to external credentials file + # fileRef: + # # read credentials from this file + # path: credential.yaml + # # entry in the credential file + # entry: f5BigIP + # partition: "" + # # # (Optional) Specify a pool name if using SNAT + # # snatPoolName: "" +{% if (uc_lb_kind is defined) and ('MetalLB' == uc_lb_kind) %} + # # (Required when using "MetalLB" kind in user clusters) Specify the MetalLB configs + metalLB: + # # (Required) A list of non-overlapping IP pools used by load balancer typed services. + # # Must include ingressVIP of the cluster. + addressPools: + # # (Required) Name of the address pool + - name: "lbpool1" + # # (Required) The addresses that are part of this pool. Each address must be either + # # in the CIDR form (1.2.3.0/24) or range form (1.2.3.1-1.2.3.5). + addresses: +{% for addr in uc_lb_metallb_ips %} + - "{{ addr }}" +{% endfor %} + # # # (Optional) Avoid using IPs ending in .0 or .255. This avoids buggy consumer devices + # # # mistakenly dropping IPv4 traffic for those special IP addresses (default: false) + avoidBuggyIPs: true + # # # (Optional) Prevent IP addresses to be automatically assigned from this pool (default: + # # # false) + manualAssign: false +{% endif %} +# # (Optional) Enable dataplane v2 +enableDataplaneV2: {{ uc_dataplanev2 }} +# # (Optional) Enable support for multiple networking interfaces +# multipleNetworkInterfaces: false +# # (Optional) Enable advanced dataplane v2 networking features such as Egress NAT Gateway +# # and it requires enableDataplaneV2 to be set +# advancedNetworking: false +# enableWindowsDataplaneV2: false +# # (Optional) Storage specification for the cluster +# storage: +# # Whether to disable vSphere CSI components deployment. The feature is enabled by +# # default. +# vSphereCSIDisabled: false +# (Optional) User cluster master nodes must have either 1 or 3 replicas (default: +# 4 CPUs; 8192 MB memory; 1 replica) +masterNode: + cpus: {{ uc_masternode_cpus }} + memoryMB: {{ uc_masternode_mem }} + # How many machines of this type to deploy + replicas: {{ uc_masternode_replicas }} + # # (Optional/Preview) Enable auto resizing on master + # autoResize: + # # Whether to enable auto resize for master. Defaults to false. + # enabled: false +{% if (uc_masternode_datastore is defined) and (uc_masternode_datastore | length > 0) %} + vsphere: + datastore: "{{ uc_masternode_datastore }}" +{% endif %} +# (Required) List of node pools. The total un-tainted replicas across all node pools +# must be greater than or equal to 3 +nodePools: +{{ uc_nodepools | to_nice_yaml(indent=0) }} +# Spread nodes across at least three physical hosts (requires at least three hosts) +antiAffinityGroups: + # Set to false to disable DRS rule creation + enabled: {{ uc_antiaffinitygroups }} +# # (Optional/Preview) Track user cluster VMs with vSphere tags +# enableVMTracking: false +# # (Optional) Configure additional authentication. +# authentication: +# # (Optional) Provide an additional serving certificate for the API server +# sni: +# certPath: "" +# keyPath: "" +# (Required) Specify which GCP project to connect your GKE clusters to +gkeConnect: + projectID: "{{ uc_gkeconnect_projectid }}" + # The absolute or relative path to the key file for a GCP service account used to + # register the cluster + registerServiceAccountKeyPath: "{{ job_sakeyfolder }}/{{ connect_register_gcpsa_path}}" +# (Optional) Specify which GCP project to connect your logs and metrics to +stackdriver: + projectID: "{{ uc_stackdriver_projectid }}" + # A GCP region where you would like to store logs and metrics for this cluster. + clusterLocation: "{{ uc_stackdriver_clusterlocation }}" + enableVPC: {{ uc_stackdriver_enablevpc }} + # The absolute or relative path to the key file for a GCP service account used to + # send logs and metrics from the cluster + serviceAccountKeyPath: "{{ job_sakeyfolder }}/{{ logging_monitoring_gcpsa_path }}" + # (Optional) Disable vsphere resource metrics collection from vcenter. False by + # default + disableVsphereResourceMetrics: false +# # (Optional/Alpha) Configure the GKE usage metering feature +# usageMetering: +# bigQueryProjectID: "" +# # The ID of the BigQuery Dataset in which the usage metering data will be stored +# bigQueryDatasetID: "" +# # The absolute or relative path to the key file for a GCP service account used by +# # gke-usage-metering to report to BigQuery +# bigQueryServiceAccountKeyPath: "" +# # Whether or not to enable consumption-based metering +# enableConsumptionMetering: false +# # (Optional) Configure kubernetes apiserver audit logging +{% if (uc_cloudauditlogging_projectid is defined) and (uc_cloudauditlogging_projectid|length > 0) %} +cloudAuditLogging: + projectID: "{{ uc_cloudauditlogging_projectid }}" + # A GCP region where you would like to store audit logs for this cluster. + clusterLocation: "{{ uc_cloudauditlogging_clusterlocation }}" + # The absolute or relative path to the key file for a GCP service account used to + # send audit logs from the cluster + serviceAccountKeyPath: "{{ job_sakeyfolder }}/{{ audit_logging_gcpsa_path }}" +{% endif %} +# Enable auto repair for the cluster +autoRepair: + # Whether to enable auto repair feature. Set false to disable. + enabled: {{ uc_autorepair }} +# # Encrypt Kubernetes secrets at rest +{% if (uc_secretsencryption_mode is defined) and ('GeneratedKey' == uc_secretsencryption_mode) %} +secretsEncryption: + # Secrets Encryption Mode. Possible values are: GeneratedKey + mode: "{{ uc_secretsencryption_mode }}" + # GeneratedKey Secrets Encryption config + generatedKey: + # # key version + keyVersion: {{ uc_secretsencryption_keyversion }} + # # disable secrets encryption + # disabled: false +{% endif %} diff --git a/tools/anthosvmware-ansible-module/roles/usercluster/templates/user-cluster-1.12.yaml.j2 b/tools/anthosvmware-ansible-module/roles/usercluster/templates/user-cluster-1.12.yaml.j2 new file mode 100644 index 0000000000..2a35603f06 --- /dev/null +++ b/tools/anthosvmware-ansible-module/roles/usercluster/templates/user-cluster-1.12.yaml.j2 @@ -0,0 +1,249 @@ +{# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#} +apiVersion: v1 +kind: UserCluster +# (Required) A unique name for this cluster +name: "{{ uc_name }}" +# (Required) GKE on-prem version (example: 1.3.0-gke.16) +gkeOnPremVersion: {{ glb_anthos_version }} +# # (Optional) vCenter configuration (default: inherit from the admin cluster) +vCenter: +{% if (uc_vc_datacenter is defined) and (uc_vc_datacenter | length > 0) %} + datacenter: "{{ uc_vc_datacenter }}" +{% endif %} +{% if (uc_vc_cluster is defined) and (uc_vc_cluster | length > 0) %} + cluster: "{{ uc_vc_cluster }}" +{% endif %} +{% if (uc_vc_respool is defined) and (uc_vc_respool | length > 0) %} + resourcePool: "{{ uc_vc_respool }}" +{% endif %} +{% if (uc_vc_datastore is defined) and (uc_vc_datastore | length > 0) %} + datastore: "{{ uc_vc_datastore }}" +{% endif %} +{% if (uc_vc_cacertpath is defined) and (uc_vc_cacertpath | length > 0) %} + caCertPath: "{{ uc_vc_cacertpath }}" +{% endif %} +{% if (uc_vc_fqdn is defined) and (uc_vc_fqdn | length > 0) and (uc_vc_username is defined) and (uc_vc_username | length > 0) and (uc_vc_password is defined) and (uc_vc_password | length > 0) %} + credentials: + fileRef: + path: "{{ uc_vc_credfile }}" + entry: "{{ uc_vc_credentry }}" +{% endif %} +{% if (uc_vc_folder is defined) and (uc_vc_folder | length > 0) %} + folder: "{{ uc_vc_folder }}" +{% endif %} +# (Required) Network configuration; vCenter section is optional and inherits from +# the admin cluster if not specified +network: + # # (Optional) This section overrides ipBlockFile values. Use with ipType "static" mode. + hostConfig: + # # List of DNS servers + dnsServers: +{% for ip in uc_nw_dns %} + - "{{ ip }}" +{% endfor %} + # # List of NTP servers + ntpServers: +{% for ntp in uc_nw_ntp %} + - "{{ ntp }}" +{% endfor %} + searchDomainsForDNS: +{% for domain in uc_nw_searchdomains %} + - "{{ domain }}" +{% endfor %} + ipMode: + # (Required) Define what IP mode to use ("dhcp" "static" or "none"(multinic only)) + type: {{ uc_nw_ipallocmode }} + # # (Required when using "static" mode) The absolute or relative path to the yaml file + # # to use for static IP allocation. Hostconfig part will be overwritten by network.hostconfig + # # if specified + ipBlockFilePath: "{{ uc_nw_ipfile }}" + # (Required) The Kubernetes service CIDR range for the cluster. Must not overlap + # with the pod CIDR range + serviceCIDR: {{ uc_nw_servicecidr }} + # (Required) The Kubernetes pod CIDR range for the cluster. Must not overlap with + # the service CIDR range + podCIDR: {{ uc_nw_podcidr }} + vCenter: + # vSphere network name + networkName: {{ uc_nw_vc_net }} + # # (Optional) List of additional node network interfaces feature enabled by multipleNetworkInterfaces + # additionalNodeInterfaces: + # # vSphere network name + # - networkName: "" + # # (Required) Define what IP mode to use ("dhcp" "static" or "none"(multinic only)) + # type: dhcp + # # # (Required when using "static" mode) The absolute or relative path to the yaml file + # # # to use for static IP allocation. Hostconfig part will be overwritten by network.hostconfig + # # # if specified + # # ipBlockFilePath: "" +# (Required) Load balancer configuration +loadBalancer: + # (Required) The VIPs to use for load balancing + vips: + # Used to connect to the Kubernetes API + controlPlaneVIP: "{{ uc_lb_vips_cp }}" + # Shared by all services for ingress traffic + ingressVIP: "{{ uc_lb_vips_ingress }}" + # (Required) Which load balancer to use "F5BigIP" "Seesaw" "ManualLB" or "MetalLB". + # The automation script is configured to use MetalLB. + # Uncomment the corresponding field below to provide the detailed spec + kind: {{ uc_lb_kind }} + # # (Required when using "ManualLB" kind) Specify pre-defined nodeports + # manualLB: + # # NodePort for ingress service's http (only needed for user cluster) + # ingressHTTPNodePort: 30243 + # # NodePort for ingress service's https (only needed for user cluster) + # ingressHTTPSNodePort: 30879 + # # NodePort for konnectivity server service (only needed for user cluster) + # konnectivityServerNodePort: 30563 + # # NodePort for control plane service + # controlPlaneNodePort: 30562 + # # NodePort for addon service (only needed for admin cluster) + # addonsNodePort: 0 + # # (Required when using "F5BigIP" kind) Specify the already-existing partition and + # # credentials + # f5BigIP: + # address: "" + # credentials: + # # reference to external credentials file + # fileRef: + # # read credentials from this file + # path: credential.yaml + # # entry in the credential file + # entry: f5BigIP + # partition: "" + # # # (Optional) Specify a pool name if using SNAT + # # snatPoolName: "" +{% if (uc_lb_kind is defined) and ('MetalLB' == uc_lb_kind) %} + # # (Required when using "MetalLB" kind in user clusters) Specify the MetalLB configs + metalLB: + # # (Required) A list of non-overlapping IP pools used by load balancer typed services. + # # Must include ingressVIP of the cluster. + addressPools: + # # (Required) Name of the address pool + - name: "lbpool1" + # # (Required) The addresses that are part of this pool. Each address must be either + # # in the CIDR form (1.2.3.0/24) or range form (1.2.3.1-1.2.3.5). + addresses: +{% for addr in uc_lb_metallb_ips %} + - "{{ addr }}" +{% endfor %} + # # # (Optional) Avoid using IPs ending in .0 or .255. This avoids buggy consumer devices + # # # mistakenly dropping IPv4 traffic for those special IP addresses (default: false) + avoidBuggyIPs: true + # # # (Optional) Prevent IP addresses to be automatically assigned from this pool (default: + # # # false) + manualAssign: false +{% endif %} +# # (Optional) Enable dataplane v2 +enableDataplaneV2: {{ uc_dataplanev2 }} +# # (Optional) Enable support for multiple networking interfaces +# multipleNetworkInterfaces: false +# # (Optional) Enable advanced dataplane v2 networking features such as Egress NAT Gateway +# # and it requires enableDataplaneV2 to be set +# advancedNetworking: false +# enableWindowsDataplaneV2: false +# # (Optional) Storage specification for the cluster +# storage: +# # Whether to disable vSphere CSI components deployment. The feature is enabled by +# # default. +# vSphereCSIDisabled: false +# (Optional) User cluster master nodes must have either 1 or 3 replicas (default: +# 4 CPUs; 8192 MB memory; 1 replica) +masterNode: + cpus: {{ uc_masternode_cpus }} + memoryMB: {{ uc_masternode_mem }} + # How many machines of this type to deploy + replicas: {{ uc_masternode_replicas }} + # # (Optional/Preview) Enable auto resizing on master + # autoResize: + # # Whether to enable auto resize for master. Defaults to false. + # enabled: false +{% if (uc_masternode_datastore is defined) and (uc_masternode_datastore | length > 0) %} + vsphere: + datastore: "{{ uc_masternode_datastore }}" +{% endif %} +# (Required) List of node pools. The total un-tainted replicas across all node pools +# must be greater than or equal to 3 +nodePools: +{{ uc_nodepools | to_nice_yaml(indent=0) }} +# Spread nodes across at least three physical hosts (requires at least three hosts) +antiAffinityGroups: + # Set to false to disable DRS rule creation + enabled: {{ uc_antiaffinitygroups }} +# # (Optional/Preview) Track user cluster VMs with vSphere tags +# enableVMTracking: false +# # (Optional) Configure additional authentication. +# authentication: +# # (Optional) Provide an additional serving certificate for the API server +# sni: +# certPath: "" +# keyPath: "" +# (Required) Specify which GCP project to connect your GKE clusters to +gkeConnect: + projectID: "{{ uc_gkeconnect_projectid }}" + # The absolute or relative path to the key file for a GCP service account used to + # register the cluster + registerServiceAccountKeyPath: "{{ job_sakeyfolder }}/{{ connect_register_gcpsa_path}}" +# (Optional) Specify which GCP project to connect your logs and metrics to +stackdriver: + projectID: "{{ uc_stackdriver_projectid }}" + # A GCP region where you would like to store logs and metrics for this cluster. + clusterLocation: "{{ uc_stackdriver_clusterlocation }}" + enableVPC: {{ uc_stackdriver_enablevpc }} + # The absolute or relative path to the key file for a GCP service account used to + # send logs and metrics from the cluster + serviceAccountKeyPath: "{{ job_sakeyfolder }}/{{ logging_monitoring_gcpsa_path }}" + # (Optional) Disable vsphere resource metrics collection from vcenter. False by + # default + disableVsphereResourceMetrics: false +# # (Optional/Alpha) Configure the GKE usage metering feature +# usageMetering: +# bigQueryProjectID: "" +# # The ID of the BigQuery Dataset in which the usage metering data will be stored +# bigQueryDatasetID: "" +# # The absolute or relative path to the key file for a GCP service account used by +# # gke-usage-metering to report to BigQuery +# bigQueryServiceAccountKeyPath: "" +# # Whether or not to enable consumption-based metering +# enableConsumptionMetering: false +# # (Optional) Configure kubernetes apiserver audit logging +{% if (uc_cloudauditlogging_projectid is defined) and (uc_cloudauditlogging_projectid|length > 0) %} +cloudAuditLogging: + projectID: "{{ uc_cloudauditlogging_projectid }}" + # A GCP region where you would like to store audit logs for this cluster. + clusterLocation: "{{ uc_cloudauditlogging_clusterlocation }}" + # The absolute or relative path to the key file for a GCP service account used to + # send audit logs from the cluster + serviceAccountKeyPath: "{{ job_sakeyfolder }}/{{ audit_logging_gcpsa_path }}" +{% endif %} +# Enable auto repair for the cluster +autoRepair: + # Whether to enable auto repair feature. Set false to disable. + enabled: {{ uc_autorepair }} +# # Encrypt Kubernetes secrets at rest +{% if (uc_secretsencryption_mode is defined) and ('GeneratedKey' == uc_secretsencryption_mode) %} +secretsEncryption: + # Secrets Encryption Mode. Possible values are: GeneratedKey + mode: "{{ uc_secretsencryption_mode }}" + # GeneratedKey Secrets Encryption config + generatedKey: + # # key version + keyVersion: {{ uc_secretsencryption_keyversion }} + # # disable secrets encryption + # disabled: false +{% endif %} diff --git a/tools/anthosvmware-ansible-module/scripts/art-manual-push-test.bash b/tools/anthosvmware-ansible-module/scripts/art-manual-push-test.bash new file mode 100644 index 0000000000..f2b1b86265 --- /dev/null +++ b/tools/anthosvmware-ansible-module/scripts/art-manual-push-test.bash @@ -0,0 +1,120 @@ +#!/bin/bash + +ART_USER="" +ART_PASSWD="" +ART_URL="" + +IMAGES="gcr.io/gke-on-prem-release/kube-controller-manager-amd64:v1.22.8-gke.200 \ + gcr.io/gke-on-prem-release/cluster-autoscaler-amd64:v1.21.0-gke.5 \ + gcr.io/gke-on-prem-release/cilium/cilium:v1.11.1-anthos1.11-gke3.4.7 \ + gcr.io/gke-on-prem-release/onprem-user-cluster-controller:1.11.1-gke.53 \ + gcr.io/gke-on-prem-release/node-firewall-controller:v0.0.20 \ + gcr.io/gke-on-prem-release/antrea-amd64:v1.2.0-gke.18 \ + gcr.io/gke-on-prem-release/k8s-gmsa-webhook:release-0.3.0-gke.0 \ + gcr.io/gke-on-prem-release/proxy-agent-amd64:v0.0.25-gke.0 \ + gcr.io/gke-on-prem-release/k8s-dns-kube-dns-amd64:1.22.2-gke.0 \ + gcr.io/gke-on-prem-release/snapshot-controller:v4.2.1-gke.0 \ + gcr.io/gke-on-prem-release/calico/cni:v3.19.1-gke.0 \ + gcr.io/gke-on-prem-release/clusterdns-controller:gke_clusterdns_controller_20210908.00_p0 \ + gcr.io/gke-on-prem-release/metallb/speaker:v0.9.6-gke.0 \ + gcr.io/gke-on-prem-release/windows-webhook:v0.1.0-gke.6 \ + gcr.io/gke-on-prem-release/ang/ang-daemon:1.0.7-gke.1 \ + gcr.io/gke-on-prem-release/whereabouts:v0.3.1-gke.9 \ + gcr.io/gke-on-prem-release/kube-proxy-amd64:v1.22.8-gke.200 \ + gcr.io/gke-on-prem-release/csi-node-driver-registrar:v2.3.0-gke.1 \ + gcr.io/gke-on-prem-release/cluster-metrics-webhook:v0.1.0-gke.5 \ + gcr.io/gke-on-prem-release/kube-scheduler-amd64:v1.22.8-gke.200 \ + gcr.io/gke-on-prem-release/csi-provisioner:v3.0.0-gke.1 \ + gcr.io/gke-on-prem-release/auto-resize-controller:1.11.1-gke.53 \ + gcr.io/gke-on-prem-release/gke-resource-tracker:v0.12.0 \ + gcr.io/gke-on-prem-release/cilium/operator:v1.11.1-anthos1.11-gke3.4.7 \ + gcr.io/gke-on-prem-release/ang/ang-manager:1.0.7-gke.1 \ + gcr.io/gke-on-prem-release/nad-admission-control:v1.1.1-gke.3 \ + gcr.io/gke-on-prem-release/seesaw-group-controller:1.11.1-gke.53 \ + gcr.io/gke-on-prem-release/kube-rbac-proxy:v0.6.0-gke.0 \ + gcr.io/gke-on-prem-release/cluster-proportional-autoscaler-amd64:1.8.1-gke.0 \ + gcr.io/gke-on-prem-release/seesaw-vserver-controller:1.11.1-gke.53 \ + gcr.io/gke-on-prem-release/calico/typha:v3.19.1-gke.0 \ + gcr.io/gke-on-prem-release/load-balancer-f5:1.11.1-gke.53 \ + gcr.io/gke-on-prem-release/vsphere-metrics-exporter:1.11.1-gke.53 \ + gcr.io/gke-on-prem-release/vsphere-csi-syncer:v2.4.0-gke.0 \ + gcr.io/gke-on-prem-release/k8s-dns-sidecar-amd64:1.22.2-gke.0 \ + gcr.io/gke-on-prem-release/controller-manager:1.11.1-gke.53 \ + gcr.io/gke-on-prem-release/k8s-bigip-ctlr:v1.14.0-gke.10 \ + gcr.io/gke-on-prem-release/etcddefrag:gke_master_etcddefrag_20210802.00_p0 \ + gcr.io/gke-on-prem-release/ais:hybrid_identity_charon_20220307_RC02 \ + gcr.io/gke-on-prem-release/etcd:v3.4.13-1-gke.3 \ + gcr.io/gke-on-prem-release/oidc_proxy:onprem_idp_proxy_20200601_RC00 \ + gcr.io/gke-on-prem-release/asm/pilot:1.13.2-asm.2-distroless \ + gcr.io/gke-on-prem-release/k8s-dns-dnsmasq-nanny-amd64:1.22.2-gke.0 \ + gcr.io/gke-on-prem-release/calico/node:v3.19.1-gke.7 \ + gcr.io/gke-on-prem-release/csi-snapshot-validation-webhook:v4.2.1-gke.0 \ + gcr.io/gke-on-prem-release/vsphere-controller-manager:1.11.1-gke.53 \ + gcr.io/gke-on-prem-release/preflight:1.11.1-gke.53 \ + gcr.io/gke-on-prem-release/calico-controller-manager:1.11.1-gke.53 \ + gcr.io/gke-on-prem-release/ang/vpn-daemon:1.0.7-gke.1 \ + gcr.io/gke-on-prem-release/asm/proxyv2:1.13.2-asm.2-distroless \ + gcr.io/gke-on-prem-release/cluster-proportional-autoscaler-amd64:1.8.4-gke.1 \ + gcr.io/gke-on-prem-release/coredns:v1.8.0-gke.1 \ + gcr.io/gke-on-prem-release/kube-apiserver-amd64:v1.22.8-gke.200 \ + gcr.io/gke-on-prem-release/csi-resizer:v1.3.0-gke.0 \ + gcr.io/gke-on-prem-release/vsphere-csi-driver:v2.4.0-gke.0 \ + gcr.io/gke-on-prem-release/auditproxy:gke_master_auditproxy_20201115_RC00 \ + gcr.io/gke-on-prem-release/cluster-health-controller:1.11.1-gke.53 \ + gcr.io/gke-on-prem-release/monitoring-operator:v1.11.1-gke.10 \ + gcr.io/gke-on-prem-release/metallb/controller:v0.9.6-gke.0 \ + gcr.io/gke-on-prem-release/multus-cni:v3.8.1-gke.6 \ + gcr.io/gke-on-prem-release/net-res-injector:v1.1.0-gke.6 \ + gcr.io/gke-on-prem-release/asm/proxyv2:1.13.2-asm.5 \ + gcr.io/gke-on-prem-release/metrics-server-operator:v1.11.1-gke.10 \ + gcr.io/gke-on-prem-release/csi-attacher:v3.3.0-gke.0 \ + gcr.io/gke-on-prem-release/ang/bgp-daemon:1.0.7-gke.1 \ + gcr.io/gke-on-prem-release/stackdriver-operator:v1.11.1-gke.10 \ + gcr.io/gke-on-prem-release/addon-resizer:1.8.14-gke.0 \ + gcr.io/gke-on-prem-release/k8s-prometheus-adapter-amd64:v0.7.0-gke.6 \ + gcr.io/gke-on-prem-release/etcd-amd64:v3.4.13-1-gke.3 \ + gcr.io/gke-on-prem-release/etcd-util:1.11.1-gke.53 \ + gcr.io/gke-on-prem-release/fluent-bit:v1.8.12-gke.3 \ + gcr.io/gke-on-prem-release/gkeconnect-gce:20220211-01-00 \ + gcr.io/gke-on-prem-release/grafana:5.4.5-distroless-gke.6 \ + gcr.io/gke-on-prem-release/cert-manager-cainjector:v1.5.4-gke.0 \ + gcr.io/gke-on-prem-release/cert-manager-controller:v1.5.4-gke.0 \ + gcr.io/gke-on-prem-release/cert-manager-webhook:v1.5.4-gke.0 \ + gcr.io/gke-on-prem-release/kindest/node:v0.11.1-gke.32-v1.22.6-gke.2100 \ + gcr.io/gke-on-prem-release/kube-state-metrics:2.4.1-gke.1 \ + gcr.io/gke-on-prem-release/anthos-metadata-agent:1.1.4-gke.0 \ + gcr.io/gke-on-prem-release/metadata-agent-go:1.2.0 \ + gcr.io/gke-on-prem-release/pause-amd64:3.1-gke.5 \ + gcr.io/gke-on-prem-release/prometheus-alertmanager:0.21.0-gke.3 \ + gcr.io/gke-on-prem-release/prometheus-nodeexporter:1.0.1-gke.4 \ + gcr.io/gke-on-prem-release/prometheus-reloader:v1.0.25 \ + gcr.io/gke-on-prem-release/prometheus:2.18.1-gke.8 \ + gcr.io/gke-on-prem-release/pushprox-client:v0.0.5 \ + gcr.io/gke-on-prem-release/pushprox-proxy:v0.0.5 \ + gcr.io/gke-on-prem-release/stackdriver-prometheus-sidecar:0.8.0-gke.0 \ + gcr.io/gke-on-prem-release/gke-metrics-agent:1.1.0-anthos.10 \ + gcr.io/gke-on-prem-release/metrics-server-amd64:v0.4.5-gke.0 \ + gcr.io/gke-on-prem-release/target-discovery:v1.0.23 \ + gcr.io/gke-on-prem-release/debug-toolbox:v0.0.23 \ + gcr.io/gke-on-prem-release/hsm-kms-plugin:v0.0.28-gke.0 \ + gcr.io/gke-on-prem-release/proxy-server-amd64:v0.0.24-gke.0" + +# Pull images from public Google Container Registry +for gcr_im in ${IMAGES}; do + echo "Pulling ${gcr_im}" + docker pull ${gcr_im} +done + +# Login to private Artifactory +docker login -u ${ART_USER} -p ${ART_PASSWD} + +# Tag GCR images and push them to private Artifactory +for im in ${IMAGES}; do + im_name="${im##*/}" + echo "${im_name}" + art_name="${ART_URL}/${im_name}" + echo "Tagging ${im} as ${art_name}" + docker tag ${im} ${art_name} + echo "Pushing image ${art_name}" + docker push ${art_name} +done diff --git a/tools/anthosvmware-ansible-module/scripts/checkAPIs.sh b/tools/anthosvmware-ansible-module/scripts/checkAPIs.sh new file mode 100755 index 0000000000..d9052cf821 --- /dev/null +++ b/tools/anthosvmware-ansible-module/scripts/checkAPIs.sh @@ -0,0 +1,60 @@ +#!/bin/bash + +# Set proxy IP as needed +export PROXY_IP_PORT="" + +# COLOR Support; +export RED='\033[0;31m'; +export NC='\033[0m' # No Color; + +export APIs=( + 'accounts.google.com' + 'anthos-migrate.gcr.io' + 'anthos.googleapis.com' + 'anthosaudit.googleapis.com' + 'anthosgke.googleapis.com' + 'cloudresourcemanager.googleapis.com' + 'container.googleapis.com' + 'dl.google.com' # redirects to www.google.com/chrome + 'www.google.com/chrome' + 'docker.io' + 'gcr.io' # redirects to cloud.google.com/container-registry + 'cloud.google.com/container-registry' + 'gkeconnect.googleapis.com' + 'gkehub.googleapis.com' + 'google.com' + 'hashicorp.com' + 'iam.googleapis.com' + 'iamcredentials.googleapis.com' + 'k8s.io' + 'logging.googleapis.com' + 'monitoring.googleapis.com' + 'oauth2.googleapis.com' + 'opsconfigmonitoring.googleapis.com' + 'quay.io' + 'securetoken.googleapis.com' + 'servicecontrol.googleapis.com' + 'serviceusage.googleapis.com' + 'stackdriver.googleapis.com' + 'storage.googleapis.com' + 'storage-api.googleapis.com' + 'storage-component.googleapis.com' + 'sts.googleapis.com' + # below for Anthos Service Mesh + 'cloudtrace.googleapis.com' + 'meshconfig.googleapis.com' + # below for testing the response codes + 'thisshoudnotroutexyz.biz' +) + +# Check connectivity to endpoints +for api in "${APIs[@]}"; +do + printf '%s: %b' "$api" "${RED}"; + if [[ -z "${PROXY_IP_PORT// }" ]]; then + curl -s https://"${api}" -w"%{http_code}\\n" -o /dev/null -m 5; # Skip Proxy + else + curl -sx "$PROXY_IP_PORT" https://"$api" -w"%{http_code}\\n" -o /dev/null -m 5; # Use Proxy + fi + printf '%b' "${NC}"; +done \ No newline at end of file