This is a repo to store the scripts in the Migrate TMC SaaS to SM doc, which guides how to migrate resources from TMC SaaS to Self-Managed.
Script | Description | Status | Notes |
---|---|---|---|
001-base-saas_stack-connect.sh | Authenticate and connect to the SaaS platform | READY | - Include both CLI and API options - Once the token or context expired, rerun the script to regenerate one |
002-base-clustergroups-export.sh | Export cluster groups | READY | |
003-base-workspaces-export.sh | Export workspaces | READY | |
004-admin-roles-export.sh | Export roles under Administration | READY | |
005-admin-credentials-export.sh | Export credentials under Administration-Accounts | READY | |
006-admin-access-export.sh | Export access under Administration | READY | |
007-admin-proxy-export.sh | Export proxy configuration under Administration | READY | |
008-admin-image-registry-export.sh | Export image-registry under Administration | READY | |
009-admin-settings-export.sh | Export settings under Administration | READY | |
010-clustergroup-secrets-export.sh | Export k8s secret resources of cluster groups | READY | |
011-clustergroup-secret-exports-export.sh | Export k8s secret export resources of cluster groups | READY | |
012-clustergroup-continuous-deliveries-export.sh | Export fluxcd resources of cluster groups | READY | |
013-clustergroup-repository-credentials-export.sh | Export git repo credential resources of cluster groups | READY | SaaS API call required |
014-clustergroup-git-repositories-export.sh | Export git repository resources of cluster groups | READY | |
015-clustergroup-kustomizations-export.sh | Export kustomization resources of cluster groups | READY | |
016-clustergroup-helms-export.sh | Export helm resources of cluster groups | READY | |
017-clustergroup-helm-releases-export.sh | Export helm release resources of cluster groups | READY | |
018-cluster-namespaces-export.sh | Export managed namespace resources of clusters | READY | |
019-cluster-secrets-export.sh | Export k8s secret resources of clusters | READY | |
020-cluster-secret-exports-export.sh | Export k8s secret export resources of clusters | READY | |
021-cluster-continuous-deliveries-export.sh | Export fluxcd resources of clusters | READY | |
022-cluster-repository-credentials-export.sh | Export git repo credential resources of clusters | READY | |
023-cluster-git-repositories-export.sh | Export git repository resources of clusters | READY | |
024-cluster-kustomizations-export.sh | Export kustomization resources of clusters | READY | |
025-cluster-helms-export.sh | Export helm resources of clusters | READY | |
026-cluster-helm-releases-export.sh | Export helm release resources of clusters | READY | |
027-cluster-data_protection-export.sh | Export data protection resources | READY | |
028-base-access-policies-export.sh | Export access policies | READY | |
029-base-policy-templates-export.sh | Export policy templates | READY | |
030-base-policy-assignments-export.sh | Export policy assignments | READY | |
031-base-managed_clusters-export.sh | Export the metadata of the managed TKG clusters from TMC SaaS before offboarding | READY | VKS (aka. TKGs) and TKGm clusters |
031-base-managed_clusters-offboard.sh | Offboard the managed TKG clusters from TMC SaaS | READY | VKS (aka. TKGs) and TKGm clusters |
032-base-attached_non_npc_clusters-export.sh | Export the metadata of the attached non-NPC clusters from TMC SaaS before offboarding | READY | Attached Non-NPC clusters |
032-base-attached_non_npc_clusters-offboard.sh | Offboard the attached non-NPC clusters from TMC SaaS | READY | Attached Non-NPC clusters |
033-base-sm_stack-connect.sh | Connect to the TMC SM stack | READY | |
034-base-clustergroups-import.sh | Import cluster-groups into TMC SM | READY | |
035-base-workspaces-import.sh | Import workspaces into TMC SM | READY | |
036-admin-roles-import.sh | Import roles into TMC SM | READY | |
037-admin-credentials-create-template.sh | Create post template yaml for each credential | READY | Notes: User need to manually fill in the missing field values such as credentials or CA/Certs |
037-admin-credentials-import.sh | Import credentials with template yaml | READY | Run 037-admin-credentials-create-template.sh before execute this step. |
038-admin-proxy-create-template.sh | Create post template yaml for each proxy | READY | Notes: User need to manually fill in the missing field values such as credentials or CA/Certs |
038-admin-proxy-import.sh | Import proxy with template yaml | READY | Run 038-admin-proxy-create-template.sh before execute this step. |
039-admin-image-registry-create-template.sh | Create post template yaml for each image-registry | READY | Notes: User need to manually fill in the missing field values such as credentials or CA/Certs |
039-admin-image-registry-import.sh | Import image-registry with template yaml | READY | Run 039-admin-image-registry-create-template.sh before execute this step. |
040-clustergroup-secrets-import.sh | Import k8s secret resources to cluster groups | READY | Users must manually fill in the missing data field depending on the type of k8s secret |
041-clustergroup-secret-exports-import.sh | Import k8s secret export resources to cluster groups | READY | |
042-clustergroup-continuous-deliveries-import.sh | Import fluxcd resources to cluster groups | READY | |
043-clustergroup-repository-credentials-import.sh | Import git repository credential resources to cluster groups | READY | Users must manually fill in the missing data field depending on the type of credential |
044-clustergroup-git-repositories-import.sh | Import git repository resources to cluster groups | READY | |
045-clustergroup-kustomizations-import.sh | Import kustomization resources to cluster groups | READY | |
046-clustergroup-helms-import.sh | Import helm resources to cluster groups | READY | |
047-clustergroup-helm-releases-import.sh | Import helm release resources to cluster groups | READY | |
048-base-managed_clusters-onboard.sh | Onboard the managed TKG clusters to TMC SM | READY | - VKS (aka. TKGs) and TKGm clusters - Prepare the required MC Kubeconfig index file with 048-base-managed_clusters-input_from_user.sh |
049-base-non_npc_clusters-onboard.sh | Onboard the attached non-NPC clusters to TMC SM | READY | Attached Non-NPC clusters - Prepare the required WC Kubeconfig index file with 049-base-non_npc_clusters-input_from_user.sh |
049-base-whole_clusters-check_readiness.sh | Check readiness of all onboarded clusters | READY | Include both managed clusters and attached clusters |
050-cluster-namespaces-import.sh | Import managed namespace resources to clusters | READY | |
051-cluster-secrets-import.sh | Import k8s secret resources to clusters | READY | Users must manually fill in the missing data field depending on the type of k8s secret |
052-cluster-secret-exports-import.sh | Import k8s secret export resources to clusters | READY | |
053-cluster-continuous-deliveries-import.sh | Import fluxcd resources to clusters | READY | |
054-cluster-repository-credentials-import.sh | Import git repository credential resources to clusters | READY | Users must manually fill in the missing data field depending on the type of credential |
055-cluster-git-repositories-import.sh | Import git repository resources to clusters | READY | |
056-cluster-kustomizations-import.sh | Import kustomization resources to clusters | READY | |
057-cluster-helms-import.sh | Import helm resources to clusters | READY | |
058-cluster-helm-releases-import.sh | Import helm releases resources to clusters | READY | |
059-admin-settings-import.sh | Import settings to TMC SM | Ready | |
060-admin-access-import.sh | Import access to TMC SM | Ready | |
061-base-access-policies-import.sh | Import access policies on organization/clustegroups/workspaces | READY | The customer should manually edit the access policies with correct user and usergroup identities in the idP of TMC SM after imported. |
061-cluster-access-policies-import.sh | Import access policies on clusters/namespaces | READY | The customer should manually edit the access policies with correct user and usergroup identities in the idP of TMC SM after imported. |
062-base-policy-templates-import.sh | Import policy templates | READY | |
063-base-policy-assignments-import.sh | Import policy assignments on organization/clustergroups/workspaces | READY | |
063-cluster-policy-assignments-import.sh | Import policy assignments on clusters | READY | |
064-cluster-data_protection-import.sh | Import data protections | READY |
Note:
Script file name follows pattern <index>-<scope>-<resource>-<operation>.sh
.
The scope includes:
-
Base
-
Administration
-
Cluster group
-
Cluster
Operation includes:
-
Connect: script used to authenticate and connect to the TMC stack (SaaS or SM)
-
Export: script used to export the resources from SaaS
-
Import: script used to import the exported resource to SM
-
Offboard: unmanage the workload cluster and deregister management cluster from SaaS
-
Onboard: register the management cluster to SM and manage the workload clusters
-
Export the necessary environment variables to set up connection context of SaaS.
export TANZU_API_TOKEN=<CSP-TOKEN> export ORG_NAME=<YOUR-ORG-IDENTITY>
Run script 001-base-saas_stack-connect.sh to create a context for connecting the SaaS stack.
For STG environment, you can export below environment to override the PROD URL.
export CSP_URL=https://console-stg.tanzu.broadcom.com/csp/gateway/am/api/auth/api-tokens/authorize export TMC_ENDPOINT=trh.tmc-dev.tanzu.broadcom.com
-
Export the related resources from the SaaS stack by running scripts 002 - 030.
-
Export the managed clusters with script 031-base-managed_clusters-export.sh. Then offboard the managed clusters from the SaaS stack by running script 031-base-managed_clusters-offboard.sh. Set the environment variable
TMC_MC_FILTER
to export the specified clusters only.# Define the management cluster filter. e.g. "my_mc_1, my_mc_2". export TMC_MC_FILTER="my_mc_1, my_mc_2"
-
Export the managed cluster with script 032-base-attached_non_npc_clusters-export.sh. Then offboard the attached non-NPC clusters from the SaaS stack by running script 032-base-attached_non_npc_clusters-offboard.sh. Set the environment variable
CLUSTER_NAME_FILTER
to export the specified attached clusters only.export CLUSTER_NAME_FILTER="attached1, attached2"
-
Export the necessary environment variables to set up connection context of SM.
export TMC_SELF_MANAGED_USERNAME=admin-user@customer.com export TMC_SELF_MANAGED_PASSWORD=Fake@Pass export TMC_SELF_MANAGED_DNS=tmc.tanzu.io export TMC_SM_CONTEXT=tmc-sm
Run script 033-base-sm_stack-connect.sh to create context for connecting the SM stack.
Notes: If MFA is enabled for the IDP, export this environment variable
export TMC_SM_IDP_MFA_ENABLED=true
. -
Import resources
[cluster group, workspace, roles]
into SM by running scripts 034-036. -
[π€ USER ACTION REQUIRED] List user actions needed for running scripts 037-039.
-
7.1 Run 037-admin-credentials-create-template.sh to generate template yaml for each credential
# data/credentials/template/*.yaml # Notes: User need to manually fill in the missing field values for each template yaml. ./037-admin-credentials-create-template.sh
Template spec formats:
# 1.Spec Format for Self-provisioned: AWS S3 or S3 compatible spec: capability: DATA_PROTECTION data: keyValue: data: aws_access_key_id: "<Your aws_access_key_id>" aws_secret_access_key: "<Your aws_secret_access_key>" type: SECRET_TYPE_UNSPECIFIED meta: provider: GENERIC_S3 temporaryCredentialSupport: false # 2.Spec Format for Self-provisioned: Azure Blob spec: capability: DATA_PROTECTION data: azureCredential: servicePrincipal: azureCloudName: <AzurePublicCloud | AzureUSGovernmentCloud | AzureChinaCloud | AzureGermanCloud> clientId: <Your clientId> clientSecret: <Your clientSecret> resourceGroup: <Your resource group> subscriptionId: <Your subscriptionId> tenantId: <Your tenantId> meta: provider: AZURE_AD temporaryCredentialSupport: false #3.Spec Format for Self-provisioned: AWS_EC2 spec: capability: DATA_PROTECTION data: awsCredential: accountId: "<Your accountId or empty string>" iamRole: arn: "<Your arn>" extId: "<Your extId>" meta: provider: AWS_EC2 temporaryCredentialSupport: false
-
7.2 Run 037-admin-credentials-import.sh to import credentials with template yaml files.
# Please make sure you have already fill in the missing values for each template yaml file. # Notes: User need to manually fill in the missing field values for each template yaml. ./037-admin-credentials-import.sh
-
7.3 Run 038-admin-proxy-create-template.sh to generate template yaml for each proxy
# data/proxy/template/*.yaml # Notes: User need to manually fill in the missing field values for each template yaml. ./038-admin-proxy-create-template.sh
Template spec formats:
# remove the key pair under spec.data.data if empty or it can not pass the base64 validation by backend API. spec: capability: PROXY_CONFIG data: keyValue: data: httpUserName: "<base64 string>" httpPassword: "<base64 string>" httpsUserName: "<base64 string>" httpsPassword: "<base64 string>" proxyCABundle: "<base64 string>" type: SECRET_TYPE_UNSPECIFIED meta: provider: PROVIDER_UNSPECIFIED temporaryCredentialSupport: false
-
7.4 Run 038-admin-proxy-import.sh to import proxy with template yaml files.
# Please make sure you have already fill in the missing values for each template yaml file. # Notes: User need to manually fill in the missing field values for each template yaml. ./038-admin-proxy-import.sh
-
7.5 Run 039-admin-image-registry-create-template.sh to generate template yaml for each image-registry
# data/image-registry/template/*.yaml # Notes: User need to manually fill in the missing field values for each template yaml. ./039-admin-image-registry-create-template.sh
Template spec formats:
# 1. Spec Format for Image registry without username and password spec: capability: IMAGE_REGISTRY data: keyValue: data: registry-url: <registry-url in base64 string> meta: provider: GENERIC_KEY_VALUE temporaryCredentialSupport: false # 2. Spec Format for Image registry with username and password spec: capability: IMAGE_REGISTRY data: keyValue: data: .dockerconfigjson: "<base64 string or call ./utils/create-docker-config-json-base64.sh to generate base64 string>" ca-cert: "<base64 string or remove key/value if not needed >" type: DOCKERCONFIGJSON_SECRET_TYPE meta: provider: GENERIC_KEY_VALUE temporaryCredentialSupport: false
-
7.6 Run 039-admin-image-registry-import.sh to import proxy with template yaml files.
# Please make sure you have already fill in the missing values for each template yaml file. # Notes: User need to manually fill in the missing field values for each template yaml. ./039-admin-image-registry-import.sh
-
-
[π€ USER ACTION REQUIRED] List user action needed for running script 040.
User must manually fill in the missing data fields depending on the type of k8s secret into the manifests in directory
./data/clustergroup-secrets
- SECRET_TYPE_OPAQUE
spec: atomicSpec: data: # filled data field key1: base64-encoded-value1 key2: base64-encoded-value2 secretType: SECRET_TYPE_OPAQUE
- SECRET_TYPE_DOCKERCONFIGJSON
spec: atomicSpec: data: # filled data field .dockerconfigjson: base64-encoded-dockerconfig-json-file secretType: SECRET_TYPE_DOCKERCONFIGJSON
-
Import resources
[secrets-exports, CD]
into SM by running scripts 041-042. -
[π€ USER ACTION REQUIRED] List user action needed for running script 043.
Users must manually fill in the missing data field depending on the type of credential into the manifests in directory
./data/clustergroup-repository-credentials
- Username/Password
spec: atomicSpec: data: # filled data field data: username: bas64-encoded-username password: base64-encoded-password sourceSecretType: USERNAME_PASSWORD
- SSH Authentication
spec: atomicSpec: data: # filled data field data: identity: bas64-encoded-ssh-identity known_hosts: base64-encoded-ssh-known-hosts sourceSecretType: SSH
- CA Certificate
spec: atomicSpec: data: # filled data field data: ca.crt: bas64-encoded-ca-crt username: base64-encoded-username # username and password are optional password: base64-encoded-password # username and password are optional sourceSecretType: CACert
-
Imports resources
[clustergroup:git repo, clustergroup:kustomization, clustergroup:helm, clustergroup:helm-release]
by running scripts 044-047. -
Run script 048-base-managed_clusters-input_from_user.sh to generate a Kubeconfig index file for the onboarding management clusters. Replace the path placeholders
/path/to/the/real/mc_kubeconfig/file
in the generated Kubeconfig index file. Then run script 048-base-managed_clusters-onboard.sh to onboard the exported clusters onto SM. -
Run script 049-base-non_npc_clusters-input_from_user.sh to generate a Kubeconfig index file for the attached clusters. Replace the path placeholders
/path/to/the/real/wc_kubeconfig/file
in the generated Kubeconfig index file.Then run script 049-base-non_npc_clusters-onboard.sh to onboard the attached clusters onto SM.
-
Import resource
[namespace]
into SM by running script 050. -
[π€ USER ACTION REQUIRED] List user actions for 051.
Users must manually fill in the missing data field depending on the type of k8s secret into the manifests in directory
./data/cluster-secrets
- SECRET_TYPE_OPAQUE
spec: data: # filled data field key1: base64-encoded-value1 key2: base64-encoded-value2 secretType: SECRET_TYPE_OPAQUE
- SECRET_TYPE_DOCKERCONFIGJSON
spec: data: # filled data field .dockerconfigjson: base64-encoded-dockerconfig-json-file secretType: SECRET_TYPE_DOCKERCONFIGJSON
-
Import resources
[cluster:secret export, cluster:CD]
into SM by running scripts 052-053 -
[π€ USER ACTION REQUIRED] List user actions for 054.
Users must manually fill in the missing data field depending on the type of credential into the manifests in directory
./data/cluster-repository-credentials
- Username/Password
spec: data: # filled data field data: username: bas64-encoded-username password: base64-encoded-password sourceSecretType: USERNAME_PASSWORD
- SSH Authentication
spec: data: # filled data field data: identity: bas64-encoded-ssh-identity known_hosts: base64-encoded-ssh-known-hosts sourceSecretType: SSH
- CA Certificate
spec: data: # filled data field data: ca.crt: bas64-encoded-ca-crt username: base64-encoded-username # username and password are optional password: base64-encoded-password # username and password are optional sourceSecretType: CACert
-
Import resources
[cluster:git, cluster:kustomization, cluster:helm, cluster:helm-release, admin:settings, admin:access]
into SM by running scripts 055-060 -
Import resources
[access policies, policy templates, policy assignments]
into SM by running scripts 061-063. Notes: The initial access policies are imported with the identities settings from TMC SaaS, the customer should manually edit the access policies with correct user and usergroup identities in the idP of TMC SM. -
Import resources
[Data protection]
064. Notes: TBD to clarify the credentials depends on by DP should be imported in the previous steps.
-
Install the jupyter lab by following its guide.
pip install jupyterlab
-
Clone the repo and cd to the code folder.
-
Start the jupyter lab from the code folder.
# --no-browser and --allow-root is optional. jupyter lab --no-browser --ip=0.0.0.0 --port=80 --allow-root
-
Open notebook
tmc-saas-migration-toi.ipynb
to run migration steps.