Kubernetes first, multi-tenant infrastructure provisioning services
Custom platform resources
Definition can be found here Example:
apiVersion: platform.totalsoft.ro/v1alpha1
kind: Platform
metadata:
name: charismaonline.qa
spec:
code: qaDefinition can be found here Example:
apiVersion: platform.totalsoft.ro/v1alpha1
kind: Tenant
metadata:
name: tenant1
namespace: qa
spec:
code: tenant1
configs:
config1: value1
config2: value2
id: cbb451f2-a7c0-430a-949c-54d576c77b8d
platformRef: charismaonline.qamonitors infrastructure manifests and provisions the desired infrastructure for every platform / tenant.
| Variable | example value | details |
|---|---|---|
| AZURE_LOCATION | West Europe | default location used to deploy resources in azure |
| VAULT_ADDR | http://localhost:8200 | address to vault server |
| VAULT_TOKEN | {token} | vault token |
The infrastructure manifests can specify wheather the infrastructure can be provisioned for each tenant in a platform, or for the entire platform
The provisioning target can be set to Tenant, allowing provioning of 'per tenant' infrastructure resources.
Note You can skip provisioning for a list of tenants you can specify a
Blacklistfilter. To allow provisioning for a subset of the tenants, you can specify aWhitelistfilter.
Example:
apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzureDatabase
spec:
....
target:
category: Tenant
filter:
kind: Blacklist
values:
- mbfs
- bnproThe Platform target allows provisioning shared resources for the entire platform.
Example:
apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzureDatabase
metadata:
name: mercury-db
spec:
...
target:
category: PlatformAn infrastructure manifests can specify wheather the provisioned resources depend on the resources provisioned by another manifest. If a dependency is specified, the provisioning of the dependent resource is delayed until the dependency is provisioned.
Note Dependencies can be specified between resources provisioned for the same platform, tenant and domain.
The dependency list is specified in the "dependsOn" present on every provisoning manifest. A dependency is identified by kind and name.
Example:
apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzureManagedDatabase
metadata:
name: my-db
spec:
...
---
apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: HelmRelease
metadata:
name: my-server
spec:
dependsOn:
- kind: AzureManagedDatabase
name: my-db
...Definition can be found here
Example:
apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzureDatabase
spec:
dbName: origination_db
domainRef: origination
exports:
- domain: origination
dbName:
toConfigMap:
keyTemplate: MultiTenancy__Tenants__{{ .Tenant.Code }}__ConnectionStrings__Database
platformRef: charismaonline.qa
sourceDatabaseId: /subscriptions/XXXXXX-b8a5-4967-a05a-2fa3c4295710/resourceGroups/SQLMI_RG/providers/Microsoft.Sql/servers/r7ddbsrv/databases/insurance_db
sqlServer:
elasticPoolName: dbpool
resourceGroupName: SQL_RG
serverName: r7ddbsrv
Note You can skip provisioning for some tenant by adding the label
provisioning.totalsoft.ro/skip-tenant-SOME_TENANT_CODE="true"
Definition can be found here
Example:
apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzureManagedDatabase
spec:
dbName: origination_db
domainRef: origination
exports:
- domain: origination
dbName:
toConfigMap:
keyTemplate: MultiTenancy__Tenants__{{ .Tenant.Code }}__ConnectionStrings__Database
managedInstance:
name: incubsqlmi
resourceGroup: SQLMI_RG
restoreFrom:
backupFileName: origination_test_backup_2022_04_13_154723.bak
storageContainer:
uri: https://my-blobstorage.blob.core.windows.net/backup-repository
sasToken: my-saas-token
platformRef: charismaonline.qaNote You can skip provisioning for some tenant by adding the label
provisioning.totalsoft.ro/skip-tenant-SOME_TENANT_CODE="true"
Definition can be found here
Example:
apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: HelmRelease
metadata:
name: my-helm-release
namespace: my-namespace
spec:
domainRef: origination
platformRef: radu.demo
release:
interval: 10m
releaseName: my-release
chart:
spec:
version: ">=0.1.0-0"
chart: my-chart
sourceRef:
kind: HelmRepository
name: my-helm-repo
namespace: my-namespace
upgrade:
remediation:
remediateLastFailure: false
values:
global:
vaultEnvironment: "false"
runtimeConfiguration:
enabled: false
configMap: origination-aggregate
csi:
secretProviderClass: origination-aggregateNote You can skip provisioning for some tenant by adding the label
provisioning.totalsoft.ro/skip-tenant-SOME_TENANT_CODE="true"
Definition can be found here
Example:
apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzureVirtualMachine
metadata:
name: charisma-client-vm
namespace: qa-lsng
spec:
domainRef: origination
enableTrustedLaunch: false
exports:
- domain: origination
adminPassword:
toVault:
keyTemplate: >-
MultiTenancy__Tenants__{{ .Tenant.Code }}__CharismaClient_Admin_Password
adminUserName:
toVault:
keyTemplate: >-
MultiTenancy__Tenants__{{ .Tenant.Code }}__CharismaClient_Admin_UserName
computerName:
toConfigMap:
keyTemplate: >-
MultiTenancy__Tenants__{{ .Tenant.Code }}__CharismaClient_ComputerName
publicAddress:
toConfigMap:
keyTemplate: >-
MultiTenancy__Tenants__{{ .Tenant.Code }}__CharismaClient_PublicAddress
vmName:
toConfigMap:
keyTemplate: MultiTenancy__Tenants__{{ .Tenant.Code }}__CharismaClient_VM_Name
osDiskType: Standard_LRS
platformRef: charismaonline.qa
rdpSourceAddressPrefix: 128.0.57.0/25
sourceImageId: >-
/subscriptions/05a50a12-6628-4627-bd30-19932dac39f8/resourceGroups/Provisioning_Test/providers/Microsoft.Compute/galleries/MyGallery/images/ch-client-base/versions/2.0.0
subnetId: >-
/subscriptions/05a50a12-6628-4627-bd30-19932dac39f8/resourceGroups/charismaonline.qa/providers/Microsoft.Network/virtualNetworks/charismaonline-vnet/subnets/default
vmName: charisma-client
vmSize: Standard_B1s
Note You can skip provisioning for some tenant by adding the label
provisioning.totalsoft.ro/skip-tenant-SOME_TENANT_CODE="true"
Definition can be found here
Example:
apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzureVirtualDesktop
metadata:
name: charisma-client-avd
namespace: qa-lsng
spec:
applications:
- friendlyName: Charisma Enterprise
name: charisma-enterprise
path: >-
C:\Program Files (x86)\TotalSoft\Charisma Enterprise\Windows Client\Charisma.WinUI.exe
domainRef: origination
enableTrustedLaunch: false
exports:
- adminPassword:
toVault:
keyTemplate: >-
MultiTenancy__Tenants__{{ .Tenant.Code
}}__CharismaClient_Admin_Password
adminUserName:
toVault:
keyTemplate: >-
MultiTenancy__Tenants__{{ .Tenant.Code
}}__CharismaClient_Admin_UserName
computerName:
toConfigMap:
keyTemplate: >-
MultiTenancy__Tenants__{{ .Tenant.Code
}}__CharismaClient_ComputerName
domain: origination
hostPoolName:
toConfigMap:
keyTemplate: >-
MultiTenancy__Tenants__{{ .Tenant.Code
}}__CharismaClient_HostPool_Name
hostPoolName: ch-client-fs
initScript: |
param
(
[Parameter(Mandatory = $true)]
[String]$ChServerIP
)
Write-Host $ChServerIP
initScriptArgs:
- name: ChServerIP
value: 11.12.13.17
osDiskType: Premium_LRS
platformRef: charismaonline.qa
sourceImageId: >-
/subscriptions/15b38e46-ef41-4f5b-bdba-7d9354568c2d/resourceGroups/test-vm/providers/Microsoft.Compute/galleries/LFGalery/images/base-avd/versions/1.0.0
subnetId: >-
/subscriptions/15b38e46-ef41-4f5b-bdba-7d9354568c2d/resourceGroups/test-vm/providers/Microsoft.Network/virtualNetworks/ch-client-base-vnet/subnets/default
users:
admins:
- admin-mbfs@test.onmicrosoft.com
applicationUsers:
- user1-mbfs@test.onmicrosoft.com
vmNamePrefix: ch-client-fs
vmNumberOfInstances: 2
vmSize: Standard_B2s
workspaceFriendlyName: CharismaAzurePowershellScript is a Custom Resource Definition (CRD) that represents an Azure PowerShell deployment script.
Definition can be found here
The AzurePowershellScript spec has the following fields:
scriptContent: The content of the PowerShell script to be executed.scriptArguments: The arguments to be passed to the PowerShell script. These should match the parameters defined in the scriptContent.domainRef: The reference to the domain that the user belongs to.platformRef: The reference to the platform that the user belongs to.forceUpdateTag: Update this value to trigger the script even if the content or args are unchanged
Example:
apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzurePowerShellScript
metadata:
name: createresourcegroup
namespace: provisioning-test
spec:
domainRef: domain2
exports:
- scriptOutputs:
toConfigMap:
keyTemplate: MultiTenancy__Tenants__{{ .Tenant.Code }}__ScriptOutputs
platformRef: provisioning.test
scriptContent: |-
param([string] $name)
$output = "RG name: {0}" -f $name
Write-Output $output
$DeploymentScriptOutputs = @{}
$DeploymentScriptOutputs['text'] = $output
New-AzResourceGroup $name "West Europe"
scriptArguments: "-name testrg-{{ .Platform }}-{{ .Tenant.Code }}"
target:
category: TenantEntraUser is a Custom Resource Definition (CRD) that represents a user for Entra Id.
Definition can be found here
The EntraUser spec has the following fields:
userPrincipalName: The user principal name of the user. This is typically the user's email address or username.displayName: The display name of the user.initialPassword: The initial password for the user. If this is not provided, a random password will be generated.domainRef: The reference to the domain that the user belongs to.platformRef: The reference to the platform that the user belongs to.
Here's an example of an EntraUser resource:
apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: EntraUser
metadata:
name: example-user
namespace: qa-lsng
spec:
userPrincipalName: "user@tenant1-qa.example.com"
displayName: "Example User"
initialPassword: "password123"
domainRef: "entra-users"
platformRef: "qa"
exports:
- domain: entra-users
initialPassword:
toVault:
keyTemplate: InitialPassword
userPrincipalName:
toVault:
keyTemplate: UserPrincipalNamemanages external configuration for the services in the platform, read more about from the Twelve-Factor App methodology.
Definition can be found here
If aggregateConfigMaps is set, it will aggregate all config maps from it's namespace and platform's target namespace, for the specified domain and generates an output config map in the same namespace.
If aggregateSecrets is set, it will aggregate all secrets stored in vault for the specified platform namespace and domain and generates an output CSI SecretProviderClass namespace.
Example:
apiVersion: configuration.totalsoft.ro/v1alpha1
kind: ConfigurationDomain
metadata:
name: origination
namespace: qa-lsng
spec:
aggregateConfigMaps: true
aggregateSecrets: true
platformRef: charismaonline.qaNote 1 The monitored config maps can be either in the same namespace as the ConfigurationDomain or in the platform's target namespace, they are identified by
platform.totalsoft.ro/domainandplatform.totalsoft.ro/platformlabels.
Note 2 There is support for global platform config maps, in this case the
platform.totalsoft.ro/domainlabel has the value "global". These global config maps are always monitored and aggregated with the current domain config maps.
Note 3 The monitored vault secrets are organized by platform (secret engine), namespace (subfolder) and domain (subfolder). The keys of the secrets should be the environment variable names, and if they are not unique for the domain they will be overwritten.
Note 4 There is support for global platform secrets (subfolder) and global namespace secrets (subfolder), in this case the domain folder should be named "global". These global secrets are always monitored and aggregated with the current domain secrets.
For example, for the above manifest, the controller will aggregate secrets from the following paths:
- /charismaonline.qa/qa/global
- /charismaonline.qa/qa-lsng/global
- /charismaonline.qa/qa-lsng/origination/