Skip to content

About

Repository for helm charts to deploy the ClearBlade IoT Enterprise platform and monitoring

Topics

Resources

Stars

0 stars

Watchers

6 watching

Forks

Repository files navigation

helm-charts

Repository for helm charts to deploy the ClearBlade IoT Enterprise platform and monitoring

ClearBlade IoT Enterprise

The clearblade-iot-enterprise Helm chart enables you install an instance of the platform in your own infrastructure. The Helm charts support installing in your own Kubernetes environment, within Google Cloud, or within AWS.

Kubernetes

To install in your own Kubernetes cluster please use the gke-default-values.yaml file in this repo for configuring this instance. This includes the basic configurations required for installing into a normal Kubernetes environment.

Google Cloud

When installing ClearBlade IoT Enterprise in Google Cloud, a few different services are leveraged to ensure a secure and optimized install. Specifically you will need these services enabled to successfully deploy:

  • Google Kubernetes Engine
  • Google Secret Manager
  • Google Disk
  • Google IP Address
  • Google Service Account

Before installing the instance with these Helm charts, the following resources within Google Cloud must be created first:

Required Resources

Kubernetes Cluster

It is recommended to create a Kubernetes Standard Cluster for your deployment, with the following configuration items:

  • Regional Cluster
  • Specify default node locations (2 zones max)
  • Static default version
  • Cluster autoscaler enabled
  • Min and Max number of nodes per zone set
  • Disable Automatically upgrade nodes to next available version
  • N2 series machines
  • Enable Workload Identify under Cluster Security
  • Enable Managed Service for Prometheus under Cluster Features

IP Addresses

At minimum you will need 1 static public IP address provisioned to allow access to your ClearBlade IoT Enterprise from the public internet. Optinally a second IP address can be provisioned to enable MQTT connections over port 443. Using both IP addresses is the recommended configuration. These 2 IP addresses should be created in the same Region as your GKE Cluster.

Disks

Dedicated Disks are required for persisting data within your ClearBlade IoT Enterprise platform. If you are deploying with a Streaming Replica Postgres instance you will need a dedicated disk for each instance, with the same configuration for each disk. The naming of these disks must also follow a specific pattern, and match the number of Postgres replicas you deploy with:

<namespace>-postgres-0
<namespace>-postgres-1
...

Type: Regional SSD Persistent Disk
Size: 100 GB
Zones: Must match the 2 zones configured in the GKE cluster
Snapshots: Enabled at your preferred schedule for backup purposes

Secrets

The following 6 Secrets are required, and must follow the specific naming pattern provided, replacing <namespace> with the namespcae you are deploying your instance in:

Secret Name Value
<namespace>_clearblade-mek Provided by ClearBlade
<namespace>_postgres-postgres-password Postgres superuser password
<namespace>_postgres-primary-password Postgres primary password
<namespace>_postgres-replica-password Postgres replic password
<namespace>_tls-certificates JSON Object where each key is the filename for a TLS certificate and the value is the base64 encoded pem contents of the cetificate and key.
<namespace>_filestore-hmac-secret A cryptographically secure string

For example, the tls-certificates secrets should look like this:

{
    "clearblade-0.pem": "<base64 .pem file>",
    "clearblade-1.pem": "<base64 .pem file>",
}

The certificates can be referenced by filename when specifying values for platformCertName and mqttCertName.

The filestore-hmac-secret can be generated by running openssl rand -hex 32

Service Account

A Service Account is required for successful deployment of your instance. This Service Account is used internally to provide access to the above configured Secrets.

The default name for you Service Account should be clearblade-gsm-read. If you want to use a different name for any reason you will need to update this in your values.yaml file, as well as the install command.

This Service Account is required to have the following Roles:

  • Secret Manager Admin

Configuration

After all above resources exists, using the gke-default-values.yaml as a starting point, update all configuration values for your specific environment. Note that values for the imagePullerSecret and instance ID will be provided by ClearBlade.

ClearBlade TOML configuration

Most commonly used ClearBlade options and configurations can be set directly through a helm value, as shown in the gke-default-values.yaml file. These configurations will be set via container flag. However, beginning in helm chart version 3.2.0 and ClearBlade version 2025.2.0, you may add additional configurations not supported in these charts via the clearbladeConfig helm value in the ClearBlade section. This is a multiline value that will be read into the clearblade.toml configuration file. NOTE: flags will override any configs set with this method.

Installation

Finally, after connecting to your GKE cluster using the gcloud cli, you will be able to install your instance using the following command, replacing values inside the curly braces to match your environment:

helm install clearblade-iot-enterprise https://github.com/ClearBlade/helm-charts/releases/download/clearblade-iot-enterprise-2.13.6/clearblade-iot-enterprise-2.13.6.tgz -f ./my-values.yaml && gcloud --project={gcp project id} iam service-accounts add-iam-policy-binding clearblade-gsm-read@{gcp project id}.iam.gserviceaccount.com --role roles/iam.workloadIdentityUser --member "serviceAccount:{gcp project id}.svc.id.goog[{kubernetes namespace}/clearblade-gsm-read]"

Amazon Web Services

When installing ClearBlade IoT Enterprise in AWS, the following services are leveraged:

  • Elastic Kubernetes Service (EKS)
  • AWS Secrets Manager
  • Elastic Block Store (EBS)
  • Elastic IP Addresses
  • IAM (optionally with IAM Roles for Service Accounts)
  • Elastic Container Registry (optional, for mirroring images)

Use the eks-default-values.yaml file in this repo as the starting point for your values.

Required Resources

Kubernetes Cluster

An EKS cluster is required, with the following configured:

  • The EBS CSI driver add-on installed
  • The AWS Load Balancer Controller installed (the chart provisions Network Load Balancers through it)
  • An OIDC provider associated with the cluster if using IAM Roles for Service Accounts (recommended)

IP Addresses

Allocate 1 Elastic IP per public subnet the primary load balancer will serve. Optionally allocate a second set to enable MQTT connections over port 443. Set the allocation IDs (eipalloc-...) as comma-separated lists in the cb-haproxy.primaryIP and cb-haproxy.mqttIP values.

Volumes

Pre-created EBS volumes are required for persisting data (100 GB recommended for Postgres). One volume each for:

Volume Value for the volume ID
Postgres cb-postgres.volumeHandle
File hosting cb-file-hosting.volumeHandle
IoT Core sidecar (if enabled) cb-iotcore.volumeHandle
Intelligent Assets sidecar (if enabled) cb-ia.volumeHandle

EBS volumes are zonal. Set cb-postgres.volumeAZ to the availability zone of the Postgres volume so the pod is scheduled where the volume can attach. Create the other volumes in a zone where their pods can run, using node selectors to pin them if the cluster spans multiple zones.

Snapshots via Amazon Data Lifecycle Manager are recommended for backup purposes.

Secrets

The same 6 secrets described in the Google Cloud section above must be created in AWS Secrets Manager, in the same region as the cluster, using the same <namespace>_... naming pattern.

IAM Access to Secrets Manager

The pods read the secrets above at startup, and the certificate renewal controller writes renewed certificates back. Grant access one of two ways:

  • IAM Roles for Service Accounts (recommended): create an IAM role with secretsmanager:GetSecretValue, secretsmanager:DescribeSecret, secretsmanager:CreateSecret, and secretsmanager:PutSecretValue scoped to the <namespace>_* secrets, trusted by the cluster OIDC provider for the clearblade-asm-read service account in your namespace. Set the role ARN in global.awsHelmRoleArn and the chart creates the service account and attaches it to the pods that need it.
  • Node instance role: attach the same permissions to the node group role and leave global.awsHelmRoleArn unset.

replica-patches/create-helm-role.sh is run automatically by replica-patches/configure_streaming.sh when setting up a streaming replica, but can also be run standalone per account/namespace.

Image Registry (optional)

By default all ClearBlade images are pulled from ClearBlade's GCR using the global.imagePullerSecret provided by ClearBlade. To pull from an ECR mirror instead, copy the ClearBlade images into ECR keeping the image names and tags, and set global.registry to your registry prefix, for example 123456789012.dkr.ecr.us-east-1.amazonaws.com/clearblade. Third-party images (haproxy, timescaledb, redis, and others) are referenced by their public names and can be overridden per subchart if needed.

Certificate Renewal

Automatic Let's Encrypt certificate renewal (cb-haproxy.certRenewal) works on AWS with a cb_controller image built with AWS Secrets Manager support. The controller stores renewed certificates back into <namespace>_tls-certificates, so its IAM role needs the write permissions listed above. If you prefer to manage certificates outside the cluster, set cb-haproxy.certRenewal: false and update the secret yourself.

Installation

After connecting to your EKS cluster with aws eks update-kubeconfig, install with:

helm install clearblade-iot-enterprise https://github.com/ClearBlade/helm-charts/releases/download/clearblade-iot-enterprise-{version}/clearblade-iot-enterprise-{version}.tgz -f ./my-values.yaml

ClearBlade Monitoring

The clearblade-monitoring Helm chart is a secondary deployment used to monitor your enterprise instance. To install in your own Kubernetes cluster please use the gke-default-monitoring-values.yaml file in this repo for configuring this instance. This includes the basic configurations required for installing into a normal Kubernetes environment.

GMP

This helm chart is configurable to use either Google Managed Prometheus or a standard Prometheus deployment(reccomended). Be sure that the global value 'GMP' is the same for both monitoring and iot-enterprise deployments in order to ensure the proper exporters and scrape configurations are deployed. Unless you choose to use a Google Managed Prometheus instance, leave GMP set to the default 'false'.

Required Resources

IP Addresses

For monitoring you will need 1 static public IP address provisioned to allow access to your Prometheus and optional Grafana instance from the public internet. This IP address should be created in the same Region as your GKE Cluster.

Secrets

You will need one secret, <monitoring-namespace>_tls-certificates, containing the TLS Certificate and Key for the monitoring domain you will be accessing your instance at in .pem format.

Service Account

You will need the same service account credentials as your enterprise deployment. You may reuse that service account for monitoring.

Disk

If you are not using GMP, it is reccomended to create a disk for your prometheus data. This is optional, but to use simply include the name of the disk and its size in your monitoring-values yaml file. Create your disk with the following specs

Type: Regional SSD Persistent Disk
Zones: Must match the 2 zones configured in the GKE cluster
Snapshots: Enabled at your preferred schedule for backup purposes

Configuration & Installation

After all above resources exists, using the gke-default-monitoring-values.yaml as a starting point, update all configuration values for your specific environment.

Finally, after connecting to your GKE cluster using the gcloud cli, you will be able to install your monitoring instance using the following command, replacing values inside the curly braces to match your environment (the following assumes your monitoring namespace is simply named monitoring):

helm install clearblade-monitoring https://github.com/ClearBlade/helm-charts/releases/download/clearblade-monitoring-2.13.4/clearblade-monitoring-2.13.3.tgz -f ./my-monitoring-values.yaml && gcloud --project={gcp project id} iam service-accounts add-iam-policy-binding clearblade-gsm-read@{gcp project id}.iam.gserviceaccount.com --role roles/iam.workloadIdentityUser --member "serviceAccount:{gcp project id}.svc.id.goog[monitoring/clearblade-gsm-read]" && kubectl annotate serviceaccount clearblade-gsm-read     --namespace=monitoring     iam.gke.io/gcp-service-account=clearblade-gsm-read@{gcp project id}-os.iam.gserviceaccount.com

Blue Green updates

About

This document describes how to update your ClearBlade platform version in a blue/green style to prevent downtime and forced disconnects. It involves adding new ClearBlade pods/containers to your cluster, running the desired version, and having a transitional period while devices naturally reconnect to the newer pods. This comes at the cost of time and compute but should involve no service interruptions.

In our Kubernetes deployments, traffic is routed from the load balancer to ClearBlade through a Kubernetes service. By using labels in Kubernetes, we can limit the scope of this service to ClearBlade pods running only the desired version. Existing connections will persist, while new connections will only be established with the new ClearBlade version. Eventually, all devices will be connected to the new pods, and you can remove the existing pods.

Requirements

Helm charts version 2.13.6 or newer

ClearBlade version 9.34.0 or newer (before update)

The following versions are unskippable - meaning you cannot jump from a prior version to a later version in a blue green fashion, without first updating to these versions (i.e. you cannot go directly from 9.34.1 to 9.34.3)

9.34.0

9.34.2

Steps

These steps assume a blue-to-green transition. To transition from green back to blue for a second update, replace instances of global.enterpriseGreenVersion with global.enterpriseBlueVersion and instances of clearblade.greenReplicas with blueReplicas and vice versa. The replica and version values are applied to the blue pod set but are not explicitly named blue. For step 3, change the slot value to blue instead of green.

In your values file used to deploy ClearBlade with helm, set global.enterpriseGreenVersion to the platform version you want to upgrade to. Then, in the ClearBlade section, set clearblade.greenReplicas to the desired number of pods.

Run

helm upgrade <deployment name> -f <path to values file> <path to clearblade-iot-enterprise tgz file>

This will start your green pod set running the desired version. Wait for them to run, and optionally wait for them to finish autobalancing (see logs).

A useful helm plugin is helm-diff. With it installed, you can run the above command with ‘diff’ added (helm diff upgrade ….). This will not upgrade your deployment but instead show you the changes that will be made so you can be sure you are only making intended changes.

Update your values file to set global.enterpriseSlot to green. If not set, this value defaults to blue.

Run the upgrade command above again. This will update your clearblade-service to route new connections to the new green set of ClearBlade pods instead of the existing old set. Before updating the service, your green pods must be running to establish new connections immediately.

You are now in a transitional state and running two versions of ClearBlade simultaneously. Existing connections to the old ClearBlade set will persist while new connections connect to the new set. Stay in this state until all devices have reconnected to the new set for minimal interruption.

You may change the blueReplicas value to 0, run the upgrade command, and exclusively run the new version and be done with your update.

NOTE

The console and file hosting version, beginning in helm charts 3.1.11, is controlled by global.enterpriseConsoleVersion, and must be manually updated in addition to the platform version. This value can be updated prior to any of the helm upgrade commands above, which will restart those 2 containers and run the new version.

GDC

In order to install this platform on a GDC air-gapped environment, a minumum of helm chart version 3.4.2 is required. Using the gdc-default-values.yaml template file, follow the following steps to deploy:

Steps

Upload the required images to Harbor or your local image repositories. ClearBlade minimum version 2025.3.1 is required.

Fill out the template yaml file with the required image repositories and tags. Aquire mekfile and license from ClearBlade, and place in corresponding helm value overrides.

Create an image puller secret and place base64 encoded into values yaml file

Set your desired resource requests and limits.

Run

helm install <release-name> -f <path to values yaml file> <path to clearblade-iot-enterprise tgz file> --namespace <desired namespace to host helm deployment>

Create DNS record of your base URL at the external IP of the cb-haproxy-service

You should now be able to access the dev console from within your air-gapped network at your baseURL/console . You can register using your set registration key and grant yourself admin priveledge by running clearblade setadmin user@example.com on any running ClearBlade container.

CNPG

Install Plugins (Once per cluster)

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/latest/download/cert-manager.yaml

kubectl apply --server-side -f
https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.27/releases/cnpg-1.27.1.yaml

kubectl apply -f
https://github.com/cloudnative-pg/plugin-barman-cloud/releases/download/v0.11.0/manifest.yaml

Verify plugins

kubectl rollout status deployment -n cnpg-system cnpg-controller-manager kubectl rollout status deployment -n cnpg-system barman-cloud kubectl rollout status deployment -n cert-manager cert-manager

Steps

  1. Create GCP Bucket Name must be globally unique. Add the name to global.bucket in this form: gs://my-bucket-name

  2. Add Workload Identity IAM binding

    The CNPG Cluster creates a Kubernetes Service Account named cb-postgres in your namespace. This KSA is automatically annotated with your GCP service account via serviceAccountTemplate in the Helm chart. You need to create the corresponding IAM binding so the KSA can authenticate to GCS:

    gcloud iam service-accounts add-iam-policy-binding \
      <gcpGSMServiceAccount>@<gcpProject>.iam.gserviceaccount.com \
      --role roles/iam.workloadIdentityUser \
      --member "serviceAccount:<gcpProject>.svc.id.goog[<namespace>/cb-postgres]"
    

    The GCP service account also requires the following role on the GCS bucket (or at the project level):

    • roles/storage.admin

    Note: roles/storage.objectAdmin alone is not sufficient - it lacks storage.buckets.get which is required by the barman cloud plugin.

  3. Configure values

    Using the gke-default-values.yaml as a reference, set the CNPG-related global values in your values file. See the CNPG Values Reference section below for details on each value.

    At minimum you must set:

    global:
      dynamicVolumes: true
      pgReplicas: 3
      bucket: gs://your-bucket-name
      schedule: '0 0 */6 * * *'
      pass: <your-cbuser-password>
  4. Install

    helm install clearblade-iot-enterprise <path-to-chart> -f ./my-values.yaml
    

    On a fresh install, the CNPG operator will:

    • Create a Cluster resource with the specified number of replicas
    • Dynamically provision PVCs and PVs for each replica (no pre-created disks needed)
    • Initialize the admin database with cbuser as owner
    • Install extensions (pg_stat_statements, pg_trgm, and optionally timescaledb)
    • Start WAL archiving to your GCS bucket
    • Take an initial base backup
    • Begin scheduled backups on your cron schedule

    The backup destination folder inside the bucket is determined by global.folder (defaults to clearblade). This folder structure is created automatically by the barman cloud plugin on the first backup - you do not need to create it manually. The folder contains both base backups and WAL archives.

  5. Set restored: true for subsequent deploys

    After your initial deployment is running and has taken at least one successful backup, update your values file to set global.restored: true. This ensures that if the cluster is ever torn down and re-created (e.g. during a nodepool migration or disaster recovery), the CNPG operator will bootstrap from the most recent backup in your GCS bucket rather than initializing a fresh empty database.

    global:
      restored: true

    If you ever need to start completely fresh (new empty database), set restored: false and point to a new bucket or folder.

  6. Import an existing database (optional)

    If you are migrating from a single-pod PostgreSQL deployment to CNPG, you can import your data using pg_restore after the cluster is running.

    1. Ensure you have room on disk and take backup from running postgres with pg_dump -b -C -d admin -U postgres --format=c > admin.dump
    2. Copy to local then to your PRIMARY new postgres pod in new CNPG cluster
    3. Run psql -U postgres -d admin -c "SELECT timescaledb_pre_restore();"
    4. Then restore with pg_restore -U postgres -d admin -v {DUMP_LOCATION}
    5. Run psql -U postgres -d admin -c "SELECT timescaledb_post_restore();" Note: If this command errors, you may need to reinstall timescale with CREATE EXTENSION IF NOT EXISTS timescaledb;. You then may run the post_restore step.
    6. Optionally you may trigger a manual backup to get a copy into your bucket. See below

CNPG Values Reference

All CNPG-related values are documented in gke-default-values.yaml. Here is a summary:

Global Values

Value Required Default Description
global.dynamicVolumes Yes false Set to true to enable CNPG HA Postgres. When false, the legacy single-pod StatefulSet is used.
global.pgReplicas Yes - Number of PostgreSQL replicas (e.g. 3). Replicas are spread across nodes via pod anti-affinity.
global.bucket Yes - GCS bucket path for backups. Format: gs://bucket-name
global.folder No clearblade Server name / folder within the bucket. The backup directory structure is created automatically. Use a unique value per deployment if sharing a bucket.
global.schedule Yes - Cron schedule for automated backups. 6-field format: '0 0 */6 * * *' (every 6 hours).
global.pass Yes (fresh) - Password for the cbuser database user. Only used during initdb (fresh install).
global.restored No false Set to true to bootstrap from a GCS backup instead of initializing fresh.
global.nodepoolSlot No - Node selector slot label for CNPG pod placement.
global.disableTimescale No false Set to true to skip installing the TimescaleDB extension.

cb-postgres Values

Value Required Default Description
cb-postgres.storageSize No 100Gi PVC size for each PostgreSQL replica.
cb-postgres.maxConnections No 500 PostgreSQL max_connections parameter.
cb-postgres.sharedBuffers No 1GB PostgreSQL shared_buffers parameter.
cb-postgres.workMem No 64MB PostgreSQL work_mem parameter.
cb-postgres.walMaxParallel No 8 Number of parallel WAL upload workers.
cb-postgres.retentionPolicy No - Backup retention policy (e.g. 30d for 30 days). If not set, backups are retained indefinitely.
cb-postgres.requestCPU No 1 CPU request per replica.
cb-postgres.requestMemory No 4G Memory request per replica.
cb-postgres.limitCPU No 1 CPU limit per replica.
cb-postgres.limitMemory No 4G Memory limit per replica.

How It Works

When global.dynamicVolumes is set to true, the Helm chart deploys a CNPG Cluster resource instead of the legacy single-pod StatefulSet. The CNPG operator manages the full lifecycle of the PostgreSQL cluster:

  • High Availability: Multiple replicas with streaming replication. The operator automatically handles primary election and replica promotion. If the primary fails, a replica is promoted with no manual intervention.
  • Dynamic Volumes: PVCs are provisioned automatically by the operator using the specified storage class. No pre-created GCE disks or PersistentVolume resources are needed.
  • Continuous Backup: WAL files are continuously archived to GCS via the barman cloud plugin (running as a sidecar container). Scheduled base backups are taken on your configured cron schedule.
  • Restore: When restored: true, the operator bootstraps a new cluster by fetching the latest base backup and replaying WAL files from GCS, recovering to the most recent consistent state.
  • Service Discovery: CNPG creates services automatically: cb-postgres-rw (read-write, always points to primary), cb-postgres-ro (read-only replicas), and cb-postgres-r (all replicas). The ClearBlade platform connects to cb-postgres-rw.

Manual Backups

To take an on-demand backup, apply a Backup resource with a unique name:

kubectl apply -n <namespace> -f - <<EOF
apiVersion: postgresql.cnpg.io/v1
kind: Backup
metadata:
  name: backup-$(date +%s)
  namespace: <namespace>
spec:
  cluster:
    name: cb-postgres
  method: plugin
  pluginConfiguration:
    name: barman-cloud.cloudnative-pg.io
EOF

Each backup gets a unique name via the timestamp suffix. You can take as many manual backups as needed - they accumulate in the GCS bucket independently. To check backup status:

kubectl get backups -n <namespace>

kubectl cnpg Plugin

The cnpg kubectl plugin provides convenience commands for managing CNPG clusters. Install it with:

curl -sSfL https://github.com/cloudnative-pg/cloudnative-pg/raw/main/hack/install-cnpg-plugin.sh | sh

Important: All kubectl cnpg commands require the -n <namespace> flag to work.

Common Commands

# Cluster status overview
kubectl cnpg status cb-postgres -n <namespace>

# Detailed status with replication info
kubectl cnpg status cb-postgres -n <namespace> --verbose

# Promote a replica to primary (for planned switchover)
kubectl cnpg promote cb-postgres <replica-number> -n <namespace>

# List backups
kubectl get backups -n <namespace>

# Check cluster conditions and events
kubectl describe cluster cb-postgres -n <namespace>

# View CNPG operator logs
kubectl logs -n cnpg-system deployment/cnpg-controller-manager -f

About

Repository for helm charts to deploy the ClearBlade IoT Enterprise platform and monitoring

Topics

Resources

Stars

0 stars

Watchers

6 watching

Forks

Releases

Packages

Used by

Contributors

Languages