Is it Observable

Cluster API + Proxmox: Migrating from CAPMOX v0.7 to v0.8

This repository contains everything you need to migrate a Cluster API management cluster from CAPMOX v0.7 (v1alpha1) to CAPMOX v0.8 (v1alpha2), unlocking Kubernetes 1.35 and beyond.

The migration uses a Kind bootstrap cluster as a temporary staging ground, provisions a new Proxmox-hosted management cluster with the upgraded providers, and pivots management via clusterctl move — all without touching your running workloads.

---

What you will learn

• Why clusterctl upgrade apply does not work for the v0.7 → v0.8 transition (broken conversion webhook)
• The v1alpha1 → v1alpha2 API breaking changes: credentialsRef, ipv4Config, and InClusterIPPool
• How to safely back up all CAPI and CAPMOX objects before migrating
• How to bootstrap a new management cluster with CAPI v1.12 + CAPMOX v0.8 using Kind
• How to use clusterctl move to pivot the management plane from Kind to Proxmox (self-hosted)
• How to provision new workload clusters on Kubernetes 1.35.3 using the v1alpha2 API
• How Cilium is auto-deployed via CAPI's ClusterResourceSet (no manual Helm required)

---

What it showcases

• Cluster API v1.12 — Kubernetes lifecycle management via declarative manifests
• CAPMOX v0.8 — Proxmox infrastructure provider with the new v1alpha2 API
• in-cluster IPAM — InClusterIPPool for node IP address management
• kube-vip — ARP-based virtual IP for the control plane endpoint
• Cilium — kube-proxy replacement with Hubble observability
• ClusterResourceSet — automatic CNI installation triggered by cluster bootstrap
• clusterctl move — non-destructive management plane pivot

---

Try Dynatrace

To observe your Cluster API lifecycle, machines, and infrastructure with the Dynatrace 30-day free trial, no credit card required.

---

Prerequisites

Before running any scripts, make sure the following tools are installed and your old management cluster is reachable:

Tool	Minimum version	Notes
kubectl	1.28+	Configured to reach the old management cluster
clusterctl	v1.12+	Must be v1.12 — earlier versions won't install CAPMOX v0.8
kind	v0.23+	Used as a temporary bootstrap cluster
helm	v3.14+	Used to render the Cilium manifest for ClusterResourceSet
docker	24+	Required by Kind

You also need:

• A Proxmox API token with VM create/modify/delete permissions
• A Kubernetes node VM template (Ubuntu 22.04 + containerd + kubeadm) built with image-builder — note the template ID
• A free IP for the management cluster VIP (kube-vip) — outside any DHCP range
• A free IP range for management cluster nodes — non-overlapping with all other clusters
• clusterctl.yaml configured at ~/.cluster-api/clusterctl.yaml (copy and fill in from clusterctl.yaml in this repo)

---

Configuration

Fill in the following files before running any scripts. All <PLACEHOLDER> values are marked with <UPPERCASE_ANGLE_BRACKETS>.

1 — Proxmox credentials (manifests/01-proxmox-credentials.yaml)

This Secret is applied to both the Kind bootstrap cluster and the new Proxmox-hosted management cluster. The label platform.ionos.com/secret-type: proxmox-credentials is required — the CAPMOX controller uses it to discover the credentials.

stringData:
  url: "https://<YOUR_PROXMOX_NODE>:8006"   # e.g. https://192.168.1.10:8006
  token: "capmox@pve!capi"                  # Proxmox API token ID
  secret: "<YOUR_PROXMOX_TOKEN_SECRET>"      # Proxmox API token secret

2 — Management cluster manifest (manifests/10-future-mgmt-cluster.yaml)

This manifest provisions the future Proxmox-hosted management cluster.

Placeholder	Description	Example
<MGMT_VIP>	Virtual IP for the management cluster control plane (kube-vip)	192.168.1.100
<MGMT_NODE_RANGE>	IP range for management cluster nodes (must not overlap other clusters)	192.168.1.101-192.168.1.110
<GATEWAY>	Default gateway for the node network	192.168.1.1
<DNS_SERVER>	DNS server for nodes	192.168.1.1
<PVE_NODE_1>	Name of the first Proxmox node (e.g. pve1)	pve1
<PVE_NODE_2>	Name of the second Proxmox node (optional, for spreading VMs)	pve2
<TEMPLATE_ID>	Proxmox VM template ID (integer)	100

Also update kube-vip static pod definition inside the manifest — the address field must match <MGMT_VIP>.

3 — Provision script (scripts/03-provision-new-mgmt.sh)

Set MGMT_VIP at the top of the script to match your management cluster VIP:

MGMT_VIP="192.168.1.100"   # must match spec.controlPlaneEndpoint.host in 10-future-mgmt-cluster.yaml

4 — (Optional) Workload cluster manifest (manifests/02-workload-cluster.yaml)

Use this as a template for each new workload cluster you want to provision after the migration. Apply the same pattern of placeholders as the management cluster manifest, using a different VIP and IP range per cluster.

---

Deployment Steps

Note: Keep your old management cluster running throughout the entire migration. Your existing Kubernetes clusters and their workloads are never interrupted. Only the management plane moves.

Step 1 — Back up the existing management cluster

Point kubectl at your old management cluster, then run:

cd scripts/
./01-backup-existing.sh

This script pauses all workload clusters, exports every CAPI and CAPMOX resource to a timestamped capi-backup-<timestamp>/ directory, and runs clusterctl move --to-directory for a portable backup. Do not skip this step.

Step 2 — Bootstrap the Kind management cluster

./scripts/02-setup-new-mgmt.sh

This creates a local Kind cluster named capmox-mgmt-v08, then installs CAPI v1.12 + CAPMOX v0.8 + the in-cluster IPAM provider. The Kind cluster is temporary — it only exists to provision the real Proxmox-hosted management cluster in the next step.

Then apply your Proxmox credentials on Kind:

kubectl --context kind-capmox-mgmt-v08 \
  apply -f manifests/01-proxmox-credentials.yaml

Step 3 — Provision the new Proxmox-hosted management cluster

Make sure you have filled in manifests/10-future-mgmt-cluster.yaml and set MGMT_VIP in the script (see Configuration above), then run:

./scripts/03-provision-new-mgmt.sh

The script will:

1. Render a Cilium manifest and create a ClusterResourceSet on Kind (so Cilium is auto-installed as soon as the new cluster bootstraps)
2. Apply manifests/10-future-mgmt-cluster.yaml on Kind to trigger provisioning
3. Wait for the new cluster to become Available (up to 15 minutes)
4. Retrieve the new cluster's kubeconfig to ~/.kube/capmox-mgmt-prod.kubeconfig
5. Run clusterctl init on the new cluster to install all CAPI providers
6. Apply the credentials Secret on the new cluster

Step 4 — Pivot from Kind to Proxmox (self-hosted)

./scripts/04-pivot-to-proxmox.sh

This runs clusterctl move to transfer all CAPI objects from Kind to the new Proxmox-hosted management cluster. After the pivot, the new cluster manages its own lifecycle (self-hosted). Kind is then deleted — its job is done.

Proxmox VMs are never touched during the pivot. Your existing workloads stay up throughout.

Step 5 — Provision new workload clusters

Old v0.7 workload clusters use the v1alpha1 API and cannot be moved to the new management cluster with clusterctl move (the APIs are incompatible). Instead, provision fresh clusters using the v1alpha2 API, migrate your workloads, and decommission the old clusters.

For each workload cluster:

1. Copy manifests/02-workload-cluster.yaml and fill in a unique VIP, IP range, node names, and template ID.

2. Apply on the new management cluster:

   kubectl --kubeconfig ~/.kube/capmox-mgmt-prod.kubeconfig \
     apply -f manifests/02-workload-<name>.yaml

3. Wait for the cluster to become Available:

   kubectl --kubeconfig ~/.kube/capmox-mgmt-prod.kubeconfig \
     wait cluster <name> -n default --for=condition=Available --timeout=15m

4. Migrate your application workloads to the new cluster, then delete the old one.

See MIGRATION-RUNBOOK.md for full phase-by-phase detail, including the old v0.7 cluster teardown sequence.

Henrik Rexed — IsitObservable

📺 Watch the video 📦 GitHub repository