Proxmox talos cluster module

This module allows to declaratively create talos k8s cluster running inside Proxmox VE. Flexible configuration allows to overwrite parameters at multiple levels of setup giving fine grained modification controls. It creates vms, applies pve firewall rules, applies machine configs with patches and bootstraps a cluster.

Usage

main.tf:

module "pve-cluster-talos" {
  source = "github.com/deinsone/terraform-pve-talos-cluster.git"

  name   = "k8s-cluster-1"
  subnet = "10.0.0.0/24"

  defaults = {
    controlplane = {
      network = {
        interface    = "vmbr0"
        gateway-ipv4 = "10.0.0.1"
      }

      image                       = "local:iso/talos-1.9.1-metal.img"
      machine-patch-template-path = "${path.module}/controlplane.yaml.tpl"
    }
    # other custom types ...
  }

  instances = {
    controlplane = {
      k8s-cluster-cp-i1 = {
        node = "proxmox-i1"
        network = {
          address-ipv4 = "10.0.0.21"
        }
      }
    }
  }

  template-args = {
    nodes-subnet = "10.0.0.0/24"
    # other custom properties ...
  }
}

controlplane.yaml.tpl:

machine:
  ...
  kubelet:
    nodeIP:
      validSubnets:
        - ${args.nodes-subnet}
  network:
    hostname: ${name}
    interfaces:
      - deviceSelector:
          physical: true
        addresses:
          - ${network.address-ipv4}/24
        dhcp: false
        routes:
          - gateway: ${network.gateway-ipv4}
    nameservers:
      - ${network.gateway-ipv4}
  ...

Todo

• allow to specify storage drives array
• allow to specify network interfaces array
• allow to specify passthrough pcie devices array
• add more examples
• fined an alternative way to identify vm ip without required subnet specifying

Requirements

Name	Version
terraform	>= 1.5
proxmox	0.69.1
talos	0.7.0

Resources

Name	Type
proxmox_virtual_environment_firewall_options.instances	resource
proxmox_virtual_environment_vm.instances	resource
talos_cluster_kubeconfig.this	resource
talos_machine_bootstrap.this	resource
talos_machine_configuration_apply.this	resource
talos_machine_secrets.this	resource
talos_client_configuration.this	data source
talos_machine_configuration.master	data source
talos_machine_configuration.worker	data source

Inputs

Name	Description	Type	Default	Required
cluster-endpoint	(required) cluster endpoint represents domain with records pointing control plane nodes https://{{example.com}}:6443	string	n/a	yes
cluster-name	(required) name of talos k8s cluster	string	n/a	yes
control-plane-types	(optional) an instance of type from the list is considered a control plane node thus gets control plane machine config applied, others get worker node machine config	list(string)	[ "cp", "cps", "controlplane", "controlplanes", "control-plane", "control-planes", "master", "masters" ]	no
defaults	(required) the object providing configuration defaults for cluster nodes by instance type, can be used to set configuration to node groups	map(object({ node : optional(string) pool : optional(string) tags : optional(list(string), []) note : optional(string) image : optional(string) cpu : optional(number, 2) cpu-type : optional(string, "x86-64-v2") memory-mb : optional(number, 4096) memory-hugepage-mb : optional(number, 0) data-store : optional(string, "local-lvm") disk-gb : optional(number, 16) network : optional(object({ interface : optional(string) gateway-ipv4 : optional(string) vlan : optional(number) })) template-args : optional(map(any)) machine-patch-template-path : string }))	n/a	yes
image	(optional) default talos os image pve path including data store, must be specified either in image, or defaults, or instances, e.g pve-images:iso/talos-1.9.1-metal-amd64.img	string	null	no
instances	(optional) the object providing individual instances configurations by type, by instance name. Can override defaults, complex props are merged	map( map(object({ id : optional(number) node : optional(string) tags : optional(list(string)) pool : optional(string) note : optional(string) image : optional(string) cpu : optional(number) cpu-type : optional(string) memory-mb : optional(number) memory-hugepage-mb : optional(number) data-store : optional(string) disk-gb : optional(number) network : object({ interface : optional(string) address-ipv4 : string gateway-ipv4 : optional(string) vlan : optional(number) }) template-args : optional(map(any)) })) )	{}	no
pool	(optional) pve pool id to add instances to, can be specified or overwritten either in pool, or defaults, or instances	string	null	no
subnet	(required) a subnet cluster nodes are to belong to, should be /24 network covering ips of vms, is required to correctly select node ip to apply machine configs	string	n/a	yes
template-args	(optional) template args allow to pass additional arguments to a template, they can be accessed as { args._ }, is merged with values form defaults and instances	map(any)	{}	no
version-k8s	(optional) version of k8s components images to use, can be overwritten per machine patch	string	null	no
version-talos	(optional) version of talos installer image to use	string	null	no

Outputs

Name	Description
cluster-name	name of talos k8s cluster
control-plane-types	types of control planes
image	default talos os image
instances	snapshot of desired vms configurations, including overrides merged, includes sensitive template args
kubeconfig	kubeconfig collection as returned by talos kubeconfig provider of a provisioned cluster
pool	pve pool id instances are added to
subnet	a subnet cluster nodes are to belong to, should be /24 network covering ips of vms, is required to correctly select node ip to apply machine configs
talosconfig_raw	raw talosconfig file of a provisioned cluster
template-args	default template arguments passed to machine config patches
version-k8s	default version of k8s components images used
version-talos	version of talos installer image to use
vms	provisioned proxmox vms of talos nodes