Skip to content

feat: add managed flannel overlay networking - #102

Merged
hsluoyz merged 1 commit into
casosorg:masterfrom
bugkeep:feat/worker-overlay-network
Jul 21, 2026
Merged

feat: add managed flannel overlay networking#102
hsluoyz merged 1 commit into
casosorg:masterfrom
bugkeep:feat/worker-overlay-network

Conversation

@bugkeep

@bugkeep bugkeep commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • install and reconcile a managed, host-networked Flannel VXLAN DaemonSet
  • use the existing kube-controller-manager NodeIPAM as the sole PodCIDR allocator
  • deliver both the Flannel CNI plugin and CNI configuration through the DaemonSet on new and existing nodes
  • migrate the legacy 10-casos-bridge.conflist atomically while retaining the casos-bridge IPAM network name
  • use repository-standard docker.1ms.run images and add critical priority, resource bounds, and rolling updates
  • select worker readiness regressions for platform bootstrap changes and document the managed Flannel setup
  • use authorization webhook v1 so controller-created platform Pods are not rejected by a v1 to v1beta1 conversion
  • advertise Kine's existing etcd 3.6.11 watch-progress support to kube-apiserver

Root cause

Workers previously used node-local bridge networking without a cluster-wide Pod network, so cross-node Pod routing was not reliable. The first version of this PR also duplicated PodCIDR allocation in the deployer even though the controller-manager already runs with --allocate-node-cidrs=true and --cluster-cidr=10.244.0.0/16. That introduced two writers for Node PodCIDRs and could leave a pre-created, permanently NotReady Node when a later SSH or Flannel step failed.

The worker now starts kubelet, waits for the controller-manager's NodeIPAM allocation, then waits for the host-networked Flannel pod to install the plugin and formal CNI configuration before requiring Node Ready. Flannel readiness failures therefore retain the existing pod diagnostics instead of being hidden behind a generic Node Ready timeout.

Kubernetes 1.36's stale-controller consistency check correctly defers DaemonSet reconciliation until informer reads catch up with controller writes. The repeated Flannel message was not a Kine revision rollback: casosorg/kine v0.16.2-r2 already serializes revision appends. It repeated because kube-apiserver defaulted the authorization webhook to v1beta1 while the CasOS webhook serves authorization.k8s.io/v1, causing controller-created Pod authorization to fail and the DaemonSet status to be written again. CasOS now selects authorization webhook v1 explicitly. The embedded Kine server also reports the same etcd 3.6.11 version as Kine's CLI default so kube-apiserver enables the watch-progress capability Kine already implements.

Upgrade behavior

  • the DaemonSet installs /opt/cni/bin/flannel on every selected node, including nodes created before this change
  • the CNI init container writes a temporary file, removes the legacy bridge-only conflist, then atomically publishes 10-flannel.conflist
  • the formal Flannel conflist keeps the casos-bridge network name so existing host-local IPAM allocations remain in the same store
  • reconciliation preserves an existing immutable DaemonSet selector and skips no-op updates

Dependency

Base: master. No preceding PR dependency. The former Kine dependency is already included in casosorg/kine v0.16.2-r2 on current CasOS master. Follow-up #107 depends on this PR and fixes the admission boundary needed for managed platform Pods in the complete issue #101 series.

Validation

  • go test ./...
  • go test ./server ./deploy with local-only regression tests for NodeIPAM reads, CNI migration, plugin delivery, DaemonSet quality, selector preservation, and no-op reconciliation
  • go build -buildvcs=false ./...
  • go vet ./deploy
  • go test ./server and local-only regression checks for authorization webhook v1 and Kine watch-progress advertisement
  • go build ./... and go vet ./...
  • node web/scripts/select-ui-tests-check.js
  • resolved manifests for docker.1ms.run/flannel/flannel:v0.27.4 and docker.1ms.run/flannel/flannel-cni-plugin:v1.8.0-flannel1
  • no Go test files are included in the PR

Fix: #101

@bugkeep
bugkeep force-pushed the feat/worker-overlay-network branch from 686356e to e00ed22 Compare July 21, 2026 16:13
@hsluoyz
hsluoyz merged commit e295ede into casosorg:master Jul 21, 2026
4 of 5 checks passed
@bugkeep
bugkeep deleted the feat/worker-overlay-network branch August 19, 2026 10:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] make these 5 apps work

2 participants