Skip to content

fix: wait for node Ready before completing bootstrap - #132

Draft
bschimke95 wants to merge 2 commits into
mainfrom
fix/join-wait-for-bootstrap-node-registration
Draft

bschimke95 wants to merge 2 commits into
mainfrom
fix/join-wait-for-bootstrap-node-registration

Conversation

@bschimke95

@bschimke95 bschimke95 commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

onBootstrapControlPlane starts the control-plane services and returns success once kube-apiserver responds, but never waits for this node's own kubelet to register its Kubernetes Node object. That registration is asynchronous, so a node joining the cluster right after bootstrap can see zero Node objects and fail in handleNoUpgradeInProgress with "the cluster has no nodes - cannot determine cluster Kubernetes version", even though the cluster is healthy. Seen in https://github.com/canonical/k8s-snap/actions/runs/36700416543/job/109844096943.

@bschimke95
bschimke95 force-pushed the fix/join-wait-for-bootstrap-node-registration branch from b4a0f58 to 96d16b7 Compare October 6, 2026 16:04
onBootstrapControlPlane starts the control-plane services and returns
success once the kube-apiserver responds, but never waits for this
node's own kubelet to register its Kubernetes Node object. That
registration is asynchronous, so a node joining the cluster right
after bootstrap can see zero Node objects and fail with "the cluster
has no nodes - cannot determine cluster Kubernetes version" in
handleNoUpgradeInProgress, even though the cluster is healthy.

Wait for this node to report Ready before returning from bootstrap,
so bootstrap only completes once the cluster actually has a node.
@bschimke95
bschimke95 force-pushed the fix/join-wait-for-bootstrap-node-registration branch from 96d16b7 to 4d08e49 Compare October 7, 2026 08:38
@bschimke95 bschimke95 changed the title fix: retry cluster Node lookup during join when none are visible yet fix: wait for node Ready before completing bootstrap Oct 7, 2026
@bschimke95
bschimke95 requested a balanced review from Copilot October 8, 2026 08:02

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Waiting for Ready blocks supported bootstrap configurations without managed networking.

1 open finding
What changed in this PR

Adds bootstrap synchronization to prevent joins before the initial Kubernetes node registers.

Changes:

  • Waits for a Ready node after API server startup.
  • Adds readiness polling and tests.
File Description
pkg/​k8sd/​app/​hooks_bootstrap.go Adds the bootstrap readiness wait.
pkg/​k8sd/​app/​cluster_util.go Implements node readiness polling.
pkg/​k8sd/​app/​cluster_util_test.go Tests successful waiting and timeout behavior.

🧠 Review effort: Balanced


Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

Comment thread pkg/k8sd/app/hooks_bootstrap.go Outdated
@bschimke95
bschimke95 force-pushed the fix/join-wait-for-bootstrap-node-registration branch 2 times, most recently from 0adb645 to 7fe5dec Compare October 8, 2026 14:41
Waiting for the Ready condition required a working CNI, so bootstrap with
managed networking disabled (a supported, documented configuration) would
block for the full timeout and fail. It also raced NotifyFeatureController,
the only thing that installs the CNI in the common case, creating an
ordering dependency on an unrelated restart-recovery code path instead of a
deterministic sequence.

The race this closes only needs a joining node to see at least one Node
object (NodeVersions() in hooks_join.go lists all Node objects), not for
that node to be Ready. Wait for registration instead: it doesn't depend on
CNI/NotifyFeatureController at all, so it also removes the ordering
dependency.

Addresses review feedback from @copilot-pull-request-reviewer on this PR.
@bschimke95
bschimke95 force-pushed the fix/join-wait-for-bootstrap-node-registration branch from 7fe5dec to 75c9f72 Compare October 8, 2026 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants