Repository navigation
Split rule.status.NodeEvaluationResults into a separate CRD NodeReadinessEvaluation #89
Description
We discussed this in the past about eventually splitting the node evaluation results from the rule status object and while I have been going through the sig-arch api-conventions, I found below guidelines-
References in the status of the referee to the referrer may be permitted, when the references are one-to-one and do not need to be frequently updated, particularly in an edge-based manner
Our current usage seem to be in conflict with these guidelines, having 1 to many node references in rule status requiring frequent updates.
While having node results directly in the status is nice for observability, this has caused us some concurrency issues that we fixed in the past.
We need to investigate separating the node references into their own. We could decide this after some user feedback or scalability tests.
This applies to three fields - AppliedNodes, FailedNodes, and NodeEvaluations in NodeReadinessRuleStatus object.
Activity
[-]Break rule.status.NodeEvaluationResults into a separate CRD NodeReadinessEvaluation[/-][+]Split rule.status.NodeEvaluationResults into a separate CRD NodeReadinessEvaluation[/+]ajaysundark commentedon Jan 18, 2026
cc @Karthik-K-N
Karthik-K-N commentedon Jan 18, 2026
Thanks for tagging, I can explore on this share my observation.
ajaysundark commentedon Jan 25, 2026
/remove-priority backlog
ajaysundark commentedon Jan 25, 2026
In addition to handling complexity of concurrent updates, storing node-evaluation results in single rule object will likely exceed the size of the etcd 1.5MB limit -- ref: similar reasoning for scaling endpoint slices to pod objects here: https://kubernetes.io/blog/2020/09/02/scaling-kubernetes-networking-with-endpointslices/#scalability-limitations-of-the-endpoints-api
@Karthik-K-N I think we could prioritize this sooner for reliability of the controller in high scale scenarios.
ajaysundark commentedon Jan 25, 2026
Copying discussions w/ Karthik here for completeness --
I think we still need to think it through.
We'd need to split this into two separate problems:
Reading through https://kubernetes.io/blog/2020/09/02/scaling-kubernetes-networking-with-endpointslices/#scalability-limitations-of-the-endpoints-api, I realized (1) is more of a concern than (2) for scale.
To answer your question on cardinality of NodeEvaluation for (2):
we have two options -
[A] 1 node-eval per rule per node, alternatively
[B] 1 node-eval per node (merging all rule results into this single object).
[A] Assuming (1 - 15) rules x 5000 node objects this would raise to (5000 - 75000) eval objects. This will bring in lot of complexity around cleaning up / latency.
[B] 5k eval objects is better, but we are doubling the 'node-like' objects in a cluster. Cleaning up may be manageable with owner-references but I think we may uncover more complexities as we scale test.
We can handle (2) in incremental designs along with planning for scale-tests. We could plan for (1) sooner,
Karthik-K-N commentedon Jan 27, 2026
I agree, In this case
How the user is going to know whether the rule is applied to Node or not, Are events sufficient?
Karthik-K-N commentedon Jan 27, 2026
Given that I have done API changes previously I can take this up, If no one is currently working
/assign
k8s-triage-robot commentedon Apr 27, 2026
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
lifecycle/staleis appliedlifecycle/stalewas applied,lifecycle/rottenis appliedlifecycle/rottenwas applied, the issue is closedYou can:
/remove-lifecycle stale/closePlease send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
Karthik-K-N commentedon Apr 27, 2026
/remove-lifecycle stale
fix(controller): narrow node-deletion cleanup fix per review
fix(controller): narrow node-deletion cleanup fix per review
fix(controller): narrow node-deletion cleanup fix per review
k8s-triage-robot commentedon Oct 17, 2026
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
lifecycle/staleis appliedlifecycle/stalewas applied,lifecycle/rottenis appliedlifecycle/rottenwas applied, the issue is closedYou can:
/remove-lifecycle stale/closePlease send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
ajaysundark commentedon Oct 18, 2026
/remove-lifecycle rotten