BUG: ClusterSummary status showing Provisioned when deployment failed

[wahabmk]

Problem Description

1. Deployed an nginx helm chart using Profile with 1 replica (the Profile deployed via creating a ClusterDeployment in k0rdent):

apiVersion: config.projectsveltos.io/v1beta1
kind: Profile
metadata:
creationTimestamp: "2026-01-26T17:03:18Z"
finalizers:
- profilefinalizer.projectsveltos.io
generation: 3
labels:
  k0rdent.mirantis.com/managed: "true"
  projectsveltos.io/cluster-name: cdunkelb-standalone-m78rdg
  projectsveltos.io/cluster-type: Capi
  projectsveltos.io/profile-name: cdunkelb-standalone-m78rdg
name: cdunkelb-standalone-m78rdg
namespace: kcm-system
ownerReferences:
- apiVersion: k0rdent.mirantis.com/v1beta1
  blockOwnerDeletion: true
  controller: true
  kind: ServiceSet
  name: cdunkelb-standalone-m78rdg
  uid: eba38b73-d815-4850-bba3-8cc341b45f7a
spec:
clusterRefs:
- apiVersion: cluster.x-k8s.io/v1beta2
  kind: Cluster
  name: cdunkelb-standalone-m78rdg
  namespace: kcm-system
clusterSelector: {}
continueOnConflict: true
continueOnError: false
helmCharts:
- chartName: ingress-nginx
  chartVersion: 4.13.2
  helmChartAction: Install
  options:
    installOptions:
      createNamespace: true
      replace: true
  releaseName: ingress-nginx
  releaseNamespace: ingress-nginx
  repositoryName: ingress-nginx
  repositoryURL: oci://ghcr.io/k0rdent/catalog/charts
  values: |-
    controller:
      replicaCount: 1

2. Updated the values to 2 replicas:

  values: |-
    controller:
      replicaCount: 2

3. The ClusterSummary status shows Provisioned:

- apiVersion: config.projectsveltos.io/v1beta1
  kind: ClusterSummary
  metadata:
    creationTimestamp: "2026-01-26T18:00:57Z"
    finalizers:
    - clustersummaryfinalizer.projectsveltos.io
    generation: 3
    labels:
      k0rdent.mirantis.com/managed: "true"
      projectsveltos.io/cluster-name: cdunkelb-standalone-m78rdg
      projectsveltos.io/cluster-type: Capi
      projectsveltos.io/profile-name: cdunkelb-standalone-m78rdg
    name: p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg
    namespace: kcm-system
    ownerReferences:
    - apiVersion: config.projectsveltos.io/v1beta1
      kind: Profile
      name: cdunkelb-standalone-m78rdg
      uid: 93a897f6-9c12-486c-9291-0cd1b6c802de
    resourceVersion: "96716"
    uid: ec030794-dccc-427e-8a24-73e739c4002e
  spec:
    clusterName: cdunkelb-standalone-m78rdg
    clusterNamespace: kcm-system
    clusterProfileSpec:
      clusterRefs:
      - apiVersion: cluster.x-k8s.io/v1beta2
        kind: Cluster
        name: cdunkelb-standalone-m78rdg
        namespace: kcm-system
      clusterSelector: {}
      continueOnConflict: true
      continueOnError: false
      helmCharts:
      - chartName: ingress-nginx
        chartVersion: 4.13.2
        helmChartAction: Install
        options:
          atomic: false
          dependencyUpdate: false
          disableHooks: false
          disableOpenAPIValidation: false
          enableClientCache: false
          installOptions:
            createNamespace: true
            disableHooks: false
            replace: true
          skipCRDs: false
          skipSchemaValidation: false
          uninstallOptions:
            disableHooks: false
          upgradeOptions:
            cleanupOnFail: false
            disableHooks: false
            force: false
            maxHistory: 2
            recreate: false
            resetThenReuseValues: false
            resetValues: false
            reuseValues: false
            subNotes: false
            upgradeCRDs: false
          wait: false
          waitForJobs: false
        releaseName: ingress-nginx
        releaseNamespace: ingress-nginx
        repositoryName: ingress-nginx
        repositoryURL: oci://ghcr.io/k0rdent/catalog/charts
        values: |-
          controller:
            replicaCount: 2
      . . .
  status:
    dependencies: no dependencies
    featureSummaries:
    - featureID: Resources
      hash: TykvJFM9uvkodDA2BA+aMrr+sIMaVafd+QN1nFqy2Cc=
      lastAppliedTime: "2026-01-26T19:21:24Z"
      status: Provisioned
    - featureID: Helm
      hash: 3j/Tcmi/pB9bpdaoN5kPP/mJMOQ5eseUMRY6PTn+ayA=
      lastAppliedTime: "2026-01-26T19:21:24Z"
      status: Provisioned
    helmReleaseSummaries:
    - releaseName: ingress-nginx
      releaseNamespace: ingress-nginx
      status: Managing
      valuesHash: cy9BanPVkgV917PjkmARP80b8K2mW72dNWrk5Sv93NA=

4. But the actual kube Deployment remains at 1 replica.
5. So in summary:

ClusterDeployment: Values propagated successfully, generation increased from 2 to 3
ServiceSet: Values propagated successfully, generation increased from 2 to 3
Profile (sveltos): Values present in spec
Helm release (child cluster): Values propagated successfully, revision 2 shows replicaCount: 2
Actual Deployment: Values NOT applied, still showing 1 replica

This behavior was observed intermittently, we weren't able to reproduce this reliably every time. When it did occur, we could also see the following error in the addon-controller logs:

I0126 19:21:13.747027       1 clustersummary_controller.go:146] "Reconciling" controller="clustersummary" controllerGroup="config.projectsveltos.io" controllerKind="ClusterSummary" ClusterSummary="kcm-system/p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg" namespace="kcm-system" name="p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg" reconcileID="75eaa2bf-08c7-4522-bc04-4f5032c592bf"
I0126 19:21:13.747133       1 clustersummary_controller.go:342] "Reconciling ClusterSummary" controller="clustersummary" controllerGroup="config.projectsveltos.io" controllerKind="ClusterSummary" ClusterSummary="kcm-system/p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg" namespace="kcm-system" name="p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg" reconcileID="75eaa2bf-08c7-4522-bc04-4f5032c592bf"
E0126 19:21:14.145424       1 clustersummary_controller.go:422] "failed to deploy" err=<
	deploying resources failed: request is queued
	deploying helm charts failed: request is queued
 > controller="clustersummary" controllerGroup="config.projectsveltos.io" controllerKind="ClusterSummary" ClusterSummary="kcm-system/p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg" namespace="kcm-system" name="p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg" reconcileID="75eaa2bf-08c7-4522-bc04-4f5032c592bf"

So perhaps when we observed this behavior, the helm deployment failed for some reason and the actual replica remained at 1 but this error wasn't reflected in the ClusterSummary's status, which resulted in the confusion where we thought that the helm chart (with new values) was successfully provisioned when it wasn't?

System Information

CLUSTERAPI VERSION: v1.11.2
SVELTOS VERSION: v1.1.1
KUBERNETES VERSION: v1.33.4+k0s

[wahabmk]

Full support bundle dump is also available if needed.

[gianlucam76]

thank you @wahabmk will take a look and fix it in next release v1.5.0 (unless this is blocker, let me know in case)

[gianlucam76]

Hi @wahabmk I cannot repro this. Few questions for you

1. were you using values or valuesFrom?
2. did you notice if featureSummaries.lastAppliedTime (for helm) and helmReleaseSummaries.valuesHash changed?
3. can you provide the addon-controller logs around the time you made the helm values change?

This is what I did I started with

apiVersion: config.projectsveltos.io/v1beta1
kind: Profile
metadata:
  name: deploy-kyverno
spec:
  clusterSelector:
    matchLabels:
      env: fv
  syncMode: Continuous
  helmCharts:
  - repositoryURL:    https://kyverno.github.io/kyverno/
    repositoryName:   kyverno
    chartName:        kyverno/kyverno
    chartVersion:     3.5.2
    releaseName:      kyverno-latest
    releaseNamespace: kyverno
    helmChartAction:  Install
    values: |
      admissionController:
        replicas: 3
      backgroundController:
        replicas: 3
      cleanupController:
        replicas: 3
      reportsController:
        replicas: 3

  status:
    dependencies: no dependencies
    featureSummaries:
    - featureID: Helm
      hash: i22y1kpumXbsCoHC6m1rDsna011fIm96kFYyzWB2FsY=
      lastAppliedTime: "2026-01-27T15:28:30Z"
      status: Provisioned
    helmReleaseSummaries:
    - releaseName: kyverno-latest
      releaseNamespace: kyverno
      status: Managing
      valuesHash: lElAsvUieRNZJzyuG8N2h4WG554q76Nbl9riWH/rUVo=

and Kyverno pods replicas are correct

NAME                                             READY   STATUS     RESTARTS   AGE
kyverno-admission-controller-59b7957cfb-j52v2    0/1     Init:0/1   0          41s
kyverno-admission-controller-59b7957cfb-mvbsp    1/1     Running    0          41s
kyverno-admission-controller-59b7957cfb-x58mk    1/1     Running    0          41s
kyverno-background-controller-5997b86994-2z8gb   1/1     Running    0          41s
kyverno-background-controller-5997b86994-s776q   1/1     Running    0          41s
kyverno-background-controller-5997b86994-vmzsd   1/1     Running    0          41s
kyverno-cleanup-controller-55db5f8479-6hdm2      1/1     Running    0          41s
kyverno-cleanup-controller-55db5f8479-6qbh4      1/1     Running    0          41s
kyverno-cleanup-controller-55db5f8479-tjl4x      1/1     Running    0          41s
kyverno-reports-controller-846f76ddc7-csbfl      1/1     Running    0          41s
kyverno-reports-controller-846f76ddc7-kv8jq      1/1     Running    0          41s
kyverno-reports-controller-846f76ddc7-vtnlg      1/1     Running    0          41s

then I changed the Profile values to

    values: |
      admissionController:
        replicas: 3
      backgroundController:
        replicas: 3
      cleanupController:
        replicas: 1
      reportsController:
        replicas: 3

Sveltos redeployed it (note the lastAppliedTime changed) and detected values are now different (note the valuesHash for helmReleaseSummaries is different)

  status:
    dependencies: no dependencies
    featureSummaries:
    - featureID: Helm
      hash: czUXMcNxXH+O4SbQHFPaTnO0PvwDTV8bY4C46w8QCmY=
      lastAppliedTime: "2026-01-27T15:30:41Z"
      status: Provisioned
    helmReleaseSummaries:
    - releaseName: kyverno-latest
      releaseNamespace: kyverno
      status: Managing
      valuesHash: 3cUfh6JgNjtUjN2YDWSCUPcWeaLhyedANH5wd/Qhtic=

and Kyverno pods were properly updated

NAME                                             READY   STATUS    RESTARTS   AGE
kyverno-admission-controller-59b7957cfb-gr9g4    1/1     Running   0          4m56s
kyverno-admission-controller-59b7957cfb-pkq6r    1/1     Running   0          4m56s
kyverno-admission-controller-59b7957cfb-splbm    1/1     Running   0          4m56s
kyverno-background-controller-5997b86994-b4q75   1/1     Running   0          4m56s
kyverno-background-controller-5997b86994-b8tb5   1/1     Running   0          4m56s
kyverno-background-controller-5997b86994-z2ppj   1/1     Running   0          4m56s
kyverno-cleanup-controller-55db5f8479-9zmqg      1/1     Running   0          4m56s
kyverno-reports-controller-846f76ddc7-2gfv7      1/1     Running   0          4m56s
kyverno-reports-controller-846f76ddc7-9fpbb      1/1     Running   0          4m56s
kyverno-reports-controller-846f76ddc7-hg57s      1/1     Running   0          4m56s

[wahabmk]

@gianlucam76 We also weren't able to reproduce this reliably, but I can share the support dump with you on slack of the management cluster when it did occur. The best guess I have to reproduce it is to perhaps artificially induce failure here because we see this failure in the addon-controller logs when this issue occurred:

I0126 19:21:13.747027       1 clustersummary_controller.go:146] "Reconciling" controller="clustersummary" controllerGroup="config.projectsveltos.io" controllerKind="ClusterSummary" ClusterSummary="kcm-system/p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg" namespace="kcm-system" name="p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg" reconcileID="75eaa2bf-08c7-4522-bc04-4f5032c592bf"
I0126 19:21:13.747133       1 clustersummary_controller.go:342] "Reconciling ClusterSummary" controller="clustersummary" controllerGroup="config.projectsveltos.io" controllerKind="ClusterSummary" ClusterSummary="kcm-system/p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg" namespace="kcm-system" name="p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg" reconcileID="75eaa2bf-08c7-4522-bc04-4f5032c592bf"
E0126 19:21:14.145424       1 clustersummary_controller.go:422] "failed to deploy" err=<
	deploying resources failed: request is queued
	deploying helm charts failed: request is queued
 > controller="clustersummary" controllerGroup="config.projectsveltos.io" controllerKind="ClusterSummary" ClusterSummary="kcm-system/p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg" namespace="kcm-system" name="p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg" reconcileID="75eaa2bf-08c7-4522-bc04-4f5032c592bf"

[gianlucam76]

Thanks @wahabmk

The error you reported it is just a temporary one.When Sveltos detects a change that requires a re-deployment, queue the request (a Sveltos worker will then take care of it). Then periodically checks the status. What you see means, worker has not picked up the job yet or is still working on it. So it logs the request is queued. But it will check later on.

When request is queued, Sveltos moves the ClusterSummary status to provisioning (and in this case, also updates the helm values hash in the status).

I will look at the logs you provided, if this happens again, please do check the lastAppliedTime in the ClusterSummary. That should report a time after your change. And since your change involved the helm values, please check if the helm value hash in the ClusterSummary Status has changed.

[gianlucam76]

From the logs you provided (thank you @wahabmk), I see worked picked the Job

I0126 19:21:15.008267       1 worker.go:172] "worker: 13 processing request. cleanup: false" logger="deployer" worker="13" key="kcm-system:::cdunkelb-standalone-m78rdg:::Capi:::p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg:::Helm:::false"

and it did deploy it

I0126 19:21:15.627540       1 handlers_helm.go:948] "release ingress-nginx/ingress-nginx (version 4.13.2) status: deployed" logger="deployer" worker="13" key="kcm-system:::cdunkelb-standalone-m78rdg:::Capi:::p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg:::Helm:::false" cluster="kcm-system/cdunkelb-standalone-m78rdg" clusterSummary="p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg"
I0126 19:21:15.645260       1 handlers_helm.go:2112] "update deployed chart info. Number of deployed helm chart: 1" logger="deployer" worker="13" key="kcm-system:::cdunkelb-standalone-m78rdg:::Capi:::p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg:::Helm:::false" cluster="kcm-system/cdunkelb-standalone-m78rdg" clusterSummary="p--cdunkelb-standalone-m78rdg-capi-cdunkelb-standalone-m78rdg"

The problem is in info mode I dont print the values hash (will add it).

I know it might not be feasible for you (it might be too many logs), but would it be possible to set addon-controller logs to Debug level (changing the kubectl edit debuggingconfiguration) and try flipping the values few times? If the error happens again, I should be able to understand the issue then.

Meanwhile I will also try to reproduce this by running in a loop with v1.1.1