Skip to content

Changing polling interval from UI leaves Gitrepo hanged #13662

@mmartin24

Description

@mmartin24

Setup

  • Rancher version: v2.11.0-alpha11
  • Rancher UI Extensions:
  • Browser type & version: Brave 1.75

Describe the bug
On a correctly deployed gitrepo, if polling interval is changed from UI it breaks the girepo leaving it in a permanently In progress status

Related to #13661

To Reproduce
Deploy rancher head or 2.11.0-alpha11
Go to continuous delivery and deploy a working gitrepo. For instance
Once correctly deployed go to 3 dots option on the gitrepo > Edit Config it and go to last step from edit named Advance
Change polling interval to 16 seconds for instance using the UI input and save

Result
Gitrepo shows In Progress status displaying message GitRepo generation is 2, but latest observed generation is 1
Logs shown:

{"level":"error","ts":"2025-03-10T16:49:10Z","logger":"gitjob","msg":"Reconcile failed final update to git repo status","controller":"gitrepo","controllerGroup":"fleet.cattle.io","controllerKind":"GitRepo","GitRepo":{"name":"f-4zmd5","namespace":"fleet-local"},"namespace":"fleet-local","name":"f-4zmd5","reconcileID":"5f6ba2aa-8ec2-44c2-ba03-3154f945b936","generation":1,"commit":"dcf4917293ef131f64724d0c03cadc4f5b257168","conditions":[{"type":"Ready","status":"True","lastUpdateTime":"2025-03-10T16:40:31Z"},{"type":"GitPolling","status":"True","lastUpdateTime":"2025-03-10T16:40:16Z"},{"type":"Reconciling","status":"False","lastUpdateTime":"2025-03-10T16:40:17Z"},{"type":"Stalled","status":"False","lastUpdateTime":"2025-03-10T16:40:17Z"},{"type":"Accepted","status":"True","lastUpdateTime":"2025-03-10T16:40:17Z"}],"status":{"readyClusters":1,"desiredReadyClusters":1,"summary":{"ready":1,"desiredReady":1},"display":{"readyBundleDeployments":"1/1"},"conditions":[{"type":"Ready","status":"True","lastUpdateTime":"2025-03-10T16:40:31Z"},{"type":"GitPolling","status":"True","lastUpdateTime":"2025-03-10T16:40:16Z"},{"type":"Reconciling","status":"False","lastUpdateTime":"2025-03-10T16:40:17Z"},{"type":"Stalled","status":"False","lastUpdateTime":"2025-03-10T16:40:17Z"},{"type":"Accepted","status":"True","lastUpdateTime":"2025-03-10T16:40:17Z"}],"resources":[{"apiVersion":"apps/v1","kind":"Deployment","type":"apps.deployment","id":"default/frontend","namespace":"default","name":"frontend","state":"Ready","perClusterState":{"ready":["fleet-local/local"]}},{"apiVersion":"apps/v1","kind":"Deployment","type":"apps.deployment","id":"default/redis-master","namespace":"default","name":"redis-master","state":"Ready","perClusterState":{"ready":["fleet-local/local"]}},{"apiVersion":"apps/v1","kind":"Deployment","type":"apps.deployment","id":"default/redis-slave","namespace":"default","name":"redis-slave","state":"Ready","perClusterState":{"ready":["fleet-local/local"]}},{"apiVersion":"v1","kind":"Service","type":"service","id":"default/frontend","namespace":"default","name":"frontend","state":"Ready","perClusterState":{"ready":["fleet-local/local"]}},{"apiVersion":"v1","kind":"Service","type":"service","id":"default/redis-master","namespace":"default","name":"redis-master","state":"Ready","perClusterState":{"ready":["fleet-local/local"]}},{"apiVersion":"v1","kind":"Service","type":"service","id":"default/redis-slave","namespace":"default","name":"redis-slave","state":"Ready","perClusterState":{"ready":["fleet-local/local"]}}],"resourceCounts":{"ready":6,"desiredReady":6,"waitApplied":0,"modified":0,"orphaned":0,"missing":0,"unknown":0,"notReady":0},"perClusterResourceCounts":{"fleet-local/local":{"ready":6,"desiredReady":6,"waitApplied":0,"modified":0,"orphaned":0,"missing":0,"unknown":0,"notReady":0}},"observedGeneration":1,"commit":"dcf4917293ef131f64724d0c03cadc4f5b257168","gitJobStatus":"Current","lastSyncedImageScanTime":null,"lastPollingTriggered":"2025-03-10T16:49:10Z"},"error":"Operation cannot be fulfilled on gitrepos.fleet.cattle.io \"f-4zmd5\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/rancher/fleet/internal/cmd/controller/gitops/reconciler.(*GitJobReconciler).Reconcile\n\t/home/runner/_work/fleet/fleet/internal/cmd/controller/gitops/reconciler/gitjob_controller.go:205\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.2/pkg/internal/controller/controller.go:118\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.2/pkg/internal/controller/controller.go:328\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.2/pkg/internal/controller/controller.go:288\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.2/pkg/internal/controller/controller.go:249"}
{"level":"info","ts":"2025-03-10T16:49:10Z","msg":"Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler","controller":"gitrepo","controllerGroup":"fleet.cattle.io","controllerKind":"GitRepo","GitRepo":{"name":"f-4zmd5","namespace":"fleet-local"},"namespace":"fleet-local","name":"f-4zmd5","reconcileID":"5f6ba2aa-8ec2-44c2-ba03-3154f945b936"}
{"level":"error","ts":"2025-03-10T16:49:10Z","msg":"Reconciler error","controller":"gitrepo","controllerGroup":"fleet.cattle.io","controllerKind":"GitRepo","GitRepo":{"name":"f-4zmd5","namespace":"fleet-local"},"namespace":"fleet-local","name":"f-4zmd5","reconcileID":"5f6ba2aa-8ec2-44c2-ba03-3154f945b936","error":"Operation cannot be fulfilled on gitrepos.fleet.cattle.io \"f-4zmd5\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.2/pkg/internal/controller/controller.go:341\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.2/pkg/internal/controller/controller.go:288\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.2/pkg/internal/controller/controller.go:249"}
W0310 16:49:17.630810       7 reflector.go:569] pkg/mod/k8s.io/client-go@v0.32.1/tools/cache/reflector.go:251: failed to list *v1alpha1.GitRepo: time: missing unit in duration "16"
E0310 16:49:17.630850       7 reflector.go:166] "Unhandled Error" err="pkg/mod/k8s.io/client-go@v0.32.1/tools/cache/reflector.go:251: Failed to watch *v1alpha1.GitRepo: failed to list *v1alpha1.GitRepo: time: missing unit in duration \"16\"" logger="UnhandledError"

Expected Result
It should deploy correctly

Screenshots

Image

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    QA/dev-automationIssues that engineers have written automation around so QA doesn't have look at thisarea/fleetkind/bug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions