Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Requeue delay is always 1 min regardless of delay time requested #421

Closed
ewassef opened this issue Apr 25, 2022 · 3 comments
Closed

Requeue delay is always 1 min regardless of delay time requested #421

ewassef opened this issue Apr 25, 2022 · 3 comments
Labels
bug Something isn't working

Comments

@ewassef
Copy link
Contributor

ewassef commented Apr 25, 2022

Describe the bug
When a requeue is returned from the controller, the item is placed on the queue with a 1 minute interval regardless of what the code sends in
To Reproduce

In your ReconcileAsync method simple return return ResourceControllerResult.RequeueEvent(TimeSpan.FromMinutes(15));

Expected behavior

dbug: KubeOps.Operator.Kubernetes.ResourceWatcher[0]
      The server closed the connection. Trying to reconnect.
dbug: KubeOps.Operator.Controller.ManagedResourceController[0]
      Execute/Reconcile event "Reconcile" on resource "ClusterEntity/auto-sample-await".
dbug: ClusterController[0]
      Entered reconcile - state Verified - 4/25/2022 3:31:03 PM
info: KubeOps.Operator.Controller.ManagedResourceController[0]
      Event type "Reconcile" on resource "ClusterEntity/auto-sample-await" successfully reconciled. Requeue requested with delay "00:15:00".
dbug: KubeOps.Operator.Kubernetes.ResourceWatcher[0]
      The server closed the connection. Trying to reconnect.
dbug: KubeOps.Operator.Controller.ManagedResourceController[0]
      Execute/Reconcile event "Reconcile" on resource "ClusterEntity/auto-sample-await".
dbug: ClusterController[0]
      Entered reconcile - state Verified - 4/25/2022 3:32:03 PM <<== This shouldnt be every min, but should be every 15 mins

Additional context
This line seems to have something to do with the issue :
dbug: KubeOps.Operator.Kubernetes.ResourceWatcher[0] The server closed the connection. Trying to reconnect.

@ewassef ewassef added the bug Something isn't working label Apr 25, 2022
@erin-allison
Copy link
Contributor

erin-allison commented Apr 25, 2022

The watcher has a default timeout interval of 60 seconds, so it will always reconcile each resource no less frequently than once per minute.

Since a reconcile event enters the queue at that interval, the event with 15 min delay is dropped.
I forgot the de-duplication stuff isn't merged yet; the event w/ 15 minute delay should execute in addition to the "every one minute" that comes from the watcher reconnecting.

If you want to increase the timeout interval, adjust OperatorSettings.WatcherHttpTimeout to a higher value. For your example, maybe up to 1800 (30 minutes) so you can see that the re-queue is functioning.

/// <summary>
/// The timeout in seconds which the watcher has (after this timeout, the server will close the connection).
/// </summary>
public ushort WatcherHttpTimeout { get; set; } = 60;

@ewassef
Copy link
Contributor Author

ewassef commented Apr 29, 2022

Thank you for the details... ill try that

@ewassef
Copy link
Contributor Author

ewassef commented May 23, 2022

Closing as this work-around works

@ewassef ewassef closed this as completed May 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants