Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Job throttling #672

Open
profsmallpine opened this issue Nov 6, 2024 · 1 comment
Open

[Feature Request] Job throttling #672

profsmallpine opened this issue Nov 6, 2024 · 1 comment

Comments

@profsmallpine
Copy link

This might already be possible, but I haven't been able to figure it out after reading the docs. If I only want a specific type of job to run 1 at a time for example, is this possible?

@mfrister
Copy link

It looks like unique jobs go in the direction of what you want and I'm looking for something similar: A way to ensure a job doing an expensive operation can't have multiple concurrently running instances, so it doesn't put too much load on another system.

UniqueOpts with ByState look like they can almost do this. Explicitly setting the default values ensures there's no job inserted as long as there's another one in the DB, including completed jobs. Removing JobStateCompleted from the list sounds like it would do what what we want, but the documentation says (highlights by me):

The list may be safely customized to add additional states (cancelled
or discarded), though only retryable may be safely removed from the
list.

Warning: Removing any states from the default list (other than retryable)
forces a fallback to a slower insertion path that takes an advisory lock
and performs a look up before insertion. This path is deprecated and should
be avoided if possible.

So I'm thinking maybe I shouldn't use this approach. Is there a better one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants