Add periodic jobs dynamically #242
Comments
|
Hi @Catgroove, this is something we’ve been thinking about and are hoping to implement at some point. Meanwhile, is there any chance your use case would be served by a combination of periodic and unique jobs? Depending on how these jobs are structured it should be possible to set up unique jobs that will insert at most once every N minutes. This would mean your event consumer could always try to insert the relevant event for a given timeframe and would quietly skip the insert if one already exists for that timeframe. |
Thanks for the suggestion, but I'm afraid the use case is more complicated than what this solution allows. The interval could change based on the message, so there's this need of both adding, removing, and changing how a job is setup, everytime a message is consumed. |
|
I have a related use-case where I would like to be able to enable and disable periodic jobs based on some user controlled config that is updated in an API call. As it is now I think the only way to handle this would be stopping and restarting a river client. So being able to do the following would be nice
|
|
This is a deal breaker for our use case also. I’m considering using a schedule that schedules itself after it’s done to fake a dynamic periodic task but that’s far from elegant but I’m hoping it suffices until maybe this feature is supported. I love how Riverqueue is designed. Appreciate what you guys have built. Thank you |
Here, attempt to resolve #242 by providing a way for new periodic jobs to be added dynamically after a client's already started (i.e. not only by passing them to the client's initial constructor). The new API puts all functions on a separate `PeriodicJobs` bundle to avoid further polluting `Client`'s namespace, and to make all related functions easy to find. riverClient.PeriodicJobs().Add( river.NewPeriodicJob( river.PeriodicInterval(15*time.Minute), func() (river.JobArgs, *river.InsertOpts) { return PeriodicJobArgs{}, nil }, nil, ), ) Additions return a periodic job "handle" which can be used to remove an added job: periodicJobHandle := riverClient.PeriodicJobs().Add(...) riverClient.PeriodicJobs().Remove(periodicJobHandle) I used a handle because the `river.PeriodicJob` construct isn't particularly pretty to keep around, and contains a number of fields that are aren't comparable so its unsuitable for use in equality checks or as a map key. Adding a new periodic job bumps the enqueuer's run loop so that it enqueues the job immediately if it's configured with `RunOnStart`, and schedules its initial target run time. In other words, adding or removing periodic jobs should take effect ~instantly. Fixes #242.
Here, attempt to resolve #242 by providing a way for new periodic jobs to be added dynamically after a client's already started (i.e. not only by passing them to the client's initial constructor). The new API puts all functions on a separate `PeriodicJobs` bundle to avoid further polluting `Client`'s namespace, and to make all related functions easy to find. riverClient.PeriodicJobs().Add( river.NewPeriodicJob( river.PeriodicInterval(15*time.Minute), func() (river.JobArgs, *river.InsertOpts) { return PeriodicJobArgs{}, nil }, nil, ), ) Additions return a periodic job "handle" which can be used to remove an added job: periodicJobHandle := riverClient.PeriodicJobs().Add(...) riverClient.PeriodicJobs().Remove(periodicJobHandle) I used a handle because the `river.PeriodicJob` construct isn't particularly pretty to keep around, and contains a number of fields that are aren't comparable so its unsuitable for use in equality checks or as a map key. Adding a new periodic job bumps the enqueuer's run loop so that it enqueues the job immediately if it's configured with `RunOnStart`, and schedules its initial target run time. In other words, adding or removing periodic jobs should take effect ~instantly. Fixes #242.
Here, attempt to resolve #242 by providing a way for new periodic jobs to be added dynamically after a client's already started (i.e. not only by passing them to the client's initial constructor). The new API puts all functions on a separate `PeriodicJobs` bundle to avoid further polluting `Client`'s namespace, and to make all related functions easy to find. riverClient.PeriodicJobs().Add( river.NewPeriodicJob( river.PeriodicInterval(15*time.Minute), func() (river.JobArgs, *river.InsertOpts) { return PeriodicJobArgs{}, nil }, nil, ), ) Additions return a periodic job "handle" which can be used to remove an added job: periodicJobHandle := riverClient.PeriodicJobs().Add(...) riverClient.PeriodicJobs().Remove(periodicJobHandle) I used a handle because the `river.PeriodicJob` construct isn't particularly pretty to keep around, and contains a number of fields that are aren't comparable so its unsuitable for use in equality checks or as a map key. Adding a new periodic job bumps the enqueuer's run loop so that it enqueues the job immediately if it's configured with `RunOnStart`, and schedules its initial target run time. In other words, adding or removing periodic jobs should take effect ~instantly. Fixes #242.
Here, attempt to resolve #242 by providing a way for new periodic jobs to be added dynamically after a client's already started (i.e. not only by passing them to the client's initial constructor). The new API puts all functions on a separate `PeriodicJobs` bundle to avoid further polluting `Client`'s namespace, and to make all related functions easy to find. riverClient.PeriodicJobs().Add( river.NewPeriodicJob( river.PeriodicInterval(15*time.Minute), func() (river.JobArgs, *river.InsertOpts) { return PeriodicJobArgs{}, nil }, nil, ), ) Additions return a periodic job "handle" which can be used to remove an added job: periodicJobHandle := riverClient.PeriodicJobs().Add(...) riverClient.PeriodicJobs().Remove(periodicJobHandle) I used a handle because the `river.PeriodicJob` construct isn't particularly pretty to keep around, and contains a number of fields that are aren't comparable so its unsuitable for use in equality checks or as a map key. Adding a new periodic job bumps the enqueuer's run loop so that it enqueues the job immediately if it's configured with `RunOnStart`, and schedules its initial target run time. In other words, adding or removing periodic jobs should take effect ~instantly. Fixes #242.
|
I just cut v0.2.0, which includes this feature. Some docs here: https://riverqueue.com/docs/periodic-jobs#adding-periodic-jobs-after-client-start |
Based on the current documentation, it seems like it's only possible to add periodic jobs during client creation. It would be nice to be able add periodic jobs after the client has been created, similar to what you can do with scheduled jobs.
Why?
It might be a niche use case, but I have this requirement to react to events in a message driven system, and based on those events I want to create or delete periodic jobs dynamically. river would be a perfect use case for this if this feature was available.
The text was updated successfully, but these errors were encountered: