Description
I believe the DatabaseScheduler
does not support the expires
option properly. After struggling to make it work experimentally, I checked the source and couldn't find a place where this would be respected.
Here's how I was trying to set it up in my Celery configuration:
CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
CELERYBEAT_SCHEDULE = {
'periodic-task': {
'task': 'myapp.mytask',
'schedule': crontab(minute='*/1'), # schedule every minute
'options': {'expires': 5}, # expire if not run five seconds after being scheduled
}
}
This matches the advice given here in the Celery documentation (»This can be any argument supported by apply_async(), e.g. exchange, routing_key, expires, and so on«), as well as here on SO and also matches how the default entry in the same file tries to set up its own expiration.
I believe the issue is that in djcelery/schedulers.py#L128, the ModelEntry.from_entry()
function fails to read the expires
from the options
variable and insert it into fields
.
Even worse, the database model doesn't even accomodate offsets: djcelery.models.PeriodicTask.expires
is a DateTimeField
, but the Celery documentation documents it as:
Either a int, describing the number of seconds, or a datetime object that describes the absolute time and date of when the task should expire. The task will not be executed after the expiration time.
The scheduler itself appears to have support for setting the expires
on the task in the ModelEntry constructor, so I believe it would schedule the task with the right option if it was able to read that information from the database.
After removing the CELERYBEAT_SCHEDULER
and using the default scheduler, everything seems to work as intended.