It can take up to 60 seconds for job statuses to update

[vishalbollu]

Description

The status of a Job can take up to 60 seconds to reflect the actual status of a Job because the all job statuses are updated together in a single cron that runs every 60 seconds.

Possibilities for reducing update delays:

• Update job status in the worker that received the job_complete message
• Have two crons:
  • a job completion cron that runs more frequently (every 10 seconds?) and just checks cloudwatch metrics for job completion
  • a cleanup cron that runs (every minute) to cleanup failed jobs and lingering resources