Fix RQ wrongly moving jobs to FailedJobRegistry #7186
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
thiagogds
force-pushed
the
feature/fix-redash-abandoned-error
branch
from
df94793 to
2a7789a
Compare
thiagogds
changed the title
2 tasks
eradman
added a commit
to StarfishStorage/redash
that referenced
this pull request
Oct 15, 2024
Something changed in python-rq and the old code was behaving in a way that if a job ran for longer than 2 min it would be automatically set as failed, but it would continue running. This causes a problem in the UI because it is as if the job stopped, but it actually didn't. getredash#7186
|
@thiagogds can you rebase this change? We fixed the restyled app error by replacing it with a github action. |
Something changed in python-rq and the old code was behaving in a way that if a job ran for longer than 2 min it would be automatically set as failed, but it would continue running. This causes a problem in the UI because it is as if the job stopped, but it actually didn't
thiagogds
force-pushed
the
feature/fix-redash-abandoned-error
branch
from
2a7789a to
3003649
Compare
|
@eradman done :) |
eradman
approved these changes
Oct 17, 2024
|
@eradman Cool! Just to add there, I've been running a fork of Redash with this fix for a few weeks and so far it's good and I don't have the problem anymore :) |
|
Merged. Thanks @thiagogds |
harveyrendell
pushed a commit
to pushpay/redash
that referenced
this pull request
Jan 8, 2025
Something changed in python-rq and the old code was behaving in a way that if a job ran for longer than 2 min it would be automatically set as failed, but it would continue running. This causes a problem in the UI because it is as if the job stopped, but it actually didn't
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Description
Something changed in the newer version of python-rq, and because Redash was overwriting parts of it I started to see that if a job ran for longer than 2 minutes, it would be automatically set as failed but continue running until it completed as success.
It fails with the error:
Moved to FailedJobRegistry, due to AbandonedJobErrorThis causes a problem in the UI because it is as if the job failed, so the query results won't load.
I updated the code with the new version of rq and kept the changes from Redash. Given the changes in the code, it looks like rq changed how it sends a heartbeat for running queries.
How is this tested?
To be able to reproduce locally I needed to:
After the code changes it's now working.
Related Tickets & Documents