Handle pending documents that get stuck in processing

[duckduckgrayduck]

Currently, if a document gets caught in pending, the progress bar gets stalled. Might be nice to do some kind of scheduled sweep of pending documents that have been caught in pending for >24 hrs to auto-cancel them

[eyeseast]

Can we keep track of when documents enter a pending state?

Can we see all the pending documents?

[duckduckgrayduck]

We can see all pending documents! In search this is status:pending
As of right now there are 288 pending documents. Some as old as October 2023:
https://www.documentcloud.org/documents/?q=status%3Apending
A lot of them are files that fail to upload from MuckRock (usually due to a DocumentCloud outage).
I think even having a job that runs once monthly and clears out the pending documents would suffice.