re-indexing gets "stuck"

[HatBeardZero]

i set up with a separate postgres and redis. I have it setup using the json config to talk to 2 different gitlabs. it seems to be fine to start, but eventually something happens, and then everything queues, but nothing indexes.

we were able to resolve it previously be clearing out the redis and the db, but that doesn't seem like a sustainable solution. I don't see any errors, and re-starting the container does not help. things that have been re-indexing seem to be doing so, and that shows up in the logs, but nothing "failing"

[msukkari]

That's strange, does it work well if you don't have it hooked up to a seperate postgres and redis?

We usually troubleshoot these types of issues by inspecting the DB, since that is the source of truth for the repos that the Sourcebot backend is aware of and their indexing status'. Are you able to go into your psql instance and observe the repoIndexingStatus of the repos?

[HatBeardZero]

i checked one that hasnt been indexed since last week, and it says, "IN_INDEX_QUEUE"

[msukkari]

The flow in Sourcebot is that it picks up repos in the config and then adds them in a job queue in redis. The repo being in IN_INDEX_QUEUE means it was picked up but the job in redis was lost for some reason. Hard to know exactly why that happened, perhaps you restarted the redis instance while the indexing was happening? Or perhaps your redis instance doesn't have enough memory and these extra jobs were lost?

If you change the status from IN_INDEX_QUEUE to NEW it should trigger a reindex of those repos in the meantime