[ENG-9176] Stuck Registrations are unable to be restarted

[antkryt]

Ticket

ENG-9176

Purpose

force archive: optimize verify stage to avoid getting 502

Changes

• run verify synchronously only in dry mode
• do not verify permissible addons in dry mode
• minor verify() optimization
• fix Check archive status (include children permissible addons)

Side Effects

@adlius we still have two problems with this task:

1. Code optimization won’t fully resolve the issue, it'll only reduce how often it occurs (the same applies to increasing limits on the cloud side). I excluded verify stage when running force archive, but:
2. The following code is really heavy:

  addons = set(DEFAULT_PERMISSIBLE_ADDONS)
  for reg in registration.node_and_primary_descendants():
      addons.update(reg.registered_from.get_addon_names())

It makes a request to GV and several database queries just to determine the permissible addons (for each child registration as well). Maybe we should completely remove the verify stage (dry-mode) and run force_archive entirely as a background task, but that would eliminate any early feedback for admins. With this approach, the only way to get information about the force archive task would be through Sentry logs (in case of failure)

QE Notes

Test my changes on big registrations with 3+ children

CE Notes

Documentation

[adlius]

Code looks good. Maybe if this doesn't work we can turn force_archive into a background task and maybe send email to OSFAdmins for status updates? We can merge this one and see if it works first.