feat(aci): Ensure DetectorGroup for new events on existing error Groups

[kcons]

We want DetectorGroups for all error Groups.
This is a somewhat tricky thing to backfill, but we can do it incrementally as new events come in for the cost of a single query in the common case. This should result in full coverage in the lifespan of a Group (30 days?) and ensures that any recurring older group will have coverage, which is nice.

A real backfill would certainly be faster, but this is easy, safe. and largely makes a real backfill unnecessary.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #103419      +/-   ##
===========================================
- Coverage   80.67%    80.67%   -0.01%     
===========================================
  Files        9243      9243              
  Lines      394826    394936     +110     
  Branches    25152     25152              
===========================================
+ Hits       318538    318615      +77     
- Misses      75841     75874      +33     
  Partials      447       447              

[mifu67]

You're only touching error issues here—ensure_association_with_detector is never called with a detector ID. Did you want to handle other grouptypes in a separate PR?
In any case, I think that save_issue_occurrence is a good place to handle healing the relationship for metric issues. We can get the detector ID using occurrence_data.evidence_data. Let me know too if you want me to handle that.

[mifu67]

Just confirming that you know what this is doing (I have never seen flags like this before but I trust you 😅)

[kcons]

Unless I'm missing something, I've used option values like this for infrastructural kill switches a few times in the past. We usually use flags that let us roll things out, but if the real goal is just be able to turn something off very quickly and it's not necessarily and org or project specific thing, this seems the easiest way.

[kcons]

You're only touching error issues here—ensure_association_with_detector is never called with a detector ID. Did you want to handle other grouptypes in a separate PR? In any case, I think that save_issue_occurrence is a good place to handle healing the relationship for metric issues. We can get the detector ID using occurrence_data.evidence_data. Let me know too if you want me to handle that.

I was initially just targeting errors here, since for metric issues there was already a migration PR out, but I intentionally left the helper method more general to allow us to extend it to other issue types. So, happy to do that here.