Should we rename the resolved action group?

[mikecote]

@arisonl, @gmmorris and @bmcconaghy shared a concern on the action group name for the new alert on resolve feature. Some proposed alternatives: cleared and recovered with a stronger preference of cleared so far.

[elasticmachine]

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

[pmuellr]

We'll likely want to change the event generated for these as well, currently resolved-instance (it has peers new-instance and active-instance). Currently we only use this internally during the calculation of the alert instance summary, but we can accommodate a new name here by checking for both the old one and new one.

[pmuellr]

Also, brought up in our last meeting where this came up, that it would be nice to have "verb forms" of "new-instance" and "resolved-instance", as we at least informally often refer to active "rules" (nee alerts) as "firing" or "triggering" or "scheduling action groups" - having a well-known term for this would be good, and same for "resolved", but I suppose that's implicit in it's name and the suggestions of "cleared" and "recovered".

[ymao1]

I looked at what terminology some other applications use and saw a few uses of recovered. To me, both cleared and resolved seem to imply some sort of external intervention while recovered does not. That might just be me though :)

[arisonl]

I agree with @ymao1, both resolved and cleared to me imply intervention that takes the alert out of an active status. For example through a manual action or with a timer (resolve it if it's on for more than X minutes). Cleared less so, and I understand it if others prefer cleared over recovered (does recovered sound ambiguous?). Bear in mind that in the competition there is also the notion of recovery thresholds for mitigating alert flapping (thresholds lower than the alert threshold that give you wiggle room before you consider the alert recovered, when the alert fluxes around the alert threshold).

[pmuellr]

Can we queue up a discussion on all these verb-y things?

• rules which schedule actions that didn't before - triggered, activated, fired, etc
• rules which don't schedule actions that did before - recovered, resolved, cleared
• do we need a verb for other "state-y" changes - moving from one action group to another, "started generating errors", the eventual "no data" "state", etc
• also need to distinguish from an alert being active, and and actually scheduling actions - an alert can be active, but throttled, so not schedule actions; what I'm looking for is a verb indicating actions are actually being scheduled, something better than "scheduled actions" (how I refer to it today)

[gmmorris]

I looked at what terminology some other applications use and saw a few uses of recovered. To me, both cleared and resolved seem to imply some sort of external intervention while recovered does not. That might just be me though :)

Yup, this has definitely been my experience - that alerts recover, but much of that experience is rooted in one tool (as ubiquitous as it might be), Nagios, so I don't know how representative it is.

[gmmorris]

I've done some research:

Term	Used By	Thoughts
Resolved	Prometheus, PagerDuty	We feel this term implies intervention that took the alert out of an active status , but in out case, it's that the rule no longer detects the conditions it was detecting before. That said, Prometheus use it to denote auto resolution which is the same as us. This feels different. DataDog actually use this for a button which manually resets a monitor, which further confirms the feeling that it implies intervention.
Recovered	DataDog, Nagios, New Relic	This term is used by Nagios and New Relic to imply that a data source enters a non-violating state after being in a violating state. Additionally, DataDog use "Recovery Threshold" to define a threshold at which the monitor is no longer in a violating state. This feels in line with our approach, where a detected condition, which caused an alert, can no longer be detected.
Cleared	I can't actually find anyone who uses it. Anyone know?
Completed	openDistro	This just seems wrong to me. What has completed? The alert execution? The Actions? The user has completed the investigation? Unclear and not very helpful.
OK	DataDog	This implies that the monitored source of the alert is actively reporting that it's OK. This is a different concept than our framework where we think in detection, rather than monitoring.
Auto-Close	OpsGenie	This happens in response to an Auto-Close Policy which is a condition for detecting resolution. We don't detect resolution, but rather stop detecting an alerting condition.
Auto-Resolved	VictorOps	Like resolved, but makes it clear it was resolved automatically. 🤔

I also looked at InfoSec platforms, such as the Hive, and they don't seem to have any kind of auto resolution of alert, as they need to be manually investigated. For such use cases, this seems less relevant... which makes sense, as this aligns with Case and how they use our framework.

[gmmorris]

Can we queue up a discussion on all these verb-y things?

@pmuellr definitely worth doing, but probably belongs more in the scope of Working Group's terminology discussion than in this issue, right? Fancy spinning off a new issue, otherwise this discussion might get lost in this one (We can also make this a meta issue and open two smaller issues for these two things, up to you).

[peterschretlen]

I know Datadog use the term "OK" for the state, but they also have a concept of "recovery" thresholds which is similar to what this action group does: https://docs.datadoghq.com/monitors/faq/what-are-recovery-thresholds/