[Metrics Alerts] Create Metric Threshold Alert Type and Executor

[Zacqary]

Based off Alerting POC, create a metric threshold alert type and executor function using the new Alerting API.

Requirements

• Name of alert
• Frequency of check
• Frequency of re-notification after being in an alert state
• Conditions:
  • Metric to check
  • Threshold value
  • Comparator (greater than/less than/gt or eq/lt or eq)
• Boolean logic (for multiple conditions: A & B | C)

Implementation Notes

• We should use the same APIs we use for the UI to generate the values for the alert checks
  • Use the snapshot api with a filter applied for alerts based on the Snapshot metrics
  • Use TSVB api for the Metrics Explorer metrics

[elasticmachine]

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

[hbharding]

Boolean logic (for multiple conditions: A & B | C)

I decided not to support the OR logic separator for alert conditions in my designs. It's an interesting idea, but it opened up too many questions and felt out of scope for an initial implementation. Every "expression row" begins with the word "AND", which means the alert will only trigger if all conditions are true.

As a workaround to achieving A & B | C, the user can make two alerts instead.
Alert 1: A & B
Alert 2: C