[Metrics UI] Add integration tests for Metric Threshold Rule and refactor to fire correctly

[simianhacker]

Summary

This PR adds integration tests along with a refactor to the Metric Threshold Alerts so that they fire as expected. Fixes #110912

Data and Expectations

For this PR I've added a new data set under x-pack/test/functional/es_archives/infra/alerts_test_data that contains 2 real world examples. The first is a gauge style example where the data is indexed every 5 minutes for two different environments, dev and prod. The values for dev are all set to 0 and the values for prod range from 0 to 3 randomly. The very last event looks like this:

{ "@timestamp": "2021-01-01T01:00:00Z", "value": 1, "env": "prod" }

The expectation is that if the rule ran at 2021-01-01T01:00:00Z with the following criteria:

{
  timeSize: 5,
  timeUnit: 'm',
  threshold: [1],
  comparator: Comparator.GT_OR_EQ,
  aggType: Aggregators.SUM,
  metric: 'value',
}

It should trigger a shouldFire event for the prod environment. Here is the same data in a chart with the threshold line.

This failed on the current implementation because it was adding a timeUnit to the end value of the time frame, then moving the end to the start of the timeUnit. Also, the date_range bucket was being offset by 60s which also contributed the situation where it would miss the bucket.

This PR removed the end modification for both gauge and counter style queries. For gauge style queries, the date_range bucketing has been removed. The date_range bucketing was problematic because it uses the following logic @timestamp >= from && @timestamp < to which excluded the last event (which is included in the range query), which we expect to trigger the rule.

The implementation in this PR now relies on the range in the query along with a basic metric aggregation. For gauge style rules, it will only query the same amount of data that is expressed with timeUnit and timeSize; for the example above, that would be the last 5 minutes.

For counter style rules that uses Aggregators.RATE, everything should work the same as before and only trigger on the last "full" bucket. The chart below is from the second data set and has been confirmed with an integration test.

Checklist

• Unit or functional tests were updated or added to match the most common scenarios

Glossary

• gauge style metrics are queries that use basic metric aggregations like sum, avg, max, min, percentile to aggregate values across multiple documents. What makes a metric a gauge is that the value is a snapshot in time. For example system.cpu.total.pct is the total CPU usage being used at that point in time.
• counter style metrics are monotonically increasing numbers where the value is a cumulative sum over time. To find the value at a point in time you have to typically use derivative pipeline aggregation or rate (which is derivative(max(value))).

[elasticmachine]

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 69c425b
• Storybooks Preview

Metrics [docs]

✅ unchanged

History

• 💚 Build #149910 succeeded 26e31f1
• 💚 Build #148946 succeeded 8a23ae2
• 💚 Build #148775 succeeded 06cb9bf
• 💔 Build #148715 failed 3398496
• 💔 Build #148684 failed 58da924

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[jasonrhodes]

LGTM, thanks for writing tests

[neptunian]

Tested the branch and was able to get metric threshold alerts to trigger for both metric types.

[kibanamachine]

💚 Backport successful

Status	Branch	Result
✅	7.15
✅	7.x

The backport PRs will be merged automatically after passing CI.

[kibanamachine]

Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync.