Skip to content

auto_date_histogram fails where date_histogram does not #44914

@igoristic

Description

@igoristic

We are planning to port most of Stack Monitoring current aggregations to use auto_date_histogram in lieu of date_histogram as part of elastic/kibana#37246.

We noticed that auto_date_histogram supports most of aggregations that date_histogram does, but fails (on fetch phase) with bucket_script. As a first pass, I tried replacing the existing date_histogram usage with auto_date_histogram and it broke:

GET .monitoring-logstash-*/_search
{
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "logstash_stats.timestamp": {
              "format": "epoch_millis",
              "gte": 1563551274281,
              "lte": 1563554874281
            }
          }
        }
      ]
    }
  },
  "aggs": {
    "check": {
      "auto_date_histogram": {
        "field": "logstash_stats.timestamp",
        "buckets": 10
      },
      "aggs": {
        "pipelines_nested": {
          "nested": {
            "path": "logstash_stats.pipelines"
          },
          "aggs": {
            "by_pipeline_id": {
              "terms": {
                "field": "logstash_stats.pipelines.id",
                "size": 1000
              },
              "aggs": {
                "by_pipeline_hash": {
                  "terms": {
                    "field": "logstash_stats.pipelines.hash",
                    "size": 1000
                  },
                  "aggs": {
                    "by_ephemeral_id": {
                      "terms": {
                        "field": "logstash_stats.pipelines.ephemeral_id",
                        "size": 1000
                      },
                      "aggs": {
                        "events_stats": {
                          "stats": {
                            "field": "logstash_stats.pipelines.events.out"
                          }
                        },
                        "throughput": {
                          "bucket_script": {
                            "script": "params.max - params.min",
                            "buckets_path": {
                              "min": "events_stats.min",
                              "max": "events_stats.max"
                            }
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

This query fails where the same query works if you replace auto_date_histogram with date_histogram (and use interval instead of buckets). Anyone attempting to run this on an existing cluster will likely need to shift the date range query.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions