Skip to content

[WIP] Add DownSampleAction to downsampling time_series index #83771

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

weizijun
Copy link
Contributor

@weizijun weizijun commented Feb 10, 2022

I add a new action named DownSampleAction. It's used to downsample time_series index.
The API is a simplification of RollupAction:

POST /{index}/_downsample/{downsample_index}
{
  "fixed_interval": "10m"
}

All parameters are as follows:

{
  "fixed_interval": "xxx",
  "calendar_interval": "xxx",
  "time_zone": "xxx",
  "settings": {
    "index.number_of_shards": "2"
  }
}

Only one interval can be configure in calendar_interval and fixed_interval.

The DownSampleAction only supports time_series index.
It builds the RollupActionConfig from time_series index mapping, and calls the RollupAction.
the build logic is:

  • build the date_histogram config from the down sample configure, and the field name is @timestamp
  • find the time_series_dimension = true field, and add to terms group
  • find the time_series_metric field, and add to metrics group

The code is a POC, I didn't finish and test it. If the design is ok, I'll go ahead and finish it.

@elasticsearchmachine elasticsearchmachine added v8.2.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Feb 10, 2022
@csoulios csoulios self-assigned this Feb 10, 2022
@csoulios csoulios added :StorageEngine/Rollup Turn fine-grained time-based data into coarser-grained data :StorageEngine/TSDB You know, for Metrics labels Feb 10, 2022
@sethmlarson sethmlarson added the Team:Clients Meta label for clients team label Feb 15, 2022
@csoulios
Copy link
Contributor

csoulios commented Jun 9, 2022

Closing this PR because it overlaps with the work at PR #85708

@csoulios csoulios closed this Jun 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
external-contributor Pull request authored by a developer outside the Elasticsearch team :StorageEngine/Rollup Turn fine-grained time-based data into coarser-grained data :StorageEngine/TSDB You know, for Metrics Team:Clients Meta label for clients team v8.4.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants