Skip to content

[FEATURE] Support more than one retaining events dedup pushdown with Calcite #3973

@LantaoJin

Description

@LantaoJin

Is your feature request related to a problem?
#3913 only support dedup pushdown when

  1. number of retaining events no larger than 1
  2. dedup field list only contain one field
  3. the dedup field is one of field of index
  4. the dedup field type is keyword (or contain keyword subfield) or numeric

For the cases not matched (1) and (2), we can support them by using inner_hits. But currently, the response parser cannot handle inner_hits from response.

What solution would you like?
A better response parser to handle inner_hits and new rule to handle the cases (1) and (2).

To support the case (1):

{
  "from": 0,
  "size": 10000,
  "timeout": "1m",
  "_source": false,
  "collapse": {
    "field": "gender.keyword",
    "inner_hits": {
      "name": "level_1",
      "size": 2,
      "_source": {
        "includes": [
          "account_number",
          "gender",
          "age"
        ],
        "excludes": []
      }
    }
  }
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    calcitecalcite migration releatedenhancementNew feature or requestpushdownpushdown related issues

    Type

    No type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions