Skip to content

[ML] Support CCS for data frame transforms source index #43201

Closed
@benwtrent

Description

@benwtrent

Since it is possible to search and aggregated with CCS, it should be possible to use a remote index (given that security is properly set up) as a source.index in data frames.

There are a handful of things preventing this currently:

  • The local cluster cannot get the mapping information from a remote index. We should move to using _field_caps for determining potential mapping types
  • We also do validations around the source index existing or not. These will not work with a remote index as those indices are not in the cluster state. It may be beneficial to not verify index existence and simply rely on _field_caps to make sure all the fields that the user wants to pivot on and aggregate exist and are aggregatable.
  • Validations around permissions against remote indices should be skipped as well. Only the privileges that exist on the remote cluster are the ones that matter, and there is no good way to determine that outside of just trying to search against the index.

The searching, aggregation, etc. should all "just work" once the data frame transform is running.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions