Skip to content

[BUG] System ingest pipeline not triggered when request is from a non-ingest node #18909

@bzhangam

Description

@bzhangam

Describe the bug

When we have a non-ingest node(e.g. a dedicated master node) receiving an index request with the system ingest pipeline then we will resolve it on that non-ingest node and also create the pipeline. But we will not execute it because it's not an ingest node. We will forward the request to an ingest node. If that ingest node hasn't processed any index request for the same target index before then it will not have the system pipeline in the cache. In that case we will try to resolve the system pipeline again. But we didn't reset the indexRequest.isPipelineResolved so we will think the pipeline is already resolved and do nothing.

Related component

Indexing

To Reproduce

  1. Set up a cluster with a dedicated master node and a data node with the ingest role.
  2. Set up an index with a semantic field.
  3. Index a doc for it on the master node.
  4. We will not auto generate the embedding for the semantic field.

Expected behavior

Even when the index request is forwarded from a non-ingest node to an ingest node the system ingest pipeline should be resolved and created properly.

We should reset the indexRequest.isPipelineResolved before we try to resolve the system ingest pipeline again.

Additional Details

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    IndexingIndexing, Bulk Indexing and anything related to indexingbugSomething isn't workinguntriaged

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions