generated from amazon-archives/__template_Apache-2.0
-
Couldn't load subscription status.
- Fork 621
[DOC] Add join processor documentation #5985
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
d5dbe98
Add join processor documentation
vagimeli 2b17dc9
Merge remote-tracking branch 'origin/main' into join-processor
vagimeli 5bbecdd
Add examples and explanatory text
vagimeli 62bad3c
Address tech review comments
vagimeli b1783a1
Update _ingest-pipelines/processors/join.md
vagimeli b47abd1
Merge branch 'main' into join-processor
vagimeli 761f8fd
Merge branch 'main' into join-processor
vagimeli 1613f7f
Update _ingest-pipelines/processors/join.md
vagimeli 0441266
Update _ingest-pipelines/processors/join.md
vagimeli 6cd64bd
Update _ingest-pipelines/processors/join.md
vagimeli cc946b2
Update _ingest-pipelines/processors/join.md
vagimeli 6d8ba0e
Update _ingest-pipelines/processors/join.md
vagimeli 5b5805a
Update _ingest-pipelines/processors/join.md
vagimeli ac85946
Update _ingest-pipelines/processors/join.md
vagimeli 3f2f9da
Update _ingest-pipelines/processors/join.md
vagimeli dd985ea
Merge branch 'main' into join-processor
vagimeli File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,135 @@ | ||
| --- | ||
| layout: default | ||
| title: Join | ||
| parent: Ingest processors | ||
| nav_order: 160 | ||
| --- | ||
|
|
||
| # Join processor | ||
|
|
||
| The `join` processor concatenates the elements of an array into a single string value, using a specified separator between each element. It throws an exception if the provided input is not an array. | ||
|
|
||
| The following is the syntax for the `join` processor: | ||
|
|
||
| ```json | ||
| { | ||
| "join": { | ||
| "field": "field_name", | ||
| "separator": "separator_string" | ||
| } | ||
| } | ||
| ``` | ||
| {% include copy-curl.html %} | ||
|
|
||
| ## Configuration parameters | ||
|
|
||
| The following table lists the required and optional parameters for the `join` processor. | ||
|
|
||
| Parameter | Required/Optional | Description | | ||
| |-----------|-----------|-----------| | ||
| `field` | Required | The name of the field to which the join operator is applied. Must be an array. | ||
| `separator` | Required | A string separator to use when joining field values. If not specified, then the values are concatenated without a separator. | ||
| `target_field` | Optional | The field to assign the cleaned value to. If not specified, then the field is updated in place. | ||
| `description` | Optional | A description of the processor's purpose or configuration. | ||
| `if` | Optional | Specifies to conditionally execute the processor. | ||
| `ignore_failure` | Optional | Specifies to ignore failures for the processor. See [Handling pipeline failures]({{site.url}}{{site.baseurl}}/ingest-pipelines/pipeline-failures/). | ||
| `on_failure` | Optional | Specifies to handle failures for the processor. See [Handling pipeline failures]({{site.url}}{{site.baseurl}}/ingest-pipelines/pipeline-failures/). | ||
| `tag` | Optional | An identifier for the processor. Useful for debugging and metrics. | ||
|
|
||
| ## Using the processor | ||
|
|
||
| Follow these steps to use the processor in a pipeline. | ||
|
|
||
| ### Step 1: Create a pipeline | ||
|
|
||
| The following query creates a pipeline named `example-join-pipeline` that uses the `join` processor to concatenate all the values of the `uri` field, separating them with the specified separator `/`: | ||
|
|
||
| ```json | ||
| PUT _ingest/pipeline/example-join-pipeline | ||
| { | ||
| "description": "Example pipeline using the join processor", | ||
| "processors": [ | ||
| { | ||
| "join": { | ||
| "field": "uri", | ||
| "separator": "/" | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
| {% include copy-curl.html %} | ||
|
|
||
| ### Step 2 (Optional): Test the pipeline | ||
|
|
||
| It is recommended that you test your pipeline before you ingest documents. | ||
| {: .tip} | ||
|
|
||
| To test the pipeline, run the following query: | ||
|
|
||
| ```json | ||
| POST _ingest/pipeline/example-join-pipeline/_simulate | ||
| { | ||
| "docs": [ | ||
| { | ||
| "_source": { | ||
| "uri": [ | ||
| "app", | ||
| "home", | ||
| "overview" | ||
| ] | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
| {% include copy-curl.html %} | ||
|
|
||
| #### Response | ||
|
|
||
| The following example response confirms that the pipeline is working as expected: | ||
|
|
||
| ```json | ||
| { | ||
| "docs": [ | ||
| { | ||
| "doc": { | ||
| "_index": "_index", | ||
| "_id": "_id", | ||
| "_source": { | ||
| "uri": "app/home/overview" | ||
| }, | ||
| "_ingest": { | ||
| "timestamp": "2024-05-24T02:16:01.00659117Z" | ||
| } | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
| {% include copy-curl.html %} | ||
|
|
||
| ### Step 3: Ingest a document | ||
|
|
||
| The following query ingests a document into an index named `testindex1`: | ||
|
|
||
| ```json | ||
| POST testindex1/_doc/1?pipeline=example-join-pipeline | ||
| { | ||
| "uri": [ | ||
| "app", | ||
| "home", | ||
| "overview" | ||
| ] | ||
| } | ||
| ``` | ||
| {% include copy-curl.html %} | ||
vagimeli marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ### Step 4 (Optional): Retrieve the document | ||
|
|
||
| To retrieve the document, run the following query: | ||
|
|
||
| ```json | ||
| GET testindex1/_doc/1 | ||
| ``` | ||
| {% include copy-curl.html %} | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.