generated from amazon-archives/__template_Apache-2.0
-
Couldn't load subscription status.
- Fork 621
[DOC] Add ingest processors documentation #4299
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
286 commits
Select commit
Hold shift + click to select a range
49722a8
Writing processors backlog documentation
vagimeli e56cab1
Update append.md
vagimeli 0437b2b
Update searchable snapshot documentation to be more correct (#4203)
adaisley 7e41042
Update plugins.md (#4353)
wbeckler 698fff6
Add date nanoseconds field type (#4348)
kolchfa-aws 3bb298b
Add model access control documentation for ML Commons (#4223)
kolchfa-aws bc02ea2
Update nested.md (#4363)
kolchfa-aws c1ddaa7
Reformat supported field types index page (#4349)
kolchfa-aws c30af78
Update links in field types index (#4379)
kolchfa-aws a2cf71e
updatestomigrationdoc (#4343)
hdhalter fe7943f
Writing processors backlog documentation
vagimeli 47bbb69
Writing processors backlog documentation
vagimeli fabc15e
Writing processors backlog documentation
vagimeli 2d7205e
Writing processors backlog documentation
vagimeli bc20602
Writing processors backlog documentation
vagimeli 1827fae
Writing processors backlog documentation
vagimeli 765c287
Writing processors backlog documentation
vagimeli 417373a
Writing processors backlog documentation
vagimeli 3e73879
Writing processors backlog documentation
vagimeli f499ddb
Writing processors backlog documentation
vagimeli 1451142
Writing processors backlog documentation
vagimeli 8605786
Writing processors backlog documentation
vagimeli f338778
Writing processors backlog documentation
vagimeli 0fa8fd6
Revert "updatestomigrationdoc (#4343)"
vagimeli 9f6913c
Revert "Update links in field types index (#4379)"
vagimeli 44533ad
Revert "Reformat supported field types index page (#4349)"
vagimeli 5756737
Revert "Update nested.md (#4363)"
vagimeli e50c25b
Revert "Add model access control documentation for ML Commons (#4223)"
vagimeli 2a2821f
Revert "Add date nanoseconds field type (#4348)"
vagimeli 05ae4aa
Revert "Update plugins.md (#4353)"
vagimeli d282c22
Revert "Update searchable snapshot documentation to be more correct (…
vagimeli 8e88c6e
Update ingest-processors.md
vagimeli 478bc16
Move files to separate PRs
vagimeli 714e2ba
Move files to separate PRs
vagimeli a23449e
Update ingest-processors.md
vagimeli 3495404
Update information architecture
vagimeli 55cfa3b
Add first draft remove processor
vagimeli a907c5f
Add first draft remove processor
vagimeli 6db6827
Address tech review feedback
vagimeli a89ca6a
Address tech review feedback
vagimeli 7f7c807
Address tech review feedback
vagimeli 2ae1cc4
Address tech review feedback
vagimeli ad8133b
Address tech review feedback
vagimeli ae1c7e8
Update _api-reference/ingest-apis/ingest-processors.md
vagimeli 18c5d5e
Update _api-reference/ingest-apis/ingest-processors.md
vagimeli 428bfd9
Update ingest-processors.md
vagimeli 7c2a1d9
Update ingest-processors.md
vagimeli b0060da
Address tech review feedback
vagimeli 85b57a7
Update pipeline example
vagimeli 308ba02
Update pipeline example
vagimeli 057a146
Update pipeline example
vagimeli 1a60a13
Update pipeline example
vagimeli 2d36887
Update pipeline example
vagimeli 5fd0a64
Update pipeline example
vagimeli 37fa2c0
Update pipeline example
vagimeli 1b1535c
Update pipeline example
vagimeli 807431c
Update pipeline example
vagimeli 444123d
Update pipeline example
vagimeli 9612019
Update pipeline example
vagimeli 8c97658
Update pipeline example
vagimeli 3f66760
Update append.md
vagimeli 5dd3154
Copy edits
vagimeli 56be338
Update _api-reference/ingest-apis/ingest-processors.md
vagimeli b48f124
Update _api-reference/ingest-apis/ingest-processors.md
vagimeli 3825a13
Update _api-reference/ingest-apis/ingest-processors.md
vagimeli a3940bf
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli f907ec9
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli 5bf0146
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli 07c0776
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli d2e0aa8
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli 0ca8746
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli 5c14e8c
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli dc86e10
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli 34bbd60
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli 3ed7c02
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli ee1dbb5
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli badbb42
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli 11814e1
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli 71f249c
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli dedc3f7
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli a4b23af
Update _api-reference/ingest-apis/processors-reference/append.md
vagimeli 86af726
Update _api-reference/ingest-apis/processors-reference/bytes.md
vagimeli 3ea45e8
Update _api-reference/ingest-apis/processors-reference/bytes.md
vagimeli 5bcf36e
Update _api-reference/ingest-apis/processors-reference/bytes.md
vagimeli 41cce56
Update _api-reference/ingest-apis/processors-reference/bytes.md
vagimeli 67a2c29
Update _api-reference/ingest-apis/processors-reference/bytes.md
vagimeli ef609b1
Update _api-reference/ingest-apis/processors-reference/convert.md
vagimeli 6d4eae3
Update _api-reference/ingest-apis/processors-reference/convert.md
vagimeli 392cef1
Update _api-reference/ingest-apis/processors-reference/convert.md
vagimeli 97e0df5
Update _api-reference/ingest-apis/processors-reference/csv.md
vagimeli 9b3fd4d
Update _api-reference/ingest-apis/processors-reference/csv.md
vagimeli 07c78de
Update _api-reference/ingest-apis/processors-reference/date.md
vagimeli 04b7f33
Update _api-reference/ingest-apis/processors-reference/date.md
vagimeli 26de00d
Update _api-reference/ingest-apis/processors-reference/date.md
vagimeli 523c441
Continue writing
vagimeli 5166d1f
Continue writing
vagimeli 26b0dc2
Continue writing
vagimeli f3dcef1
Continue writing
vagimeli 67a4d50
Revised to include SME feedback
vagimeli ee0d394
Update _api-reference/ingest-apis/ingest-pipelines.md
vagimeli dd17c5d
Update _api-reference/ingest-apis/ingest-pipelines.md
vagimeli 5276b90
Update _api-reference/ingest-apis/ingest-pipelines.md
vagimeli 91ef17b
Update _api-reference/ingest-apis/ingest-pipelines.md
vagimeli ca51a56
Update _api-reference/ingest-apis/processors-reference/date.md
vagimeli 1f1a0d4
Address SME and doc reviewer input
vagimeli 6b28932
Add simulate content
vagimeli 0385a7e
Writing
vagimeli 3207dad
Writing
vagimeli ce5c972
Copy edits
vagimeli ea6110f
Writing
vagimeli cb908a9
Writing
vagimeli 4d078ae
Revised content
vagimeli 042204e
copy edit
vagimeli 3a38b2e
Update file name
vagimeli 4e4a932
Update file name
vagimeli f2622a8
Update file name
vagimeli 41562a5
Update file name
vagimeli ec4a354
Update file name
vagimeli fdc6a1b
Update file name
vagimeli 55861a8
Add bytes content and examples
vagimeli 9591dcc
Add bytes content and examples
vagimeli ef4cb53
Testing code snippets and optional configuration parameters
vagimeli 5aae19f
Testing code snippets and optional configuration parameters
vagimeli 5f6dd0e
Update _api-reference/ingest-apis/create-ingest.md
vagimeli f91ccf5
Update _api-reference/ingest-apis/create-ingest.md
vagimeli 94a86c3
QA pipeline testing
vagimeli 05e3ed6
Update _api-reference/ingest-apis/pipeline-failures.md
vagimeli 74fb7f1
Address SME comments
vagimeli 96616d8
Add copy label
vagimeli 88d3805
Update append.md
vagimeli 4412bfc
Fix broken links
vagimeli 3766c7b
Fix broken links
vagimeli dac1701
Update create-ingest.md
vagimeli 17d4211
Update append.md
vagimeli 417242c
Update bytes.md
vagimeli d95391c
Update convert.md
vagimeli b253f3a
Update convert.md
vagimeli 28482a9
Update csv.md
vagimeli 7b48521
Update date.md
vagimeli e51b1f7
Update lowercase.md
vagimeli 26e7db0
Update remove.md
vagimeli 06ad0e1
Update uppercase.md
vagimeli 753ea67
Update append.md
vagimeli 9548191
Update bytes.md
vagimeli 2280597
Update convert.md
vagimeli 1cda6ba
Update csv.md
vagimeli d7d3e68
Update date.md
vagimeli 64d4d30
Update lowercase.md
vagimeli de5c1d0
Update remove.md
vagimeli 9c76cc1
Update uppercase.md
vagimeli 056ea7b
Address doc review feedback
vagimeli 7e096db
Copy edits
vagimeli 55357a8
Copy edits
vagimeli 14294a3
Copy edits
vagimeli c879c9d
Copy edits
vagimeli 9b467cc
Revise order of steps
vagimeli 12347ec
Update processor steps
vagimeli a8f4ceb
Copy edits
vagimeli 9d29c86
Update _api-reference/ingest-apis/create-ingest.md
vagimeli 6fe30f5
Update _api-reference/ingest-apis/create-ingest.md
vagimeli a20ac44
Update _api-reference/ingest-apis/create-ingest.md
vagimeli fc95582
Update _api-reference/ingest-apis/create-ingest.md
vagimeli 80eb62f
Update _api-reference/ingest-apis/create-ingest.md
vagimeli 967bf9c
Update _api-reference/ingest-apis/create-ingest.md
vagimeli aafd5d0
Update _api-reference/ingest-apis/delete-ingest.md
vagimeli 7f0a5ff
Update _api-reference/ingest-apis/index.md
vagimeli 9402e7b
Update _api-reference/ingest-apis/ingest-pipelines.md
vagimeli 47a7040
Update _api-reference/ingest-apis/ingest-pipelines.md
vagimeli 65ba5e0
Update _api-reference/ingest-apis/ingest-pipelines.md
vagimeli f531945
Update _api-reference/ingest-apis/ingest-pipelines.md
vagimeli 32846b1
Update _api-reference/ingest-apis/ingest-pipelines.md
vagimeli 904f386
Update _api-reference/ingest-apis/ingest-pipelines.md
vagimeli f4add7c
Update _api-reference/ingest-apis/ingest-pipelines.md
vagimeli 69a05d8
Update _api-reference/ingest-apis/ingest-processors.md
vagimeli c405e98
Update _api-reference/ingest-apis/pipeline-failures.md
vagimeli 63effc5
Update _api-reference/ingest-apis/pipeline-failures.md
vagimeli 1dbf2a6
Update _api-reference/ingest-apis/processors/append.md
vagimeli baad585
Update _api-reference/ingest-apis/processors/append.md
vagimeli ced1072
Update _api-reference/ingest-apis/processors/append.md
vagimeli 4065a79
Update _api-reference/ingest-apis/processors/append.md
vagimeli 761c6c6
Update _api-reference/ingest-apis/processors/append.md
vagimeli c73de52
Update _api-reference/ingest-apis/processors/append.md
vagimeli e3616bc
Update _api-reference/ingest-apis/processors/append.md
vagimeli 3f8e587
Update _api-reference/ingest-apis/processors/append.md
vagimeli 867a2ba
Update _api-reference/ingest-apis/processors/append.md
vagimeli 8a7e49f
Update _api-reference/ingest-apis/processors/append.md
vagimeli 1e235c7
Update _api-reference/ingest-apis/processors/append.md
vagimeli fd30afb
Update _api-reference/ingest-apis/processors/bytes.md
vagimeli ae5ab17
Update _api-reference/ingest-apis/processors/bytes.md
vagimeli 9985ddd
Update _api-reference/ingest-apis/processors/bytes.md
vagimeli b6b9f22
Update _api-reference/ingest-apis/processors/bytes.md
vagimeli e53d1cc
Address editorial feedback
vagimeli f588107
Update _api-reference/ingest-apis/processors/convert.md
vagimeli da0a4b6
Update _api-reference/ingest-apis/processors/convert.md
vagimeli 07ac669
Update _api-reference/ingest-apis/processors/convert.md
vagimeli bf9b669
Update _api-reference/ingest-apis/processors/convert.md
vagimeli 1aa34dc
Update _api-reference/ingest-apis/processors/convert.md
vagimeli a166ab1
Update _api-reference/ingest-apis/processors/convert.md
vagimeli e19c406
Update _api-reference/ingest-apis/processors/convert.md
vagimeli 9ae3dc7
Address editorial feedback
vagimeli bde9ba9
Address editorial changes
vagimeli 9e62879
Update _api-reference/ingest-apis/processors/convert.md
vagimeli be5056c
Update _api-reference/ingest-apis/processors/csv.md
vagimeli 8212b0a
Update _api-reference/ingest-apis/processors/csv.md
vagimeli 482b854
Update _api-reference/ingest-apis/processors/csv.md
vagimeli c9310ca
Update _api-reference/ingest-apis/processors/csv.md
vagimeli 9967966
Update _api-reference/ingest-apis/processors/csv.md
vagimeli 042207e
Update _api-reference/ingest-apis/processors/csv.md
vagimeli ca48924
Update _api-reference/ingest-apis/processors/csv.md
vagimeli c282647
Address editorial feedback
vagimeli 2ade25c
Address editorial feedback
vagimeli 60af276
Update _api-reference/ingest-apis/processors/date.md
vagimeli ce1030c
Update _api-reference/ingest-apis/processors/date.md
vagimeli 2fc7870
Update _api-reference/ingest-apis/processors/date.md
vagimeli eecc24a
Update _api-reference/ingest-apis/processors/date.md
vagimeli 88a4373
Update _api-reference/ingest-apis/processors/date.md
vagimeli 0e92ea4
Update _api-reference/ingest-apis/processors/date.md
vagimeli e23e4d7
Update _api-reference/ingest-apis/processors/date.md
vagimeli 11a591e
Update _api-reference/ingest-apis/processors/date.md
vagimeli 0f5b034
Update _api-reference/ingest-apis/processors/lowercase.md
vagimeli 749a549
Update _api-reference/ingest-apis/processors/lowercase.md
vagimeli 4beaca8
Update _api-reference/ingest-apis/processors/lowercase.md
vagimeli 04f4fac
Update _api-reference/ingest-apis/processors/lowercase.md
vagimeli 7ce2840
Update _api-reference/ingest-apis/processors/lowercase.md
vagimeli d3dc5cf
Update _api-reference/ingest-apis/processors/lowercase.md
vagimeli 5c34ca7
Update _api-reference/ingest-apis/processors/lowercase.md
vagimeli 3e3fa5b
Update _api-reference/ingest-apis/processors/lowercase.md
vagimeli eb258db
Address editorial feedback
vagimeli bf8e9a6
Update _api-reference/ingest-apis/processors/remove.md
vagimeli ecd54df
Update _api-reference/ingest-apis/processors/remove.md
vagimeli 1853ec2
Update _api-reference/ingest-apis/processors/remove.md
vagimeli b528834
Update _api-reference/ingest-apis/processors/remove.md
vagimeli 1a32fc9
Update _api-reference/ingest-apis/processors/remove.md
vagimeli ef80080
Update _api-reference/ingest-apis/processors/remove.md
vagimeli 9ff0234
Address editorial review feedback
vagimeli 98726c7
Update _api-reference/ingest-apis/processors/uppercase.md
vagimeli 93d3028
Update _api-reference/ingest-apis/processors/uppercase.md
vagimeli d2e6261
Update _api-reference/ingest-apis/processors/uppercase.md
vagimeli 115ca45
Update _api-reference/ingest-apis/processors/uppercase.md
vagimeli 55b45ac
Update _api-reference/ingest-apis/processors/uppercase.md
vagimeli 3869d73
Update _api-reference/ingest-apis/processors/uppercase.md
vagimeli 477b5b5
Update _api-reference/ingest-apis/processors/uppercase.md
vagimeli 2d6281a
Address editorial review feedback
vagimeli 9445229
Address editorial review feedback
vagimeli f7d8a6d
Address editorial review feedback
vagimeli 09bc013
Address editorial review feedback
vagimeli 3cf22cd
Address editorial review feedback
vagimeli 74f887b
Address editorial review feedback
vagimeli 36f3c53
Address editorial review feedback
vagimeli 7e9c29d
Update _api-reference/ingest-apis/processors/append.md
vagimeli File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| --- | ||
| layout: default | ||
| title: Create pipeline | ||
| parent: Ingest pipelines | ||
| grand_parent: Ingest APIs | ||
| nav_order: 10 | ||
| redirect_from: | ||
| - /opensearch/rest-api/ingest-apis/create-update-ingest/ | ||
| --- | ||
|
|
||
| # Create pipeline | ||
|
|
||
| Use the create pipeline API operation to create or update pipelines in OpenSearch. Note that the pipeline requires you to define at least one processor that specifies how to change the documents. | ||
|
|
||
| ## Path and HTTP method | ||
|
|
||
| Replace `<pipeline-id>` with your pipeline ID: | ||
|
|
||
| ```json | ||
| PUT _ingest/pipeline/<pipeline-id> | ||
| ``` | ||
| #### Example request | ||
|
|
||
| Here is an example in JSON format that creates an ingest pipeline with two `set` processors and an `uppercase` processor. The first `set` processor sets the `grad_year` to `2023`, and the second `set` processor sets `graduated` to `true`. The `uppercase` processor converts the `name` field to uppercase. | ||
|
|
||
| ```json | ||
| PUT _ingest/pipeline/my-pipeline | ||
| { | ||
| "description": "This pipeline processes student data", | ||
| "processors": [ | ||
| { | ||
| "set": { | ||
| "description": "Sets the graduation year to 2023", | ||
| "field": "grad_year", | ||
| "value": 2023 | ||
| } | ||
| }, | ||
| { | ||
| "set": { | ||
| "description": "Sets graduated to true", | ||
| "field": "graduated", | ||
| "value": true | ||
| } | ||
| }, | ||
| { | ||
| "uppercase": { | ||
| "field": "name" | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
| {% include copy-curl.html %} | ||
|
|
||
| To learn more about error handling, see [Handling pipeline failures]({{site.url}}{{site.baseurl}}/api-reference/ingest-apis/pipeline-failures/). | ||
|
|
||
| ## Request body fields | ||
|
|
||
| The following table lists the request body fields used to create or update a pipeline. | ||
|
|
||
| Parameter | Required | Type | Description | ||
| :--- | :--- | :--- | :--- | ||
| `processors` | Required | Array of processor objects | An array of processors, each of which transforms documents. Processors are run sequentially in the order specified. | ||
| `description` | Optional | String | A description of your ingest pipeline. | ||
|
|
||
| ## Path parameters | ||
|
|
||
| Parameter | Required | Type | Description | ||
| :--- | :--- | :--- | :--- | ||
| `pipeline-id` | Required | String | The unique identifier, or pipeline ID, assigned to the ingest pipeline. | ||
|
|
||
| ## Query parameters | ||
|
|
||
| Parameter | Required | Type | Description | ||
| :--- | :--- | :--- | :--- | ||
| `cluster_manager_timeout` | Optional | Time | Period to wait for a connection to the cluster manager node. Defaults to 30 seconds. | ||
| `timeout` | Optional | Time | Period to wait for a response. Defaults to 30 seconds. | ||
|
|
||
| ## Template snippets | ||
|
|
||
| Some processor parameters support [Mustache](https://mustache.github.io/) template snippets. To get the value of a field, surround the field name in three curly braces, for example, `{% raw %}{{{field-name}}}{% endraw %}`. | ||
|
|
||
| #### Example: `set` ingest processor using Mustache template snippet | ||
|
|
||
| The following example sets the field `{% raw %}{{{role}}}{% endraw %}` with a value `{% raw %}{{{tenure}}}{% endraw %}`: | ||
|
|
||
| ```json | ||
| PUT _ingest/pipeline/my-pipeline | ||
| { | ||
vagimeli marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| "processors": [ | ||
| { | ||
| "set": { | ||
| "field": "{% raw %}{{{role}}}{% endraw %}", | ||
| "value": "{% raw %}{{{tenure}}}{% endraw %}" | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
| {% include copy-curl.html %} | ||
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,44 +1,27 @@ | ||
| --- | ||
| layout: default | ||
| title: Delete a pipeline | ||
| parent: Ingest APIs | ||
| nav_order: 14 | ||
| title: Delete pipeline | ||
| parent: Ingest pipelines | ||
| grand_parent: Ingest APIs | ||
| nav_order: 13 | ||
| redirect_from: | ||
| - /opensearch/rest-api/ingest-apis/delete-ingest/ | ||
| --- | ||
|
|
||
| # Delete a pipeline | ||
| # Delete pipeline | ||
|
|
||
| If you no longer want to use an ingest pipeline, use the delete ingest pipeline API operation. | ||
| Use the following request to delete a pipeline. | ||
|
|
||
| ## Example | ||
| To delete a specific pipeline, pass the pipeline ID as a parameter: | ||
|
|
||
| ``` | ||
| DELETE _ingest/pipeline/12345 | ||
| ```json | ||
| DELETE /_ingest/pipeline/<pipeline-id> | ||
| ``` | ||
| {% include copy-curl.html %} | ||
|
|
||
| ## Path and HTTP methods | ||
|
|
||
| Delete an ingest pipeline based on that pipeline's ID. | ||
|
|
||
| ``` | ||
| DELETE _ingest/pipeline/ | ||
| ``` | ||
|
|
||
| ## URL parameters | ||
|
|
||
| All URL parameters are optional. | ||
|
|
||
| Parameter | Type | Description | ||
| :--- | :--- | :--- | ||
| master_timeout | time | How long to wait for a connection to the master node. | ||
| timeout | time | How long to wait for the request to return. | ||
|
|
||
| ## Response | ||
| To delete all pipelines in a cluster, use the wildcard character (`*`): | ||
|
|
||
| ```json | ||
| { | ||
| "acknowledged" : true | ||
| } | ||
| ``` | ||
| DELETE /_ingest/pipeline/* | ||
| ``` | ||
| {% include copy-curl.html %} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,59 +1,62 @@ | ||
| --- | ||
| layout: default | ||
| title: Get ingest pipeline | ||
| parent: Ingest APIs | ||
| nav_order: 10 | ||
| title: Get pipeline | ||
| parent: Ingest pipelines | ||
| grand_parent: Ingest APIs | ||
| nav_order: 12 | ||
| redirect_from: | ||
| - /opensearch/rest-api/ingest-apis/get-ingest/ | ||
| --- | ||
|
|
||
| ## Get ingest pipeline | ||
| # Get pipeline | ||
|
|
||
| After you create a pipeline, use the get ingest pipeline API operation to return all the information about a specific ingest pipeline. | ||
| Use the get ingest pipeline API operation to retrieve all the information about the pipeline. | ||
|
|
||
| ## Example | ||
| ## Retrieving information about all pipelines | ||
|
|
||
| ``` | ||
| GET _ingest/pipeline/12345 | ||
| The following example request returns information about all ingest pipelines: | ||
|
|
||
| ```json | ||
| GET _ingest/pipeline/ | ||
| ``` | ||
| {% include copy-curl.html %} | ||
|
|
||
| ## Path and HTTP methods | ||
| ## Retrieving information about a specific pipeline | ||
|
|
||
| Return all ingest pipelines. | ||
| The following example request returns information about a specific pipeline, which for this example is `my-pipeline`: | ||
|
|
||
| ```json | ||
| GET _ingest/pipeline/my-pipeline | ||
| ``` | ||
| GET _ingest/pipeline | ||
| ``` | ||
|
|
||
| Returns a single ingest pipeline based on the pipeline's ID. | ||
|
|
||
| ``` | ||
| GET _ingest/pipeline/{id} | ||
| ``` | ||
|
|
||
| ## URL parameters | ||
|
|
||
| All parameters are optional. | ||
|
|
||
| Parameter | Type | Description | ||
| :--- | :--- | :--- | ||
| master_timeout | time | How long to wait for a connection to the master node. | ||
| {% include copy-curl.html %} | ||
|
|
||
| ## Response | ||
| The response contains the pipeline information: | ||
|
|
||
| ```json | ||
| { | ||
| "pipeline-id" : { | ||
| "description" : "A description for your pipeline", | ||
| "processors" : [ | ||
| "my-pipeline": { | ||
| "description": "This pipeline processes student data", | ||
| "processors": [ | ||
| { | ||
| "set" : { | ||
| "field" : "field-name", | ||
| "value" : "value" | ||
| "set": { | ||
| "description": "Sets the graduation year to 2023", | ||
| "field": "grad_year", | ||
| "value": 2023 | ||
| } | ||
| }, | ||
| { | ||
| "set": { | ||
| "description": "Sets graduated to true", | ||
| "field": "graduated", | ||
| "value": true | ||
| } | ||
| }, | ||
| { | ||
| "uppercase": { | ||
| "field": "name" | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| ``` | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.