[FEATURE] add processors: set: copy_from

[anubisg1]

Describe the bug
With Elasticsearch 7.11 (i.e. the version immediately after OpenSearch fork), "copy_from" option for the "set" processor has been introduced, here - https://www.elastic.co/guide/en/elasticsearch/reference/7.11/set-processor.html

this breaks a lot of pipelines, registering pipelines will throw the following error:

Exiting: 1 error: error loading pipeline for fileset system/auth: couldn't load pipeline: couldn't load json. Error: 400 Bad Request: {"error":{"root_cause":[{"type":"parse_exception","reason":"[value] required property is missing","processor_type":"set","property_name":"value"}],"type":"parse_exception","reason":"[value] required property is missing","processor_type":"set","property_name":"value"},"status":400}. Response body: {"error":{"root_cause":[{"type":"parse_exception","reason":"[value] required property is missing","processor_type":"set","property_name":"value"}],"type":"parse_exception","reason":"[value] required property is missing","processor_type":"set","property_name":"value"},"status":400}

a workaround is available and requires to change

processors:
  - set:
      field: myfiled
      copy_from: myvalue

to

processors:
- set:
    field: myfiled
    value: '{{myvalue}}'

[dblock]

Are you suggesting we implement copy_from in OpenSearch? Is this a bug in OpenSearch?

Please feel free to contribute an implementation without looking at/copying any code from Elasticseach.

[anubisg1]

i am asking to implement something that allows compatibility with pipelines created for elasticsearch 7.11 and newer and use copy_from

i'm not asking to copy their implementation of that field .

happy to help in any way, but i am not a software developer unfortunately so i wouldn't know where to start

[fbarbeira]

I arrived here searching how to properly rename a field on a ingest pipeline using ingest processors.

Previously on Elasticsearch I used to apply rename processor but it seems like it is not available on Opensearch. Then I tried to find another approach and landed here. Does it exist a supported "out of the box" solution to rename a field on Opensearch?

Please tell me there is a better solution than this:

{
  "description": "Renames a field using a script",
  "processors": [
    {
      "script": {
        "source": """
          if (ctx.containsKey('old_field_name')) {
            ctx['new_field_name'] = ctx['old_field_name'];
            ctx.remove('old_field_name');
          }
        """
      }
    }
  ]
}

[fbarbeira]

OMG the dev tools suggest me to use "rename":

I dit not even tried to use that because It does not appear on the official doc section.

It seems like it worked! :) maybe it should be added to the documentation?