You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -14,44 +14,126 @@ The `copy_values` processor copies values within an event and is a [mutate event
14
14
15
15
You can configure the `copy_values` processor with the following options.
16
16
17
-
| Option | Required | Description |
18
-
:--- | :--- | :---
19
-
|`entries`| Yes | A list of entries to be copied in an event. |
20
-
|`from_key`| Yes | The key of the entry to be copied. |
21
-
|`to_key`| Yes | The key of the new entry to be added. |
22
-
|`overwrite_if_to_key_exists`| No | When set to `true`, the existing value is overwritten if `key` already exists in the event. The default value is `false`. |
17
+
| Option | Required | Type | Description |
18
+
:--- | :--- | :--- | :---
19
+
|`entries`| Yes |[entry](#entry)| A list of entries to be copied in an event. See [entry](#entry) for more information. |
20
+
|`from_list`| No | String | The key for the list of objects to be copied. |
21
+
|`to_list`| No | String | The key for the new list to be added. |
22
+
|`overwrite_if_to_list_exists`| No | Boolean | When set to `true`, the existing value is overwritten if the `key` specified by `to_list` already exists in the event. Default is `false`. |
23
+
24
+
## entry
25
+
26
+
For each entry, you can configure the following options.
27
+
28
+
| Option | Required | Type | Description |
29
+
:--- | :--- | :--- | :---
30
+
|`from_key`| Yes | String | The key for the entry to be copied. |
31
+
|`to_key`| Yes | String | The key for the new entry to be added. |
32
+
|`overwrite_if_to_key_exists`| No | Boolean | When set to `true`, the existing value is overwritten if the `key` already exists in the event. Default is `false`. |
33
+
23
34
24
35
## Usage
25
36
26
-
To get started, create the following `pipeline.yaml` file:
37
+
The following examples show you how to use the `copy_values` processor.
38
+
39
+
### Example: Copy values and skip existing fields
40
+
41
+
The following example shows you how to configure the processor to copy values and skip existing fields:
42
+
43
+
```yaml
44
+
...
45
+
processor:
46
+
- copy_values:
47
+
entries:
48
+
- from_key: "message1"
49
+
to_key: "message2"
50
+
- from_key: "message1"
51
+
to_key: "message3"
52
+
...
53
+
```
54
+
{% include copy.html %}
55
+
56
+
When the input event contains the following data:
57
+
58
+
```json
59
+
{"message1": "hello", "message2": "bye"}
60
+
```
61
+
62
+
The processor copies "message1" to "message3" but not to "message2" because "message2" already exists. The processed event contains the following data:
The following example shows you how to configure the processor to copy values:
27
71
28
72
```yaml
29
-
pipeline:
30
-
source:
31
-
...
32
-
....
73
+
...
33
74
processor:
34
75
- copy_values:
35
76
entries:
36
-
- from_key: "message"
37
-
to_key: "newMessage"
38
-
overwrite_if_to_key_exists: true
39
-
sink:
77
+
- from_key: "message1"
78
+
to_key: "message2"
79
+
overwrite_if_to_key_exists: true
80
+
- from_key: "message1"
81
+
to_key: "message3"
82
+
...
40
83
```
41
84
{% include copy.html %}
42
85
43
-
Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with that filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper).
86
+
When the input event contains the following data:
87
+
88
+
```json
89
+
{"message1": "hello", "message2": "bye"}
90
+
```
44
91
45
-
For example, before you run the `copy_values` processor, if the `logs_json.log` file contains the following event record:
92
+
The processor copies "message1" to both "message2" and "message3", overwriting the existing value in "message2". The processed event contains the following data:
0 commit comments