Skip to content

Commit bbe8135

Browse files
DOCSP-10823 improve out to s3 (#36)
* DOCSP-10823 improve out to s3
1 parent 8551574 commit bbe8135

File tree

1 file changed

+17
-9
lines changed

1 file changed

+17
-9
lines changed

source/reference/pipeline/out.txt

Lines changed: 17 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -167,7 +167,10 @@ Fields
167167
:ref:`created dynamically <adl-out-stage-egs>` from the
168168
fields in the documents that reach the |out| stage. Any
169169
filename expression you provide must evaluate to a
170-
``string`` data type.
170+
``string`` data type. If there are any files on |s3|
171+
with the same name and path as the newly generated files,
172+
|out| overwrites the existing files with the newly
173+
generated files.
171174

172175
.. important::
173176

@@ -408,7 +411,7 @@ Examples
408411
"$concat": [
409412
"persons/",
410413
"$name", "/",
411-
"$unique-id"
414+
"$unique-id", "/"
412415
]
413416
},
414417
"format": {
@@ -423,9 +426,11 @@ Examples
423426
file, |out| concatenates:
424427

425428
- A constant string ``persons/`` and, from the documents:
429+
426430
- The string value of the ``name`` field,
427-
- A forward slash (``/``), and
428-
- The string value of the ``unique-id`` field.
431+
- A forward slash (``/``),
432+
- The string value of the ``unique-id`` field, and
433+
- A forward slash (``/``).
429434

430435
Each |bson| file contains all of the documents with the same
431436
``name`` and ``unique-id`` values. |out| names each file using
@@ -458,7 +463,7 @@ Examples
458463
{
459464
"$toString": "$sale-date"
460465
}, "/",
461-
"$part-id"
466+
"$part-id", "/"
462467
]
463468
},
464469
"format": {
@@ -474,11 +479,14 @@ Examples
474479
``sale-date``, and ``part-id`` values. To name each file, |out|
475480
concatenates:
476481

477-
- A constant string value of ``big-box-store``,
482+
- A constant string value of ``big-box-store/``,
478483
- A string value of a unique store number in the
479484
``store-number`` field,
480-
- A string value of the date from the ``sale-date`` field, and
481-
- A string value of part ID from the ``part-id`` field.
485+
- A forward slash (``/``),
486+
- A string value of the date from the ``sale-date`` field,
487+
- A forward slash (``/``),
488+
- A string value of part ID from the ``part-id`` field, and
489+
- A forward slash (``/``).
482490

483491
.. tab:: Atlas Cluster
484492
:tabid: atlas
@@ -541,7 +549,7 @@ Examples
541549
"to": "string",
542550
"onNull": ""
543551
}
544-
}
552+
}, "/"
545553
]
546554
},
547555
"format": {

0 commit comments

Comments
 (0)