[C++] String concatenate aggregate kernel

[asfimport]

Like MySQL/Impala group_concat and PostgreSQL string_agg. Takes a string array and a separator (possibly optional?) and returns one scalar string (one per group in the case of group aggregation) representing all the string values in the array concatenated together, with the separator added between each pair of concatenated values.

For example, in the case of no grouping and using separator "-", this would take input:

Array<string>
[ 
  "foo",
  "bar",
  "baz"
]

and return the following string scalar as output:

"foo-bar-baz"

 

Reporter: Ian Cook / @ianmcook

Related issues:

• [C++][Compute] Support tagging ExecBatches with arbitrary extra information (is blocked by)
• [C++] String algorithm library for StringArray/BinaryArray (is a child of)
• [C++] Implement hash_aggregate kernels (umbrella issue) (is a child of)
• [R] Bindings for paste(collapse), str_c(collapse), and str_flatten() (is depended upon by)

_{Note: This issue was originally created as ARROW-12710. Please see the migration documentation for further details.}

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
Is this already covered by ARROW-10959? (or I don't fully understand the difference)

In Python, you can do a simple concatenation of an array of strings with the sum()} function, or using {{str.join for using a separator:

>>> ",".join(['a', 'b', 'c'])
'a,b,c'

[asfimport]

Ian Cook / @ianmcook:
@jorisvandenbossche  no, this is different from ARROW-10959. I added examples to the descriptions of all these string join kernel issues to make the distinctions between them clearer.

[asfimport]

Niranda Perera / @nirandaperera:
@ianmcook I checked the postgres string_agg (https://www.postgresqltutorial.com/postgresql-aggregate-functions/postgresql-string_agg-function/). It seems to have an order_by option. Do you we need that part as well? If not, AFAIU, this would be a straight-forward scalar agg kernel, and I can take it up. :)

[asfimport]

Antoine Pitrou / @pitrou:
Is our current scalar aggregation has deterministic output order, then we don't need the "order_by" option.

[asfimport]

Ben Kietzman / @bkietz:
Our current scalar aggregation has deterministic ordering, but that will fall down rapidly when those functions are used in an ExecPlan (where no ordering is guaranteed).

If we implement this as a ScalarAggregateKernel, the KernelState of the string concat agg kernel will need to include ordering criteria so that merge(move(state1), &state0) can be guaranteed equivalent to merge(move(state0), &state1). Furthermore, merge cannot actually concatenate anything because if we happened to first merge(move(state0), &state3) we'd have no way to insert state1, state2 in the middle later. Actual concatenation would have to wait for finalize.

Those ordering criteria could be synthesized from (for example) fragment/batch indices in a dataset scan, but the presence of O(N) state in a scalar agg kernel's State is suspect to me and I'm not sure it's a great match for ScalarAggregateKernel.

[asfimport]

Antoine Pitrou / @pitrou:
This is actually doable now by passing a ListScalar to the binary_join compute function. For example in Python:

>>> arr = pa.array(["abc", "def", "ghi"])
>>> scal = pa.ListArray.from_arrays([0, len(arr)], arr)[0]
>>> scal
<pyarrow.ListScalar: ['abc', 'def', 'ghi']>
>>> pc.binary_join(scal, "-")
<pyarrow.StringScalar: 'abc-def-ghi'>

(note that creating the ListScalar is a bit cumbersome in Python, I filed ARROW-16254 for this)

@ianmcook Can we then close this issue?