Partitioned Distinct/DistinctCount

[jasperjiaguo]

For high cardinality columns, the local/intermediate/global merging phase of distinct(count) can be pretty memory/cpu heavy as the merger will need to ser/de and merge multiple large sets from the responses. In this case, if the distinct(count) column is partitioned into disjoint sets, then the merger can simply concat (for distinct) or add (for distinctcount) the intermediate results. This change can significantly reduce the set ser/de, transmission, and merge time/memory footprint. Meanwhile, it can be applicable to different levels of the processing depending on the partition granularity.

[Jackie-Jiang]

For some context, #9304 can be leveraged to achieve this as long as broker knows the segments are partitioned

cc @61yao

[walterddr]

as a longer-term solution we are also planning to change the dispatch mechanism in routing manager to include partition to segment list mapping so this decision can be made on server. see #9611 (partition dispatch)