[FEATURE] Support input with multiple categories

[joshuali925]

Is your feature request related to a problem?
This is for AD command in PPL. Sometimes the inputs will have a tag/category, and data within each category should be separately analyzed. For example

[
  {"category": "A", "value": 1},
  {"category": "A", "value": 2},
  {"category": "B", "value": 20},
  {"category": "B", "value": 40}
]

And user would like AD to evaluate by category, but source=nyc_taxi | fields category, value | AD doesn't work.

What solution would you like?
Support source=nyc_taxi | fields category, value | AD category_field='category' or something similar so that input is categorized then predicted. The category value is irrelevant for prediction.

What alternatives have you considered?
Temp workaround in PPL opensearch-project/sql#952

Do you have any additional context?
I'll try to work on this before 2.4 release, but not sure if i have the time

[ylwu-amzn]

@joshuali925 , hi, Joshua, this is a good feature. If you have bandwidth, welcome to contribute!
I can see some challenges by supporting such categorized input data.

1. Memory pressure. Multi-category will bring more data, that will increase memory usage. I think we should limit category numbers in request.
2. Latency. If we run each category one by one, the latency will be linearly increased. We can run several categories in parallel to speed up.
3. How to support multiple category fields, for example {"category1": "A","category2": "B", "value": 1},. Maybe we can start from supporting only 1 category field.