generated from amazon-archives/__template_Custom
-
Notifications
You must be signed in to change notification settings - Fork 176
Closed
Labels
PPLPiped processing languagePiped processing languageenhancementNew feature or requestNew feature or requestv3.3.0
Description
Is your feature request related to a problem?
Currently max(<value>)
and min(<value>)
in PPL behave numerically and don’t match SPL’s behavior on non-numeric and mixed type fields.
What solution would you like?
- Align PPL
min
/max
function with SPL's ordering rule in https://help.splunk.com/en/splunk-enterprise/search/spl-search-reference/10.0/statistical-and-charting-functions/aggregate-functions#usage-6. - Pushdown optimization: DSL min/max aggregation doesn't support non-numeric fields. Probably we need to decide to pushdown to min/max or script aggregation based on input type [TBD].
What alternatives have you considered?
N/A
Do you have any additional context?
Here is the test result with PPL Calcite v3:
source=accounts | stats max(gender);
TransportError(500, 'SearchPhaseExecutionException', {'error': {'reason':
'Error occurred in OpenSearch engine: all shards failed', 'details': 'Shard[0]: java.lang.IllegalArgumentException:
Field [gender.keyword] of type [keyword] is not supported for aggregation [max]\n\n
For more details, please send request for Json format to see the raw response from OpenSearch engine.',
'type': 'SearchPhaseExecutionException'}, 'status': 400})
Caused by: java.lang.IllegalArgumentException: Field [gender.keyword] of type [keyword] is not supported for aggregation [max]
at org.opensearch.search.aggregations.support.ValuesSourceRegistry.getAggregator(ValuesSourceRegistry.java:202) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.search.aggregations.support.ValuesSourceAggregationBuilder.doBuild(ValuesSourceAggregationBuilder.java:417) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.search.aggregations.support.ValuesSourceAggregationBuilder$LeafOnly.doBuild(ValuesSourceAggregationBuilder.java:149) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.search.aggregations.AbstractAggregationBuilder.build(AbstractAggregationBuilder.java:155) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.search.aggregations.AggregatorFactories$Builder.build(AggregatorFactories.java:462) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.search.SearchService.parseSource(SearchService.java:1524) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.search.SearchService.createContext(SearchService.java:1207) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.search.SearchService.executeQueryPhase(SearchService.java:772) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.search.SearchService$2.lambda$onResponse$0(SearchService.java:738) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:74) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:89) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.threadpool.TaskAwareRunnable.doRun(TaskAwareRunnable.java:78) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:59) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:975) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[opensearch-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT]
... 3 more
Metadata
Metadata
Assignees
Labels
PPLPiped processing languagePiped processing languageenhancementNew feature or requestNew feature or requestv3.3.0
Type
Projects
Status
New
Status
Done