Skip to content

do not identify function types by throwing exceptions#8137

Merged
richardstartin merged 2 commits intoapache:masterfrom
richardstartin:function-type-identification
Feb 8, 2022
Merged

do not identify function types by throwing exceptions#8137
richardstartin merged 2 commits intoapache:masterfrom
richardstartin:function-type-identification

Conversation

@richardstartin
Copy link
Member

This PR allows function (transform/aggregation) type to be identified without throwing exceptions when an expression (e.g. "ASC") is not a function type, this reduces allocation rate.

@codecov-commenter
Copy link

codecov-commenter commented Feb 5, 2022

Codecov Report

Merging #8137 (7f7974e) into master (a47af49) will increase coverage by 39.46%.
The diff coverage is 90.32%.

Impacted file tree graph

@@              Coverage Diff              @@
##             master    #8137       +/-   ##
=============================================
+ Coverage     30.69%   70.15%   +39.45%     
- Complexity        0     4303     +4303     
=============================================
  Files          1613     1623       +10     
  Lines         83952    84302      +350     
  Branches      12597    12637       +40     
=============================================
+ Hits          25768    59141    +33373     
+ Misses        55889    21071    -34818     
- Partials       2295     4090     +1795     
Flag Coverage Δ
integration1 ?
integration2 27.61% <25.80%> (-0.11%) ⬇️
unittests1 67.90% <90.32%> (?)
unittests2 14.17% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...apache/pinot/common/utils/SqlResultComparator.java 0.00% <0.00%> (-18.85%) ⬇️
...che/pinot/segment/spi/AggregationFunctionType.java 90.24% <85.71%> (+90.24%) ⬆️
...e/pinot/common/function/TransformFunctionType.java 100.00% <100.00%> (+10.95%) ⬆️
...ot/common/request/context/RequestContextUtils.java 74.37% <100.00%> (+20.10%) ⬆️
...pql/parsers/PinotQuery2BrokerRequestConverter.java 89.86% <100.00%> (+6.08%) ⬆️
...inot/pql/parsers/pql2/ast/OutputColumnAstNode.java 95.12% <100.00%> (+36.58%) ⬆️
...org/apache/pinot/sql/parsers/CalciteSqlParser.java 87.92% <100.00%> (+21.41%) ⬆️
...pinot/minion/exception/TaskCancelledException.java 0.00% <0.00%> (-100.00%) ⬇️
...nverttorawindex/ConvertToRawIndexTaskExecutor.java 0.00% <0.00%> (-100.00%) ⬇️
...e/pinot/common/minion/MergeRollupTaskMetadata.java 0.00% <0.00%> (-94.74%) ⬇️
... and 1168 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a47af49...7f7974e. Read the comment docs.

Copy link
Contributor

@Jackie-Jiang Jackie-Jiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree on avoiding the exception in normal execution path. Do we have some performance number back this change?

@richardstartin
Copy link
Member Author

richardstartin commented Feb 7, 2022

Agree on avoiding the exception in normal execution path. Do we have some performance number back this change?

I had numbers for a few of these PRs combined on #8134 (which have been submitted separately because some users prefer tightly scoped changes) and this was just an incremental improvement. I can measure this separately.

@richardstartin richardstartin force-pushed the function-type-identification branch 2 times, most recently from e027ec3 to 830301a Compare February 7, 2022 22:41
@richardstartin
Copy link
Member Author

@Jackie-Jiang here are some numbers for the query I was tuning (on my MacBook so incomparable to numbers on different PRs)

master

Benchmark                                                                               (_intBaseValue)  (_numRows)  Mode  Cnt        Score          Error   Units
BenchmarkFilteredAggregations.testNonFilteredAggregations                                             0     1500000  avgt    5    32506.601 ±     4192.330   us/op
BenchmarkFilteredAggregations.testNonFilteredAggregations:·gc.alloc.rate                              0     1500000  avgt    5       22.952 ±        2.747  MB/sec
BenchmarkFilteredAggregations.testNonFilteredAggregations:·gc.alloc.rate.norm                         0     1500000  avgt    5  1171670.000 ±    15616.821    B/op
BenchmarkFilteredAggregations.testNonFilteredAggregations:·gc.churn.G1_Eden_Space                     0     1500000  avgt    5       37.526 ±      323.107  MB/sec
BenchmarkFilteredAggregations.testNonFilteredAggregations:·gc.churn.G1_Eden_Space.norm                0     1500000  avgt    5  1914496.826 ± 16484395.047    B/op
BenchmarkFilteredAggregations.testNonFilteredAggregations:·gc.churn.G1_Old_Gen                        0     1500000  avgt    5        7.749 ±       66.725  MB/sec
BenchmarkFilteredAggregations.testNonFilteredAggregations:·gc.churn.G1_Old_Gen.norm                   0     1500000  avgt    5   395366.090 ±  3404221.273    B/op
BenchmarkFilteredAggregations.testNonFilteredAggregations:·gc.count                                   0     1500000  avgt    5        1.000                 counts
BenchmarkFilteredAggregations.testNonFilteredAggregations:·gc.time                                    0     1500000  avgt    5       13.000                     ms

branch

Benchmark                                                                      (_intBaseValue)  (_numRows)  Mode  Cnt       Score       Error   Units
BenchmarkFilteredAggregations.testNonFilteredAggregations                                    0     1500000  avgt    5   28881.564 ±  2888.988   us/op
BenchmarkFilteredAggregations.testNonFilteredAggregations:·gc.alloc.rate                     0     1500000  avgt    5      20.376 ±     1.846  MB/sec
BenchmarkFilteredAggregations.testNonFilteredAggregations:·gc.alloc.rate.norm                0     1500000  avgt    5  922164.440 ± 30175.150    B/op
BenchmarkFilteredAggregations.testNonFilteredAggregations:·gc.count                          0     1500000  avgt    5         ≈ 0              counts

@richardstartin
Copy link
Member Author

The test failure should be addressed by #8154

@richardstartin richardstartin force-pushed the function-type-identification branch from 830301a to f12498d Compare February 8, 2022 08:03
@richardstartin richardstartin force-pushed the function-type-identification branch from f12498d to 7f7974e Compare February 8, 2022 18:23
@richardstartin richardstartin merged commit 5fa4737 into apache:master Feb 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants