Add commonly used math, string and date scalar functions in Pinot

[KKcorps]

There are some functions which are missing from Pinot currently which are part of standard SQL. We can still do some of these operations (except for trignometric ones) by writing composite SQL expressions using existing functions. This however leads to complex queries and it is better we support these functions directly in Pinot.

For better performance, Transform function classes have also been added for the new Math functions. The classes have the same function name as scalar and will be given first preference when function is used.

This also helps in providing better support for third party tools such as Tableau.

[Jackie-Jiang]

Thanks for adding these missing functions. Can you please add unit test to ensure the desired behavior?

[codecov-commenter]

Codecov Report

Merging #8304 (ccbba40) into master (515f7a2) will increase coverage by 0.20%.
The diff coverage is 77.16%.

@@             Coverage Diff              @@
##             master    #8304      +/-   ##
============================================
+ Coverage     70.76%   70.96%   +0.20%     
- Complexity     4278     4283       +5     
============================================
  Files          1655     1661       +6     
  Lines         86665    87046     +381     
  Branches      13067    13134      +67     
============================================
+ Hits          61327    61772     +445     
+ Misses        21095    21012      -83     
- Partials       4243     4262      +19     

Flag	Coverage Δ
integration1	28.80% <14.18%> (+0.13%)	⬆️
integration2	27.34% <14.18%> (-0.03%)	⬇️
unittests1	67.07% <77.16%> (+0.09%)	⬆️
unittests2	14.15% <0.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
.../pinot/common/function/DateTimePatternHandler.java	80.00% <0.00%> (-20.00%)	⬇️
...common/function/scalar/TrigonometricFunctions.java	0.00% <0.00%> (ø)
...ot/common/function/scalar/ArithmeticFunctions.java	69.56% <37.50%> (+16.23%)	⬆️
.../pinot/common/function/scalar/StringFunctions.java	71.05% <64.28%> (-2.80%)	⬇️
...nsform/function/RoundDecimalTransformFunction.java	71.79% <71.79%> (ø)
...orm/function/TruncateDecimalTransformFunction.java	71.79% <71.79%> (ø)
...tor/transform/function/PowerTransformFunction.java	80.76% <80.76%> (ø)
...inot/common/function/scalar/DateTimeFunctions.java	95.23% <83.33%> (-0.73%)	⬇️
...form/function/TrigonometricTransformFunctions.java	86.25% <86.25%> (ø)
...orm/function/SingleParamMathTransformFunction.java	95.23% <90.90%> (-0.42%)	⬇️
... and 44 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 515f7a2...ccbba40. Read the comment docs.

[Jackie-Jiang]

@KKcorps There are 6 hidden conversations (github automatically collapsed them) unaddressed, please take a look.

[KKcorps]

@Jackie-Jiang @richardstartin I have added the unit-tests as well.

[richardstartin]

🥇

[richardstartin]

This looks good to me, and I appreciate that this has extended a long way beyond its original scope so thanks for being so patient and for your determination. Can we just get rid of the stray semicolons in the trig functions before merging please.

[Jackie-Jiang]

Mostly good

[Jackie-Jiang]

We may avoid the boxing/unboxing using primitive double and use NaN to represent not set, same for other functions

[KKcorps]

Made the change for this class. Is it fine if I use Integer.MIN_VALUE for other classes or leaved the boxed implementation as it is for now? I would prefer the latter.

[Jackie-Jiang]

Using boxed value is fine if we unbox the value before getting into the loop (not sure if JVM will do the optimization, cc @richardstartin)

[richardstartin]

I would just use a flag rather than sentinel values. I don't know what the best thing to do is, but it probably isn't very important anyway. If this ever shows up as a bottleneck we can fix it.

[Jackie-Jiang]

I like the idea of using a boolean flag, so that it is guaranteed to have the desired behavior without potentially paying the cost of unboxing the values

[KKcorps]

Yeah, boolean flags are better. using them.

[Jackie-Jiang]

LGTM otherwise

[Jackie-Jiang]

(minor) Rename the second parameter to scale for readability

[Jackie-Jiang]

Can we use Math.round() for this function as BigDecimal calculation is much more costly? If not, please add some comments explaining the reason. Same for the one with scale

[Jackie-Jiang]

Can simplify the handling by setting _scale = 0 and _fixedScale = true

[KKcorps]

The reason for doing it this way is to use optimal Math.round and Math.floor for this case instead of relying on BigDecimal represenation.