generated from amazon-archives/__template_Custom
-
Couldn't load subscription status.
- Fork 176
Add partial udf:String/condition/math built-in functions #3363
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
xinyual
wants to merge
18
commits into
opensearch-project:feature/calcite-engine
from
xinyual:addPartialUDF
Closed
Changes from all commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
7f1eba4
add condition and string func
xinyual 86f1b9c
add atan
xinyual 7d46cbc
[WIP] map opensearch math functions to calcite implementations
yuancu ce264dd
add transfer for log
xinyual 06a7b65
Merge branch 'mapToCalciteBuiltIn' into map2calcite
xinyual 25d7d51
Merge pull request #1 from yaunchun/map2calcite
xinyual 6cc2452
fix log and atan
xinyual 7ff8a1c
add udf and udaf percentile and Mod
xinyual 19e11d6
add condition udf and take agg
xinyual 25867df
add unified test framework
xinyual 65fd487
Merge remote-tracking branch 'origin/feature/calcite-engine' into add…
xinyual 7274a05
fix UT/IT
xinyual 6ccfd97
merge
xinyual f6120dc
fix UT/IT
xinyual 4fa915d
remove useless code change
xinyual 99b33cd
apply spotless
xinyual 0b28fab
add license and move it
xinyual 651db98
remove useless change
xinyual File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
10 changes: 10 additions & 0 deletions
10
core/src/main/java/org/opensearch/sql/calcite/udf/Accumulator.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.sql.calcite.udf; | ||
|
|
||
| public interface Accumulator { | ||
| Object result(); | ||
| } | ||
15 changes: 15 additions & 0 deletions
15
core/src/main/java/org/opensearch/sql/calcite/udf/UserDefinedAggFunction.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.sql.calcite.udf; | ||
|
|
||
| public interface UserDefinedAggFunction<S extends Accumulator> { | ||
| S init(); | ||
|
|
||
| Object result(S accumulator); | ||
|
|
||
| // Add values to the accumulator | ||
| S add(S acc, Object... values); | ||
| } |
10 changes: 10 additions & 0 deletions
10
core/src/main/java/org/opensearch/sql/calcite/udf/UserDefinedFunction.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.sql.calcite.udf; | ||
|
|
||
| public interface UserDefinedFunction { | ||
| Object eval(Object... args); | ||
| } |
21 changes: 21 additions & 0 deletions
21
core/src/main/java/org/opensearch/sql/calcite/udf/conditionUDF/IfFunction.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.sql.calcite.udf.conditionUDF; | ||
|
|
||
| import org.opensearch.sql.calcite.udf.UserDefinedFunction; | ||
|
|
||
| public class IfFunction implements UserDefinedFunction { | ||
| @Override | ||
| public Object eval(Object... args) { | ||
| Object condition = args[0]; | ||
| Object trueValue = args[1]; | ||
| Object falseValue = args[2]; | ||
| if (condition instanceof Boolean) { | ||
| return (Boolean) condition ? trueValue : falseValue; | ||
| } | ||
| return trueValue; | ||
| } | ||
| } |
21 changes: 21 additions & 0 deletions
21
core/src/main/java/org/opensearch/sql/calcite/udf/conditionUDF/IfNullFunction.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.sql.calcite.udf.conditionUDF; | ||
|
|
||
| import java.util.Objects; | ||
| import org.opensearch.sql.calcite.udf.UserDefinedFunction; | ||
|
|
||
| public class IfNullFunction implements UserDefinedFunction { | ||
| @Override | ||
| public Object eval(Object... args) { | ||
| Object conditionValue = args[0]; | ||
| Object defaultValue = args[1]; | ||
| if (Objects.isNull(conditionValue)) { | ||
| return defaultValue; | ||
| } | ||
| return conditionValue; | ||
| } | ||
| } |
21 changes: 21 additions & 0 deletions
21
core/src/main/java/org/opensearch/sql/calcite/udf/conditionUDF/NullIfFunction.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.sql.calcite.udf.conditionUDF; | ||
|
|
||
| import java.util.Objects; | ||
| import org.opensearch.sql.calcite.udf.UserDefinedFunction; | ||
|
|
||
| public class NullIfFunction implements UserDefinedFunction { | ||
| @Override | ||
| public Object eval(Object... args) { | ||
| Object firstValue = args[0]; | ||
| Object secondValue = args[1]; | ||
| if (Objects.equals(firstValue, secondValue)) { | ||
| return null; | ||
| } | ||
| return firstValue; | ||
| } | ||
| } |
19 changes: 19 additions & 0 deletions
19
core/src/main/java/org/opensearch/sql/calcite/udf/mathUDF/CRC32Function.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.sql.calcite.udf.mathUDF; | ||
|
|
||
| import java.util.zip.CRC32; | ||
| import org.opensearch.sql.calcite.udf.UserDefinedFunction; | ||
|
|
||
| public class CRC32Function implements UserDefinedFunction { | ||
| @Override | ||
| public Object eval(Object... args) { | ||
| Object value = args[0]; | ||
| CRC32 crc = new CRC32(); | ||
| crc.update(value.toString().getBytes()); | ||
| return crc.getValue(); | ||
| } | ||
| } |
15 changes: 15 additions & 0 deletions
15
core/src/main/java/org/opensearch/sql/calcite/udf/mathUDF/EulerFunction.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.sql.calcite.udf.mathUDF; | ||
|
|
||
| import org.opensearch.sql.calcite.udf.UserDefinedFunction; | ||
|
|
||
| public class EulerFunction implements UserDefinedFunction { | ||
| @Override | ||
| public Object eval(Object... args) { | ||
| return Math.E; | ||
| } | ||
| } |
37 changes: 37 additions & 0 deletions
37
core/src/main/java/org/opensearch/sql/calcite/udf/mathUDF/ModFunction.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.sql.calcite.udf.mathUDF; | ||
|
|
||
| import org.opensearch.sql.calcite.udf.UserDefinedFunction; | ||
|
|
||
| public class ModFunction implements UserDefinedFunction { | ||
| @Override | ||
| public Object eval(Object... args) { | ||
| if (args.length < 2) { | ||
| throw new IllegalArgumentException("At least two arguments are required"); | ||
| } | ||
|
|
||
| // Get the two values | ||
| Object mod0 = args[0]; | ||
| Object mod1 = args[1]; | ||
|
|
||
| // Handle numbers dynamically | ||
| if (mod0 instanceof Integer && mod1 instanceof Integer) { | ||
| return (Integer) mod0 % (Integer) mod1; | ||
| } else if (mod0 instanceof Number && mod1 instanceof Number) { | ||
| double num0 = ((Number) mod0).doubleValue(); | ||
| double num1 = ((Number) mod1).doubleValue(); | ||
|
|
||
| if (num1 == 0) { | ||
| throw new ArithmeticException("Modulo by zero is not allowed"); | ||
| } | ||
|
|
||
| return num0 % num1; // Handles both float and double cases | ||
| } else { | ||
| throw new IllegalArgumentException("Invalid argument types: Expected numeric values"); | ||
| } | ||
| } | ||
| } |
35 changes: 35 additions & 0 deletions
35
core/src/main/java/org/opensearch/sql/calcite/udf/mathUDF/SqrtFunction.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.sql.calcite.udf.mathUDF; | ||
|
|
||
| import static java.lang.Math.sqrt; | ||
|
|
||
| import org.opensearch.sql.calcite.udf.UserDefinedFunction; | ||
|
|
||
| public class SqrtFunction implements UserDefinedFunction { | ||
| @Override | ||
| public Object eval(Object... args) { | ||
| if (args.length < 1) { | ||
| throw new IllegalArgumentException("At least one argument is required"); | ||
| } | ||
|
|
||
| // Get the input value | ||
| Object input = args[0]; | ||
|
|
||
| // Handle numbers dynamically | ||
| if (input instanceof Number) { | ||
| double num = ((Number) input).doubleValue(); | ||
|
|
||
| if (num < 0) { | ||
| throw new ArithmeticException("Cannot compute square root of a negative number"); | ||
| } | ||
|
|
||
| return sqrt(num); // Computes sqrt using Math.sqrt() | ||
| } else { | ||
| throw new IllegalArgumentException("Invalid argument type: Expected a numeric value"); | ||
| } | ||
| } | ||
| } |
57 changes: 57 additions & 0 deletions
57
core/src/main/java/org/opensearch/sql/calcite/udf/udaf/PercentileApproxFunction.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| package org.opensearch.sql.calcite.udf.udaf; | ||
|
|
||
| import com.tdunning.math.stats.AVLTreeDigest; | ||
| import java.util.Arrays; | ||
| import java.util.List; | ||
| import java.util.Objects; | ||
| import org.opensearch.sql.calcite.udf.Accumulator; | ||
| import org.opensearch.sql.calcite.udf.UserDefinedAggFunction; | ||
|
|
||
| public class PercentileApproxFunction | ||
| implements UserDefinedAggFunction<PercentileApproxFunction.PencentileApproAccumulator> { | ||
| @Override | ||
| public PencentileApproAccumulator init() { | ||
| return new PencentileApproAccumulator(); | ||
| } | ||
|
|
||
| // Add values to the accumulator | ||
| @Override | ||
| public PencentileApproAccumulator add(PencentileApproAccumulator acc, Object... values) { | ||
| List<Object> allValues = Arrays.asList(values); | ||
| Object targetValue = allValues.get(0); | ||
| if (Objects.isNull(targetValue)) { | ||
| return acc; | ||
| } | ||
| Number percentileValue = (Number) allValues.get(1); | ||
| acc.evaluate(((Number) targetValue).doubleValue(), percentileValue.intValue()); | ||
| return acc; | ||
| } | ||
|
|
||
| // Calculate the percentile | ||
| @Override | ||
| public Object result(PencentileApproAccumulator acc) { | ||
| if (acc.size() == 0) { | ||
| return null; | ||
| } | ||
| return acc.result(); | ||
| } | ||
|
|
||
| public static class PencentileApproAccumulator extends AVLTreeDigest implements Accumulator { | ||
| public static final double DEFAULT_COMPRESSION = 100.0; | ||
| private double percent; | ||
|
|
||
| public PencentileApproAccumulator() { | ||
| super(DEFAULT_COMPRESSION); | ||
| this.percent = 1.0; | ||
| } | ||
|
|
||
| public void evaluate(double value, int percent) { | ||
| this.percent = percent / 100.0; | ||
| this.add(value); | ||
| } | ||
|
|
||
| public Object result() { | ||
| return this.quantile(this.percent); | ||
| } | ||
| } | ||
| } |
55 changes: 55 additions & 0 deletions
55
core/src/main/java/org/opensearch/sql/calcite/udf/udaf/TakeAggFunction.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| package org.opensearch.sql.calcite.udf.udaf; | ||
|
|
||
| import java.util.ArrayList; | ||
| import java.util.List; | ||
| import org.opensearch.sql.calcite.udf.Accumulator; | ||
| import org.opensearch.sql.calcite.udf.UserDefinedAggFunction; | ||
|
|
||
| public class TakeAggFunction implements UserDefinedAggFunction<TakeAggFunction.TakeAccumulator> { | ||
|
|
||
| @Override | ||
| public TakeAccumulator init() { | ||
| return new TakeAccumulator(); | ||
| } | ||
|
|
||
| @Override | ||
| public Object result(TakeAccumulator accumulator) { | ||
| return accumulator.result(); | ||
| } | ||
|
|
||
| @Override | ||
| public TakeAccumulator add(TakeAccumulator acc, Object... values) { | ||
| Object candidateValue = values[0]; | ||
| int size = 0; | ||
| if (values.length > 1) { | ||
| size = (int) values[1]; | ||
| } else { | ||
| size = 10; | ||
| } | ||
| if (size > acc.size()) { | ||
| acc.add(candidateValue); | ||
| } | ||
| return acc; | ||
| } | ||
|
|
||
| public static class TakeAccumulator implements Accumulator { | ||
| private List<Object> hits; | ||
|
|
||
| public TakeAccumulator() { | ||
| hits = new ArrayList<>(); | ||
| } | ||
|
|
||
| @Override | ||
| public Object result() { | ||
| return hits; | ||
| } | ||
|
|
||
| public void add(Object value) { | ||
| hits.add(value); | ||
| } | ||
|
|
||
| public int size() { | ||
| return hits.size(); | ||
| } | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.