[FEATURE] PPL-Calcite Function Parameter Validation

[yuancu]

Is your feature request related to a problem?
#3229 implements PPL functions with the Calcite engine. However, the parameter types and lengths are not being checked for the time being. It will only throw error at execution time.

Currently, if the parameter type / length is incorrect, it will throw an error like:

Exception happened during execution
...
    Caused by java.lang.RuntimeException: java.sql.SQLException: Error while preparing plan..
    ...
        Caused by: Suppressed: java.lang.RuntimeException: while resolving method methodName(paramType1, paramType2, ...)
        ...
             Caused by: java.lang.NoSuchMethodException: methodName(paramType1, paramType2, ...)
             ...

Users will see the first error message with an 500 internal error tag in their request response. An error message with the wrong and proper parameter types will be more informative to users.

The validation mechanism from Calcite is not used -- the phase it takes effect is skipped.

What solution would you like?
Implement a function validation mechanism.

What alternatives have you considered?

Do you have any additional context?

[penghuo]

@yuancu What is the expected validation mechanism? In my opinion, Calcite throws an exception during function resolution, which serves as the validation mechanism.

[yuancu]

@yuancu What is the expected validation mechanism? In my opinion, Calcite throws an exception during function resolution, which serves as the validation mechanism.

The expected validation mechanism is similar to that in v2:

1. Resolves a function by its name
2. Retrieves expected signatures
3. Throws an exception when there isn't any matching signature with the actual one.

Our current implementation (see PPLFuncImpTable.java:L131) also follows this paradigm. The validation happens when parsing AST node into RexNode at visitFunction

In contrast, Calcite implements the validation mechanism in a dedicate "validation" phase (the "Sql Parser & Validator" in the attached graph), before converting a SqlNode to a RelNode.

SQL query
   ↓
SqlParser → SqlNode        (AST)
   ↓
SqlValidator → SqlNode     (validated AST)     <-------  where Calcite's parameter validation takes place
   ↓
SqlToRelConverter → RelNode (Logical Plan)
   ↓
Planner Rules → RelNode    (Optimized Logical Plan)
   ↓
RelTrait/Convention → RelNode (Physical Plan)

It uses a SqlValidator to check whether a SqlCall is valid. It retrieves operand information from the SqlCall, while in our implementation we get parameters as a list of RexNode from an unresolved function expression (an ast node). You can refer to SqlValidatorImpl.java:L6731 for Calcite's detailed implementation for parameter validation.