Skip to content

[FLINK-35466][table] Implement star expansion in function calls in SE… #26438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

affo
Copy link
Contributor

@affo affo commented Apr 10, 2025

…LECT clause

What is the purpose of the change

This PR adds star expansion in SQL in order to gracefully handle statements like SELECT myUDF(*) from MyTable.

Brief change log

  • Add a private visitor to perform star expansion of calls to SqlValidatorImpl that re-uses the pre-existing expandStar method in the validator itself
  • Add a step in-between unconditional rewrites and validation to expand star (unconditional rewrites would be too early to have information about scopes and perform the expansion, after validation would be too late as the type of * cannot be derived
  • Add tests for both parsing and validating

As of now, I did not update the class Javadoc with the lines of the update as I don't know if this is the best way to achieve the result. Waiting for a review to fix the Javadoc in case.

Verifying this change

This change added tests and can be verified as follows:

  • Tests for parsing to make sure foo(*) can be parsed as a valid function call
  • Tests in validation to verify that the expansion is performed correctly and only under SELECT
  • I have local tests for running a full program and check the final result, but don't know if needed

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? NA

@flinkbot
Copy link
Collaborator

flinkbot commented Apr 10, 2025

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@@ -1051,6 +1051,14 @@ private SqlNode validateScopedExpression(SqlNode topNode, SqlValidatorScope scop
if (outermostNode.isA(SqlKind.TOP_LEVEL)) {
registerQuery(scope, null, outermostNode, outermostNode, null, false);
}
// ----- FLINK MODIFICATION BEGIN -----
if (outermostNode instanceof SqlSelect) {
SqlSelect select = (SqlSelect) outermostNode;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not an expert on the table planner - but had some basic questions.

  1. I am not seeing how we know it is a select * . Is this related to if (outermostNode instanceof SqlSelect) {
  2. Will this effect all select * code , include the working non UDF select *'s? If so is there now redundant code we need to remove that dealt with the * later?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks you @davidradl for your review!

  1. I am not isolating select * for just the select node. This is about the star expansion in function calls only
  2. This would not expand * alone, but only in SqlCall. The implementation of that is quite hardcoded in the SqlValidator and I did not touch that. However, I am able to re-use the same expandStar method.

Does this answer your questions?

this.expr("foo(t1.x, t2.y, t3.z)").ok("`FOO`(`T1`.`X`, `T2`.`Y`, `T3`.`Z`)");
this.expr("foo(*)").ok("`FOO`(*)");
this.expr("foo(*^,^ x)").fails("(?s).*Encountered \",\" at line 1, column 6.*");
this.expr("foo(t1.*)").ok("`FOO`(`T1`.*)");
Copy link
Contributor

@davidradl davidradl Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about a quoted * test. To check that the select does not expand

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants