Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce array_except function #8135

Merged
merged 5 commits into from
Nov 17, 2023
Merged

Introduce array_except function #8135

merged 5 commits into from
Nov 17, 2023

Conversation

jayzhan211
Copy link
Contributor

Which issue does this PR close?

Closes #6979

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added logical-expr Logical plan and expressions physical-expr Physical Expressions sqllogictest SQL Logic Tests (.slt) labels Nov 11, 2023
@@ -1342,6 +1347,10 @@ pub fn parse_expr(
parse_expr(&args[0], registry)?,
parse_expr(&args[1], registry)?,
)),
ScalarFunction::ArrayIntersect => Ok(array_intersect(
Copy link
Contributor Author

@jayzhan211 jayzhan211 Nov 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing from #8081. Not sure why CI from #8081 does not catch this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to enhance the check during CI? maybe this is a little bug?

@@ -2454,6 +2454,113 @@ select list_has_all(make_array(1,2,3), make_array(4,5,6)),
----
false true false true

## array_except

statement ok
Copy link
Contributor Author

@jayzhan211 jayzhan211 Nov 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unlike other tests, I try a new style of test, keep creating table and drop table close together, hope this is easier for review.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is choice of style, imho no-table tests are easier to read.

query ?
select array_expect([], []), array_except(....)
----
[], ...

Copy link
Contributor Author

@jayzhan211 jayzhan211 Nov 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, we still need table to test multi-rows cases. The new style I mean is not create table in the beginning of this large file and drop table at the end of the file, but when we are done we drop it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the multirow still possible to do without creating a table

select ... from (select 1 a, 'asdf' b union all select 2 b, 'zxcv' b)

thats the matter of style of course.

Copy link
Contributor Author

@jayzhan211 jayzhan211 Nov 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We cant differentiate empty array and null currently. Both of them are null type So we can fix them in another PR

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also prefer the new style (create/query/drop table statements are organized more tightly). This will make tests more clear.

@jayzhan211 jayzhan211 marked this pull request as ready for review November 11, 2023 06:05
Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jayzhan211 please also add tests with empty arrays

And since we have introduced the new built in function we need to get it documented in
scalar_functions.md and expressions.md

@jayzhan211 jayzhan211 marked this pull request as draft November 13, 2023 23:33
@jayzhan211 jayzhan211 marked this pull request as ready for review November 14, 2023 01:31
@Veeupup
Copy link
Contributor

Veeupup commented Nov 14, 2023

LGTM!

Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm thanks @jayzhan211
Please create a followup ticket to differentiate empty array and null, that sounds important

@alamb
Copy link
Contributor

alamb commented Nov 14, 2023

There appears to be a non trivial number of conflicts in this PR now

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
jayzhan211 and others added 2 commits November 15, 2023 09:05
@alamb
Copy link
Contributor

alamb commented Nov 17, 2023

I took the liberty of merging up from main to resolve a merge conflict

@alamb alamb merged commit 7293764 into apache:main Nov 17, 2023
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
logical-expr Logical plan and expressions physical-expr Physical Expressions sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement array_except function
4 participants