Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix bug with to_timestamp and InitCap logical serialization, add roundtrip test between expression and proto, #8868

Merged
merged 7 commits into from
Jan 17, 2024

Conversation

Weijun-H
Copy link
Member

@Weijun-H Weijun-H commented Jan 15, 2024

Which issue does this PR close?

Parts. #8847

Rationale for this change

When I worked on #8847, there were wrong function calls in parse_expr, the ci did not notify it.

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?


let ctx = SessionContext::new();
let lit = Expr::Literal(ScalarValue::Utf8(None));
for builtin_fun in BuiltinScalarFunction::iter() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 👍

@alamb
Copy link
Contributor

alamb commented Jan 15, 2024

Interestingly, when I run this test on #8847 it fails with a different function


---- cases::serialize::test_expression_serialization_roundtrip stdout ----
thread 'cases::serialize::test_expression_serialization_roundtrip' panicked at datafusion/proto/tests/cases/serialize.rs:274:9:
assertion `left == right` failed
  left: "initcap"
 right: "ascii"

@alamb alamb changed the title Minor: add roundtrip test between expression and proto Minor: add roundtrip test between expression and proto, fic bug with to_timestamp and InitCap Jan 15, 2024
@@ -1588,7 +1588,7 @@ pub fn parse_expr(
Ok(character_length(parse_expr(&args[0], registry)?))
}
ScalarFunction::Chr => Ok(chr(parse_expr(&args[0], registry)?)),
ScalarFunction::InitCap => Ok(ascii(parse_expr(&args[0], registry)?)),
ScalarFunction::InitCap => Ok(initcap(parse_expr(&args[0], registry)?)),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯 for testing @Weijun-H -- your test also found two other functions with similar problems to #8847 👏

I took the liberty of fixing them

@@ -51,6 +51,7 @@ pbjson = { version = "0.6.0", optional = true }
prost = "0.12.0"
serde = { version = "1.0", optional = true }
serde_json = { workspace = true, optional = true }
strum = { version = "0.25.0", features = ["derive"] }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW since this module is only used in the tests, we should put it in the dev-dependencies section, so everyone using datafusion-proto does not need to compile it as well

I did this in 0a41737

@alamb alamb changed the title Minor: add roundtrip test between expression and proto, fic bug with to_timestamp and InitCap fix bug with to_timestamp and InitCap logical serialization, add roundtrip test between expression and proto, Jan 15, 2024
Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the strum dependency is needed for the test only?

@Weijun-H
Copy link
Member Author

the strum dependency is needed for the test only?

Yes, it is. @comphead

Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm thanks @Weijun-H

@comphead comphead merged commit 31094b0 into apache:main Jan 17, 2024
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants