Skip to content

Update to sqlparser-rs v0.50.0 #12014

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Aug 17, 2024
Merged

Update to sqlparser-rs v0.50.0 #12014

merged 5 commits into from
Aug 17, 2024

Conversation

samuelcolvin
Copy link
Contributor

@samuelcolvin samuelcolvin commented Aug 15, 2024

Which issue does this PR close?

NA

Rationale for this change

Support latest features of sqlparser-rs.

What changes are included in this PR?

Uprev sqlparser-rs to v0.50.0

Are these changes tested?

No new tests needed AFAIK.

Are there any user-facing changes?

Some new SQL syntax supported.

@github-actions github-actions bot added the sql SQL Planner label Aug 15, 2024
@@ -268,6 +268,7 @@ pub(crate) fn value_to_string(value: &Value) -> Option<String> {
Value::SingleQuotedString(s) => Some(s.to_string()),
Value::DollarQuotedString(s) => Some(s.to_string()),
Value::Number(_, _) | Value::Boolean(_) => Some(value.to_string()),
Value::UnicodeStringLiteral(s) => Some(s.to_string()),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand why value_to_string supports (returns Some(String)) for some types of string, but not others, so I'm not clear whether Value::UnicodeStringLiteral should return Some or None?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this was not carefully designed and likely is not an intentional oversight

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what you have here is ok

@samuelcolvin
Copy link
Contributor Author

Waiting for decision on apache/datafusion-sqlparser-rs#1382.

@alamb
Copy link
Contributor

alamb commented Aug 15, 2024

BTW I double checked with this branch that you can still use id as a column name without double quotes

$ git status
On branch sqlparser-main
nothing to commit, working tree clean
~/Software/datafusion/datafusion-cli$ cargo run
...
> create table foo(id int) as values (1);
0 row(s) fetched.
Elapsed 0.030 seconds.

> select id from foo;
+----+
| id |
+----+
| 1  |
+----+
1 row(s) fetched.
Elapsed 0.007 seconds.

>

@samuelcolvin samuelcolvin changed the title Support HEAD of sqlparser-rs main sqlparser-rs v0.50.0 Aug 16, 2024
@samuelcolvin samuelcolvin marked this pull request as ready for review August 16, 2024 11:12
@alamb alamb changed the title sqlparser-rs v0.50.0 Update to sqlparser-rs v0.50.0 Aug 16, 2024
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @samuelcolvin -- this looks great to me. I had some style suggestions, but they aren't required from my perspective to merge this PR

@@ -178,7 +178,7 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
SQLExpr::Value(value) => {
self.parse_value(value, planner_context.prepare_param_data_types())
}
SQLExpr::Extract { field, expr } => {
SQLExpr::Extract { field, expr, .. } => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend explicitly listing out the other fields here and returning a NotImplemented error if there is some new syntax that DataFusion doesn't support.

Otherwise what we have seen a few times in the past is that the parser accepts some new syntax element but DataFusion just igores it silently

@@ -438,7 +438,7 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
}
SQLDataType::Bytea => Ok(DataType::Binary),
SQLDataType::Interval => Ok(DataType::Interval(IntervalUnit::MonthDayNano)),
SQLDataType::Struct(fields) => {
SQLDataType::Struct(fields, _) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise here I think it would be good to name the field like

Suggested change
SQLDataType::Struct(fields, _) => {
SQLDataType::Struct(fields, _bracket_type) => {

So it is easier to understand by reading just the code what is being ignored

table_name,
..
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

per comment above, I recommend keeping the fields named explcitly

@@ -268,6 +268,7 @@ pub(crate) fn value_to_string(value: &Value) -> Option<String> {
Value::SingleQuotedString(s) => Some(s.to_string()),
Value::DollarQuotedString(s) => Some(s.to_string()),
Value::Number(_, _) | Value::Boolean(_) => Some(value.to_string()),
Value::UnicodeStringLiteral(s) => Some(s.to_string()),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what you have here is ok

@@ -848,8 +848,10 @@ SELECT EXTRACT("year" FROM timestamp '2020-09-08T12:00:00+00:00')
----
2020

query error
query R
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥳 🦜

@alamb alamb merged commit 7fa7689 into apache:main Aug 17, 2024
28 of 29 checks passed
@alamb
Copy link
Contributor

alamb commented Aug 17, 2024

Thanks again @samuelcolvin

@samuelcolvin samuelcolvin deleted the sqlparser-main branch August 17, 2024 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sql SQL Planner sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants