Skip to content

Conversation

@izveigor
Copy link
Contributor

@izveigor izveigor commented Jul 6, 2023

Follow on to apache/datafusion#6885.
The main problem is the following: input string starting with characters like @, # in many dialects can have both identifiers and operators.
Because of this, there is ambiguity in grammar.

@izveigor izveigor marked this pull request as draft July 6, 2023 19:42
@izveigor izveigor changed the title fix: parsing AtArrow fix: parsing JsonOperator Jul 10, 2023
@coveralls
Copy link

Pull Request Test Coverage Report for Build 5507745151

  • 58 of 70 (82.86%) changed or added relevant lines in 2 files are covered.
  • 3 unchanged lines in 1 file lost coverage.
  • Overall coverage decreased (-0.03%) to 86.927%

Changes Missing Coverage Covered Lines Changed/Added Lines %
tests/sqlparser_common.rs 20 21 95.24%
src/tokenizer.rs 38 49 77.55%
Files with Coverage Reduction New Missed Lines %
src/tokenizer.rs 3 88.1%
Totals Coverage Status
Change from base Build 5476034295: -0.03%
Covered Lines: 15194
Relevant Lines: 17479

💛 - Coveralls

@izveigor izveigor marked this pull request as ready for review July 10, 2023 11:58
@izveigor
Copy link
Contributor Author

cc @alamb

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM @izveigor . Thank you very much for the contribution and I apologize for the delay in review

I have some suggested style changes in 38f1864 that I have made in a follow on PR #923


#[test]
fn parse_json_ops_without_colon() {
use self::JsonOperator;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified this test covers the changes in this PR by running this test without the code in this PR and it failed (as expected):

---- parse_json_ops_without_colon stdout ----
thread 'parse_json_ops_without_colon' panicked at 'assertion failed: `(left == right)`
  left: `Ok([Query(Query { with: None, body: Select(Select { distinct: None, top: None, projection: [UnnamedExpr(JsonAccess { left: Identifier(Ident { value: "a", quote_style: None }), operator: HashArrow, right: Identifier(Ident { value: "b", quote_style: None }) })], into: None, from: [], lateral_views: [], selection: None, group_by: [], cluster_by: [], distribute_by: [], sort_by: [], having: None, named_window: [], qualify: None }), order_by: [], limit: None, offset: None, fetch: None, locks: [] })])`,
 right: `Err(ParserError("Expected end of statement, found: >"))`: Parse results with PostgreSqlDialect are different from GenericDialect', src/test_utils.rs:60:21
stack backtrace:

@alamb
Copy link
Contributor

alamb commented Jul 20, 2023

This PR introduced a regression - #926

Here is a PR to fix the regression: #927

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants