Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix up incorrect PARTITIONED BY error messages #15961

Merged
merged 4 commits into from
Feb 26, 2024

Conversation

abhishekrb19
Copy link
Contributor

@abhishekrb19 abhishekrb19 commented Feb 24, 2024

Prior to this patch, an incorrect PARTITIONED BY clause can expose options like NONE in the returned error message:

Invalid granularity[foo] specified after PARTITIONED BY clause. Expected SECOND, MINUTE, FIVE_MINUTE, TEN_MINUTE, FIFTEEN_MINUTE, THIRTY_MINUTE, HOUR, SIX_HOUR, EIGHT_HOUR, DAY, MONTH, QUARTER, YEAR, ALL, NONE, ALL TIME, FLOOR() or TIME_FLOOR()

Also, some of these granularities like SECOND and MINUTE are reserved SQL keywords that are not handled explicitly in Druid. In order for them to be properly used they should be quoted as literals. So single quote the granularity literals in the error message making it clear to a user about the correct usage.

Other related changes:

  • A general overall clean up of DruidSqlParserUtils.java:
    • A mix of Precondition checks, ValidationException and DruidException were thrown making the calling code catch these different exceptions and unify them. So migrate all the exception handling usages in this class to DruidException and add additional context to messages as needed.
    • Remove unused method, modify a few method access modifiers, removed/added @VisibleForTesting, etc.
  • Added a couple of tests for the literal granularity case noted above.
  • Fix up some typos and inaccuracies in the javadocs for the partitioned by area of code.
Key changed/added classes in this PR
  • DruidSqlParserUtils.java
  • DruidSqlParserUtilsTest.java

This PR has:

  • been self-reviewed.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • been tested in a test Druid cluster.

@@ -296,8 +296,6 @@ The string constant can also include any of the keywords mentioned above:
- `ALL TIME`
- `ALL` - Alias for `ALL TIME`

The `WEEK` granularity is deprecated and not supported in MSQ.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This conflicts with the note below, which is more accurate:

*Avoid partitioning by week, P1W, because weeks don't align neatly with months and years, making it difficult to partition by coarser granularities later.

@@ -54,13 +54,6 @@ public SqlGranularityLiteral clone(SqlParserPos pos)
return new SqlGranularityLiteral(granularity, unparseString, pos);
}

@Override
@Deprecated
public Object clone()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thought overriding a deprecated method for the sake of throwing an exception was not really needed.

))
.expectTarget("dst", targetRowSignature)
.expectResources(dataSourceRead("foo"), dataSourceWrite("dst"))
.expectQuery(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we test the expected granularity here too?

@zachjsh zachjsh merged commit 67a6224 into apache:master Feb 26, 2024
83 checks passed
@adarshsanjeev adarshsanjeev added this to the 30.0.0 milestone May 6, 2024
@abhishekrb19 abhishekrb19 deleted the partitioned_by_bug branch May 6, 2024 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants