-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Validate timeColumnName when adding/updating schema/tableConfig #5966
Conversation
We do allow to have tables without a schema. I have not had the time to go through this fully, but do you account for the fact that a (offline) table may not have a schema at all? |
Yes, I have accounted for offline table not having schema at all. I've put in the description all the cases it covers, and the case we miss out |
...ller/src/main/java/org/apache/pinot/controller/api/resources/PinotSchemaRestletResource.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor comments
...oller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableIndexingConfigs.java
Show resolved
Hide resolved
pinot-core/src/main/java/org/apache/pinot/core/util/TableConfigUtils.java
Show resolved
Hide resolved
// empty list | ||
List<TableConfig> tableConfigs = new ArrayList<>(); | ||
Schema schema = new Schema.SchemaBuilder().setSchemaName(TABLE_NAME).build(); | ||
SchemaUtils.validate(schema, tableConfigs); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: wrap with try-catch for future proofing ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do you mean a case where the schema becomes invalid, but we pass the test anyway because of not having a try-catch?
Just to be sure, I tried this out. The test fails if there's any exception in the should-pass cases.
Lmk if you were referring to something else.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I meant catching exception and explicitly 'fail'ing. It's fine though - just a nit.
// offline table | ||
// null timeColumnName | ||
TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName(TABLE_NAME).build(); | ||
SchemaUtils.validate(schema, Lists.newArrayList(tableConfig)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM with minor comments
Description
#5915
Add validation to check that timeColumnName is in sync across the table config and schema.
Cases
Corner case
If tableConfig has been created with a schemaName != rawTableName, then we will not be able to validate during schema add/upload.
Release Notes
After this change,