-
Notifications
You must be signed in to change notification settings - Fork 319
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[#3460] feat(api): Add API design for Tag system #3486
Merged
Merged
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
56d790e
Add API design for Tag system
jerryshao 5d7724b
Add javadoc for the APIs
jerryshao deae5e6
abstract a metadata object interface to be used for tag and auth
jerryshao adc46ed
address the comments
jerryshao 13b14dc
address the comments
jerryshao 0039e2d
Address the comments
jerryshao File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
79 changes: 79 additions & 0 deletions
79
api/src/main/java/com/datastrato/gravitino/MetadataObject.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,79 @@ | ||
/* | ||
* Copyright 2024 Datastrato Pvt Ltd. | ||
* This software is licensed under the Apache License version 2. | ||
*/ | ||
|
||
package com.datastrato.gravitino; | ||
|
||
import com.datastrato.gravitino.annotation.Unstable; | ||
import javax.annotation.Nullable; | ||
|
||
/** | ||
* The MetadataObject is the basic unit of the Gravitino system. It represents the metadata object | ||
* in the Gravitino system. The object can be a metalake, catalog, schema, table, topic, etc. | ||
*/ | ||
@Unstable | ||
public interface MetadataObject { | ||
/** | ||
* The type of object in the Gravitino system. Every type will map one kind of the entity of the | ||
* underlying system. | ||
*/ | ||
enum Type { | ||
/** | ||
* A metalake is a concept of tenant. It means an organization. A metalake contains many data | ||
* sources. | ||
*/ | ||
METALAKE, | ||
/** | ||
* A catalog is a collection of metadata from a specific metadata source, like Apache Hive | ||
* catalog, Apache Iceberg catalog, JDBC catalog, etc. | ||
*/ | ||
CATALOG, | ||
/** | ||
* A schema is a sub collection of the catalog. The schema can contain filesets, tables, topics, | ||
* etc. | ||
*/ | ||
SCHEMA, | ||
/** A fileset is mapped to a directory on a file system like HDFS, S3, ADLS, GCS, etc. */ | ||
FILESET, | ||
/** A table is mapped the table of relational data sources like Apache Hive, MySQL, etc. */ | ||
TABLE, | ||
/** | ||
* A topic is mapped the topic of messaging data sources like Apache Kafka, Apache Pulsar, etc. | ||
*/ | ||
TOPIC, | ||
/** A column is a sub-collection of the table that represents a group of same type data. */ | ||
COLUMN | ||
} | ||
|
||
/** | ||
* The parent full name of the object. If the object doesn't have parent, this method will return | ||
* null. | ||
* | ||
* @return The parent full name of the object. | ||
*/ | ||
@Nullable | ||
String parent(); | ||
|
||
/** | ||
* The name of th object. | ||
* | ||
* @return The name of the object. | ||
*/ | ||
String name(); | ||
|
||
/** | ||
* The full name of th object. Full name will be separated by "." to represent a string identifier | ||
* of the object, like catalog, catalog.table, etc. | ||
* | ||
* @return The name of the object. | ||
*/ | ||
String fullName(); | ||
|
||
/** | ||
* The type of the object. | ||
* | ||
* @return The type of the object. | ||
*/ | ||
Type type(); | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
35 changes: 35 additions & 0 deletions
35
api/src/main/java/com/datastrato/gravitino/exceptions/NoSuchTagException.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
/* | ||
* Copyright 2024 Datastrato Pvt Ltd. | ||
* This software is licensed under the Apache License version 2. | ||
*/ | ||
|
||
package com.datastrato.gravitino.exceptions; | ||
|
||
import com.google.errorprone.annotations.FormatMethod; | ||
|
||
/** Exception thrown when a tag with specified name is not existed. */ | ||
public class NoSuchTagException extends NotFoundException { | ||
|
||
/** | ||
* Constructs a new exception with the specified detail message. | ||
* | ||
* @param message the detail message. | ||
* @param args the arguments to the message. | ||
*/ | ||
@FormatMethod | ||
public NoSuchTagException(String message, Object... args) { | ||
super(message, args); | ||
} | ||
|
||
/** | ||
* Constructs a new exception with the specified detail message and cause. | ||
* | ||
* @param cause the cause. | ||
* @param message the detail message. | ||
* @param args the arguments to the message. | ||
*/ | ||
@FormatMethod | ||
public NoSuchTagException(Throwable cause, String message, Object... args) { | ||
super(cause, message, args); | ||
} | ||
} |
35 changes: 35 additions & 0 deletions
35
api/src/main/java/com/datastrato/gravitino/exceptions/TagAlreadyExistsException.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
/* | ||
* Copyright 2024 Datastrato Pvt Ltd. | ||
* This software is licensed under the Apache License version 2. | ||
*/ | ||
|
||
package com.datastrato.gravitino.exceptions; | ||
|
||
import com.google.errorprone.annotations.FormatMethod; | ||
|
||
/** Exception thrown when a tag with specified name already exists. */ | ||
public class TagAlreadyExistsException extends AlreadyExistsException { | ||
|
||
/** | ||
* Constructs a new exception with the specified detail message. | ||
* | ||
* @param message the detail message. | ||
* @param args the arguments to the message. | ||
*/ | ||
@FormatMethod | ||
public TagAlreadyExistsException(String message, Object... args) { | ||
super(message, args); | ||
} | ||
|
||
/** | ||
* Constructs a new exception with the specified detail message and cause. | ||
* | ||
* @param cause the cause. | ||
* @param message the detail message. | ||
* @param args the arguments to the message. | ||
*/ | ||
@FormatMethod | ||
public TagAlreadyExistsException(Throwable cause, String message, Object... args) { | ||
super(cause, message, args); | ||
} | ||
} |
49 changes: 49 additions & 0 deletions
49
api/src/main/java/com/datastrato/gravitino/tag/SupportsTags.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
/* | ||
* Copyright 2024 Datastrato Pvt Ltd. | ||
* This software is licensed under the Apache License version 2. | ||
*/ | ||
|
||
package com.datastrato.gravitino.tag; | ||
|
||
import com.datastrato.gravitino.annotation.Evolving; | ||
import com.datastrato.gravitino.exceptions.NoSuchTagException; | ||
|
||
/** | ||
* Interface for supporting getting or associate tags to objects. This interface will be mixed with | ||
* metadata objects to provide tag operations. | ||
*/ | ||
@Evolving | ||
public interface SupportsTags { | ||
|
||
/** | ||
* List all the tag names for the specific object. | ||
* | ||
* @return The list of tag names. | ||
*/ | ||
String[] listTags(); | ||
|
||
/** | ||
* List all the tags with details for the specific object. | ||
* | ||
* @return The list of tags. | ||
*/ | ||
Tag[] listTagsInfo(); | ||
|
||
/** | ||
* Get a tag by its name for the specific object. | ||
* | ||
* @param name The name of the tag. | ||
* @return The tag. | ||
*/ | ||
Tag getTag(String name) throws NoSuchTagException; | ||
|
||
/** | ||
* Associate tags to the specific object. The tagsToAdd will be added to the object, and the | ||
* tagsToRemove will be removed from the object. | ||
* | ||
* @param tagsToAdd The arrays of tag name to be added to the object. | ||
* @param tagsToRemove The array of tag name to be removed from the object. | ||
* @return The array of tag names that are associated with the object. | ||
*/ | ||
String[] associateTags(String[] tagsToAdd, String[] tagsToRemove); | ||
} |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for this method, the requestor need to know which tags are new, which are existed but to be removed, this might be a little complicated for the requestor; how about change it to "applyTags(String[] tags)", so that each time the requestor pass in a full set of the tags, no need to distinguish which are new and which are to be remove;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From the current API, users still don't need to distinguish whether the tag is new or not. They can add all the tags they want to
tagsToAdd
, I will handle this in the backend.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, the backend will compare and recognize that, which is good for the client.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If a tag appears in two arrays at the same time, what will be the result?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It will be no-op. Same as if we both enable
setProperty
andremoveProperty
on the same key for alterTable.