[EPIC] Add file type catalog support for Gravitino

[jerryshao]

Describe the proposal

Gravitino currently only supports relational type catalog, with the increased requirements of managing datasets in object store or file systems, we should add the file type catalog support in Gravitino.stores

Task list

• [Subtask] Investigate on how non-tabular data is managed #1284
• [Subtask]Complete the metadata and interface design for file type catalog #1244
• [Subtask] Add the code skeleton for fileset catalog #1520
• [Subtask] Add Schema support fileset catalog #1526
• [Subtask] Add Fileset support for fileset catalog #1527
• [Subtask] Adapt the Hadoop catalog to the CatalogOperationDispatcher #1667
• [Subtask] Reference implementation of file catalog to support local files #1489
• [FEATURE] Add Gravitino Hadoop File System #1616
• [Subtask] Consider upgrading Hadoop client to version 3 #1713
• [Subtask] Add server-side REST API support for fileset #1724
• [Subtask] Add Java client support for fileset #1838
• [Subtask] Add E2E integration test for fileset #2041
• [Subtask] Add Python Arrow FileSystem implementation for fileset. #2059
• [Subtask] enable hadoop catalog module in Gravitino #2287
• [Subtask] Add web UI support for fileset catalog #2292
• [Subtask] Formalize the storage location when stored into entity store #2307
• [Subtask] Add open API doc for fileset catalog #2367
• [Subtask] Add document about how to use fileset type catalog #2695

[coolderli]

@jerryshao Hi, I'd like to continue to work on these issues:

• Add Java client support for fileset
• Add E2E integration test for fileset

Are you working on them? Will this affect the release of version 0.4？

The last issue Add Python Arrow FileSystem implementation for fileset. may be beyond my current ability range. So I want to finish the above two issues first.

[jerryshao]

@coolderli I don't work on this, these two will not be in 0.4.0, it will target to 0.5.0. You can go ahead if you'd like to.