Core: Make Metadata tables serializable

[pvary]

When used from Hive queries it would be useful to serialize the tables at the time of the query compilation for multiple reasons:

• If we use the same snapshot during the query execution we could have consistent results
• If we have do not have to access the catalog during the query execution then we can save HMS calls

The Serialization is implemented for BaseTables in #1920. This PR aims to do the same for the Metadata tables too.

Things which might worth to check:

• Moved the SerializationUtil class to the core package - currently needed only for the tests, but I thought this would be ok
• Every writeReplace() method is exactly the same for the specific types for metadata tables. Might worth to consider moving ops, table, name to the BaseMetadataTable. Did not do this because the change I did is not significant compared to the other quasi duplicated code, and there might be other reasons I am not aware of which would prevent this refactor.

[pvary]

@marton-bod, @lcspinter: Could you please review?
Thanks,
Peter

[lcspinter]

LGTM (non-binding)

Just a couple of observations.
We store the TableOperations, Table and name redundantly in each child class of BaseMetadataTable? Is there any reason we cannot move them one level higher?

Should we consider the introduction of a new ancestor class (SerializableBaseTable) for BaseTable and BaseMetadataTable? In my opinion, it would improve the readability of the code.

@rdblue What do you think?

[pvary]

@openinx, @aokolnychyi: After talking with @rdblue, he said you might be interested in reviewing this change.

Would you be so kind to review?

Thanks,
Peter

[openinx]

@pvary I skimmed this PR, seems I need more background to understand this change. Let me see the previous committed PRs.

[pvary]

@pvary I skimmed this PR, seems I need more background to understand this change. Let me see the previous committed PRs.

Thanks @openinx for taking the time to check the PR!
Feel free to ask any questions here/or on slack/or in email if you feel it is easier than digging up everything, I would be happy to answer them!

I would like to give some context - hope this helps:
With Hive, and maybe even for other execution engines too, the query compilation and the query execution happens on different nodes and we are only sending serialized data between the them. The execution also could happen in a distributed mode and it is unnecessary (and even problematic) for every executor node to look-up the table data from the Catalogs. If during the compilation we read the table data from the Catalog and then serialize, then the executor nodes do not have to have access to the Catalog, and it could be enough for them to have S3 access to read the snapshot data themselves.

In nutshell what we are trying to archive here to have a way to Serialize/Deserialize not only BaseTable-s, but every MetadataTable as well.

Thanks,
Peter

[rdblue]

This object has access to table(), io(), and name() already. Would it be easier to expose protected metadataLocation() and metadataTableType() methods instead of writeReplace() in each implementation? Then this method could be implemented here.

[pvary]

Followed your recommendation. LGTM

The thing is that metadataLocation() is exactly the same for every implementation, but we need this if we do not want to relax the private restriction on ops. We might be further simplify the code if we move ops to BaseMetadataTable.

What do you think? Or it would be a bigger change which do no worth to do?

[rdblue]

We could make ops a protected method. Let's get this in and we can clean that up later.