Make upsert metadata manager pluggable

[Jackie-Jiang]

• Extracted interface for TableUpsertMetadataManager and PartitionUpsertMetadataManager
• Added 2 new fields in UpsertConfig for custom class and properties: metadataManagerClass and properties

[codecov-commenter]

Codecov Report

Merging #9186 (3d87195) into master (6ed17d2) will decrease coverage by 1.58%.
The diff coverage is 62.92%.

❗ Current head 3d87195 differs from pull request most recent head 9d6d172. Consider uploading reports for the commit 9d6d172 to get more accurate results

@@             Coverage Diff              @@
##             master    #9186      +/-   ##
============================================
- Coverage     70.04%   68.45%   -1.59%     
- Complexity     5003     5005       +2     
============================================
  Files          1852     1854       +2     
  Lines         98760    98782      +22     
  Branches      15021    15025       +4     
============================================
- Hits          69175    67625    -1550     
- Misses        24709    26373    +1664     
+ Partials       4876     4784      -92     

Flag	Coverage Δ
integration1	?
integration2	24.76% <0.74%> (-0.09%)	⬇️
unittests1	67.14% <62.17%> (+<0.01%)	⬆️
unittests2	15.27% <0.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...local/indexsegment/mutable/MutableSegmentImpl.java	54.25% <ø> (ø)
.../apache/pinot/segment/local/upsert/RecordInfo.java	100.00% <ø> (ø)
...apache/pinot/segment/local/upsert/UpsertUtils.java	33.33% <33.33%> (ø)
...rg/apache/pinot/spi/config/table/UpsertConfig.java	73.17% <33.33%> (-6.83%)	⬇️
...ocal/upsert/TableUpsertMetadataManagerFactory.java	47.05% <47.05%> (ø)
...t/local/upsert/BaseTableUpsertMetadataManager.java	55.55% <55.55%> (ø)
...ata/manager/realtime/RealtimeTableDataManager.java	68.35% <66.66%> (+1.31%)	⬆️
...t/ConcurrentMapPartitionUpsertMetadataManager.java	67.66% <67.66%> (ø)
...psert/ConcurrentMapTableUpsertMetadataManager.java	100.00% <100.00%> (ø)
...pinot/minion/exception/TaskCancelledException.java	0.00% <0.00%> (-100.00%)	⬇️
... and 145 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[KKcorps]

Can the DedupManager be accomodated to implement this interface or should we do something else? IMO, it doesn't make sense to keep a seperate dedup config as as well as a seperate implementation with these changes.

[Jackie-Jiang]

Can the DedupManager be accomodated to implement this interface or should we do something else? IMO, it doesn't make sense to keep a seperate dedup config as as well as a seperate implementation with these changes.

We may revisit the dedup after finishing the upsert improvement. IMO dedup won't share the same config or interface because they are different feature and trying to address different problems. The initial version of dedup shares a lot of code with upsert, but in the future we might want to handle them differently.

[KKcorps]

Makes sense. Overall LGTM!