[AMORO-3424] Add the self-optimizing.partition-filter parameter

[lintingbin]

Why are the changes needed?

Close #3424 .

Brief change log

• Add the self-optimizing.partition-filter parameter.

How was this patch tested?

• Add some test cases that check the changes thoroughly including negative and positive cases if possible

• Add screenshots for manual tests if appropriate

• Run test locally before making a pull request

Documentation

• Does this pull request introduce a new feature? yes
• If yes, how is the feature documented? docs

[lintingbin]

Here is my current simple implementation code. Please provide some feedback, and I will continue to improve it.

[zhoujinsong]

Thanks for the contribution!

The Spark core is a somewhat heavy lib to add to the common module.
JSqlParser is a lightweight lib to parse SQL, it may help.

[lintingbin]

Thanks for the contribution!

The Spark core is a somewhat heavy lib to add to the common module. JSqlParser is a lightweight lib to parse SQL, it may help.

Okay, I'll research JSqlParser.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 70.68966% with 34 lines in your changes missing coverage. Please review.

Project coverage is 27.88%. Comparing base (63aab48) to head (c66f090).
Report is 15 commits behind head on master.

Files with missing lines	Patch %	Lines
...in/java/org/apache/amoro/utils/ExpressionUtil.java	78.43%	11 Missing and 11 partials ⚠️
...java/org/apache/amoro/config/OptimizingConfig.java	0.00%	5 Missing ⚠️
...o/optimizing/plan/AbstractOptimizingEvaluator.java	0.00%	3 Missing ⚠️
...oro/optimizing/plan/AbstractOptimizingPlanner.java	0.00%	3 Missing ⚠️
...n/java/org/apache/amoro/table/TableProperties.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3426      +/-   ##
============================================
+ Coverage     21.59%   27.88%   +6.29%     
- Complexity     2353     3660    +1307     
============================================
  Files           431      603     +172     
  Lines         40347    49328    +8981     
  Branches       5711     6367     +656     
============================================
+ Hits           8712    13756    +5044     
- Misses        30903    34607    +3704     
- Partials        732      965     +233     

Flag	Coverage Δ
core	27.88% <70.68%> (?)
trino	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[lintingbin]

@zhoujinsong @czy006 I have finished the development. Please help review it. I hope it can be merged into the master before the release of 0.8.0.

[zhoujinsong]

@lintingbin Thanks for the work!

I left some comments, PTAL.

[lintingbin]

@lintingbin Thanks for the work!

I left some comments, PTAL.

It has been resolved. Please review it again.

[czy006]

Thank you for your contribution. I commented on some of the doubts.

[zhoujinsong]

Thanks for the work! @lintingbin