[SPARK-33787][SQL][3.1] Add the purge parameter to dropPartition() of SupportsPartitionManagement

[MaxGekk]

What changes were proposed in this pull request?

1. Add the boolean flag purge to dropPartition() in SupportsPartitionManagement and dropPartitions() in SupportsAtomicPartitionManagement.
2. Propagate the parameter from the logical command to V2 exec node and further to catalog implementations.
3. Remove the default implementations of dropping partitions from the SupportsPartitionManagement and SupportsAtomicPartitionManagement interfaces.
4. Update Java docs for dropTable.

Why are the changes needed?

The sql statement ALTER TABLE .. DROP PARTITION allows to specify the PURGE flag but it is ignored by v2 implementation. We should propagate it at least to SupportsPartitionManagement and to SupportsAtomicPartitionManagement, and let to implementations of those interface decide how to support it.

Does this PR introduce any user-facing change?

Should not.

How was this patch tested?

By running new UT:

$ build/sbt -Phive-2.3 -Phive-thriftserver "test:testOnly *AlterTablePartitionV2SQLSuite"

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37545/

[SparkQA]

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37545/

[SparkQA]

Test build #132942 has finished for PR 30821 at commit 9c6ef11.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[MaxGekk]

jenkins, retest this, please

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37556/

[SparkQA]

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37556/

[SparkQA]

Test build #132953 has finished for PR 30821 at commit 6cd76f7.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.