[BUG] Manifest list files are not deleted

[sfc-gh-rxing]

Describe the bug

I noticed that the current implementation of dropTableWithPurge doesn’t delete snapshots. I took a look into the TaskExecution, TableCleanupTaskHandler, ManifestFileCleanupTaskHandler, and it appears that only the metadata JSON files, manifest files, and data files are removed.

To Reproduce

1. Launch Polaris locally
2. Create a Polaris catalog: polaris_catalog
3. Configure Spark to connect to the local Polaris instance
4. Create a namespace: create namespace ns;
5. Create an Iceberg table: create table ns.t1 (id INT, name STRING);
6. Insert some data into it: insert into ns.t1 values(1, 'XJKDC')
7. Do drop table with purge: drop table ns.t1 purge;
8. Check the s3 bucket to see if there are some manifest lists file remaining.

Actual Behavior

The snapshots files (manifest list files) are not deleted.

Expected Behavior

The table location should not have any files in it after we drop the table with purge.

Additional context

Introduced in this PR: #312

System information

MacOS
Lastest Polaris

[XJDKC]

Note: spark will clean all the files by itself. If we want Polaris to clean up all the files, we can use curl command to delete the table with purge.
e.g.

curl -s -X DELETE "https://localhost:8181/catalog/v1/my-catalog/namespaces/ns1/tables/t1?purgeRequested=true" \
    -H "Authorization: Bearer ${BEARER_TOKEN}"

[singhpk234]

Resolved via Service: Cleanup Iceberg Manifest list on Purge #1039