Skip to content

[#12221] feat(iceberg): Add durable table delete lifecycle - #12225

Draft
nevzheng wants to merge 5 commits into
apache:mainfrom
nevzheng:codex/iceberg-rest-delete-20-01-delete-lifecycle
Draft

[#12221] feat(iceberg): Add durable table delete lifecycle#12225
nevzheng wants to merge 5 commits into
apache:mainfrom
nevzheng:codex/iceberg-rest-delete-20-01-delete-lifecycle

Conversation

@nevzheng

@nevzheng nevzheng commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

What changes were proposed in this pull request?

This is Stack 20.1 of 3 and depends on #12223.

  • Add the Iceberg REST soft-delete coordinator and configuration.
  • When soft delete is enabled, replace the physical request-thread drop with one relational transaction that:
    • locks the exact live table_meta row by immutable table_id with SELECT ... FOR UPDATE;
    • inserts a minimal active deletion action (deletion_id, state, retention_expires_at, purge_job_id); and
    • stamps deleted_at and the same deletion_id on the locked row using its current schema, name, and metadata version.
  • Keep table identity, name, version, ownership, and related metadata on the existing immutable table_id; the action does not duplicate them.
  • Reserve retained names for create/register and hide retained roots from ordinary table operations.
  • Treat a repeated DELETE of the already-retained name as successful.
  • Preserve the existing DELETE paths when soft delete is disabled.

This layer intentionally does not add terminal restore receipts, copied identity fields, action revisions, or a deletion-audit table. Existing entity change events remain the cache-propagation mechanism.

Review order: #12222#12223 → this PR → #12226#12227.

Incremental diff:
nevzheng/gravitino@codex/iceberg-rest-delete-10-02-action-storage...codex/iceberg-rest-delete-20-01-delete-lifecycle

Why are the changes needed?

This establishes the durable logical-delete boundary used by deleted-table reads, UNDROP, and a later asynchronous purge implementation without introducing a request-thread saga.

Related: #12221

Does this PR introduce any user-facing change?

Yes. When Iceberg REST soft deletion is enabled, DELETE retains and hides the existing table row and reserves its name. When it is disabled, existing behavior is unchanged.

How was this patch tested?

Added cross-backend relational lifecycle tests plus configuration, authorization, REST DELETE, dispatcher, name-reservation, retry, locked-current-version, and outer-transaction rollback tests. The focused Core and Iceberg REST Server suites pass.

@nevzheng
nevzheng force-pushed the codex/iceberg-rest-delete-20-01-delete-lifecycle branch 2 times, most recently from 2c226a1 to 634f7eb Compare July 28, 2026 01:01
@github-actions

Copy link
Copy Markdown

Code Coverage Report

Overall Project 67.96% +0.42% 🟢
Files changed 82.78% 🟢

Module Coverage
aliyun 1.72% 🔴
api 48.1% 🟢
authorization-common 85.96% 🟢
aws 42.04% 🟢
azure 2.47% 🔴
catalog-common 9.92% 🔴
catalog-fileset 79.74% 🟢
catalog-glue 66.91% 🟢
catalog-hive 79.4% 🟢
catalog-jdbc-common 45.7% 🟢
catalog-jdbc-doris 81.8% 🟢
catalog-jdbc-mysql 79.33% 🟢
catalog-jdbc-postgresql 83.39% 🟢
catalog-jdbc-starrocks 79.16% 🟢
catalog-kafka 77.01% 🟢
catalog-lakehouse-generic 59.18% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 85.93% 🟢
catalog-lakehouse-paimon 84.23% 🟢
catalog-model 77.72% 🟢
cli 44.5% 🟢
client-java 78.27% 🟢
common 52.07% 🟢
core 82.77% +0.1% 🟢
filesystem-hadoop3 77.28% 🟢
flink 0.0% 🔴
flink-common 47.09% +1.08% 🟢
flink-runtime 0.0% 🔴
gcp 14.12% 🔴
hadoop-auth 68.0% 🟢
hadoop-common 12.7% 🔴
hive-metastore-common 53.4% 🟢
iceberg-common 65.01% +10.26% 🟢
iceberg-rest-server 75.42% +0.42% 🟢
idp-basic 86.02% 🟢
integration-test-common 0.0% 🔴
jobs 66.17% 🟢
lance-common 20.67% 🔴
lance-rest-server 64.23% 🟢
lineage 53.02% 🟢
optimizer 83.24% 🟢
optimizer-api 21.95% 🔴
server 85.89% 🟢
server-common 76.31% -0.03% 🟢
spark 28.57% 🔴
spark-common 46.01% 🟢
tencent 69.84% 🟢
trino-connector 40.29% 🟢
Files
Module File Coverage
core DefaultMapperPackageProvider.java 100.0% 🟢
TableMetaPostgreSQLProvider.java 100.0% 🟢
EntityDeletionPO.java 100.0% 🟢
EntityDeletionService.java 100.0% 🟢
POConverters.java 88.28% 🟢
EntityDeletionSQLProvider.java 80.0% 🟢
TablePO.java 72.34% 🟢
TableMetaBaseSQLProvider.java 70.0% 🟢
TableDeletionService.java 55.81% 🔴
EntityDeletionMapper.java 0.0% 🔴
TableDeletionMapper.java 0.0% 🔴
flink-common GravitinoPaimonCatalog.java 68.32% 🟢
PropertyUtils.java 0.0% 🔴
iceberg-common IcebergConfig.java 100.0% 🟢
iceberg-rest-server IcebergDeletionAuthorization.java 100.0% 🟢
IcebergRetainedTableDeletion.java 100.0% 🟢
IcebergNamespaceOperationExecutor.java 100.0% 🟢
IcebergTableOperationExecutor.java 98.41% 🟢
IcebergTableEventDispatcher.java 94.44% 🟢
IcebergTableDeletionLifecycle.java 90.32% 🟢
IcebergTableOperations.java 82.55% 🟢
IcebergTableHookDispatcher.java 81.73% 🟢
RESTService.java 0.0% 🔴
IcebergTableOperationDispatcher.java 0.0% 🔴
server-common AuthorizationExpressionConstants.java 0.0% 🔴

@nevzheng
nevzheng force-pushed the codex/iceberg-rest-delete-20-01-delete-lifecycle branch from 634f7eb to 55fdd1f Compare July 28, 2026 03:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant