Skip to content

[#12220] feat(core): Add active deletion schema and models - #12222

Open
nevzheng wants to merge 1 commit into
apache:mainfrom
nevzheng:codex/iceberg-rest-delete-10-01-schema-models
Open

[#12220] feat(core): Add active deletion schema and models#12222
nevzheng wants to merge 1 commit into
apache:mainfrom
nevzheng:codex/iceberg-rest-delete-10-01-schema-models

Conversation

@nevzheng

@nevzheng nevzheng commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

What changes were proposed in this pull request?

This is PR 1 of 2 in the deletion-generation storage stack.

  • Add a nullable, unique table_meta.deletion_id pointer for one active table-deletion generation.
  • Add a minimal entity_deletion table containing deletion_id, lifecycle state, the fixed retention_expires_at deadline, and an optional purge_job_id.
  • Add indexes for retained-root discovery, expiry selection, and purge-job association.
  • Add the corresponding persistent-object fields and conversion coverage.
  • Add fresh-install and 1.3-to-2.0 upgrade SQL for H2, MySQL, and PostgreSQL.
  • Ensure the SQL-script test actually executes discovered upgrade scripts.

This PR deliberately does not add deletion identifiers to table-owned child rows, duplicate table identity in the action, retain terminal receipts, or introduce deletion-specific audit storage.

Stack:

After this PR, relational storage can represent a retained table root and its minimal active deletion action. It does not yet change DELETE, UNDROP, or purge runtime behavior.

Why are the changes needed?

Soft deletion needs an opaque identifier for each active deletion generation and a fixed recovery deadline. The original table_meta row remains the source of table identity, name, parent, and deletion time, while entity_deletion stores only the short-lived lifecycle and future purge-job association.

Keeping the action minimal avoids duplicating metadata that already belongs to the retained table root and lets a later transaction layer remove the action after either successful restoration or final purge.

Part of #12220

Does this PR introduce any user-facing change?

No. This PR adds relational schema and persistent-object support only. It does not change public APIs or runtime deletion behavior.

How was this patch tested?

./gradlew :core:test \
  --tests org.apache.gravitino.storage.TestSQLScripts \
  --tests org.apache.gravitino.storage.relational.utils.TestPOConverters \
  -PskipITs -PskipDockerTests=true

@nevzheng
nevzheng force-pushed the codex/iceberg-rest-delete-10-01-schema-models branch from a7e0ffa to c50ede2 Compare July 27, 2026 23:19
@nevzheng nevzheng changed the title [#12220] feat(core): Add deletion lifecycle schema and models [#12220] feat(core): Add active deletion schema and models Jul 27, 2026
@nevzheng
nevzheng marked this pull request as ready for review July 27, 2026 23:19
@github-actions

Copy link
Copy Markdown

Code Coverage Report

Overall Project 67.92% +0.23% 🟢
Files changed 86.27% 🟢

Module Coverage
aliyun 1.72% 🔴
api 48.1% 🟢
authorization-common 85.96% 🟢
aws 42.04% 🟢
azure 2.47% 🔴
catalog-common 9.92% 🔴
catalog-fileset 79.74% 🟢
catalog-glue 66.91% 🟢
catalog-hive 79.4% 🟢
catalog-jdbc-common 45.7% 🟢
catalog-jdbc-doris 81.8% 🟢
catalog-jdbc-mysql 79.33% 🟢
catalog-jdbc-postgresql 83.39% 🟢
catalog-jdbc-starrocks 79.16% 🟢
catalog-kafka 77.01% 🟢
catalog-lakehouse-generic 59.18% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 85.86% 🟢
catalog-lakehouse-paimon 84.23% 🟢
catalog-model 77.72% 🟢
cli 44.5% 🟢
client-java 78.27% 🟢
common 52.07% 🟢
core 82.82% +0.15% 🟢
filesystem-hadoop3 77.28% 🟢
flink 0.0% 🔴
flink-common 47.09% 🟢
flink-runtime 0.0% 🔴
gcp 14.12% 🔴
hadoop-auth 68.0% 🟢
hadoop-common 12.7% 🔴
hive-metastore-common 53.4% 🟢
iceberg-common 64.65% 🟢
iceberg-rest-server 75.02% 🟢
idp-basic 86.02% 🟢
integration-test-common 0.0% 🔴
jobs 66.17% 🟢
lance-common 20.67% 🔴
lance-rest-server 64.23% 🟢
lineage 53.02% 🟢
optimizer 83.17% 🟢
optimizer-api 21.95% 🔴
server 85.89% 🟢
server-common 76.31% 🟢
spark 28.57% 🔴
spark-common 46.01% 🟢
tencent 69.84% 🟢
trino-connector 40.29% 🟢
Files
Module File Coverage
core POConverters.java 88.28% 🟢
TablePO.java 72.34% 🟢
EntityDeletionPO.java 0.0% 🔴

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant