Closed as not planned
Description
Query engine
Spark
Question
I thought this might do
val table = s"iceberg_catalog.${tableIdentifier}.branch_${branch}"
val t = Spark3Util.loadIcebergTable(spark, table)
val start = System.currentTimeMillis()
try {
SparkActions.get()
.rewriteDataFiles(t)
.skipPlanDeletes(skipPlanDeletes)
.filter(Expressions.equal("ds", 20230923))
.execute()
I was assuming the data is read from the branch, and the rewrite the result is written onto the branch
but it is not, seems the change is still visible on main.
Activity