Spark 3.1: Use snapshot schema for time travel queries #3768
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Port of #3722 to v3.1.
Notes:
I took the diff from #3722, edited it to change v3.2 to v3.1 in the paths, and tried to
git apply 3722.diff
. That did not apply, so I didgit am -3 < 3722.diff
, which left conflicts in SparkTable.java and TestDeleteFrom.java.The conflicts in SparkTable.java were straightforward to resolve.
TestDeleteFrom
in v3.2 hastestDeleteFromUnpartitionedTable
andtestDeleteFromPartitionedTable
which are note present in v3.1; I had based the new test,testDeleteFromTableAtSnapshot
ontestDeleteFromUnpartitionedTable
, and it is still usable because we don't actually delete anything, just check that an exception is thrown.Also, for the negative test cases added in
TestDeleteFrom
andTestUnpartitionedWrites
, I had to exclude theSparkSessionCatalog
configuration from the parameterized testing, as otherwise I getinstead of the
IllegalArgumentException
I was looking for.