HADOOP-16255. Add ChecksumFs.rename(path, path, boolean) to rename crc file as well when FileContext.rename(path, path, options) is called. [BRANCH-2] #1413
Conversation
|
cc. @steveloughran HadoopTestBase doesn't seem to be available in branch-2 so I removed it. Except that it was clean cherry-pick. Let's see the build result. |
| /** | ||
| * This class tests the functionality of ChecksumFs. | ||
| */ | ||
| public class TestChecksumFs { |
There was a problem hiding this comment.
copy in the TestTimeout rule from the HadoopTestBase class; we need all new tests to timeout before mvn gets bored and kills the process.
There was a problem hiding this comment.
Ah OK. I thought it would be fine as HadoopTestBase doesn't exist. I'll copy the rule here.
|
Don't see yetus picking this up; it seems to like trunk the best. Can you attach it as a .patch to the JIRA with a name like HADOOP-16255-branch-2-001.patch; yetus should know to run it against branch-2 |
HeartSaVioR
changed the title
|
I just subscribed the common-dev@ and recently there is a discussion regarding build - looks like it recognizes JIRA number, so maybe it requires JIRA number to be prefixed (though not sure Yetus recognizes branch). Let me just try to modify title and wait for a hour: if Yetus didn't run I'll add a patch file to JIRA issue. |
|
I just attached the patch file to JIRA issue - looks like Yetus doesn't work for this case. Closing. |
Please refer https://issues.apache.org/jira/browse/HADOOP-16255 for more details.
FYI, FileContext.rename(path, path, options) leaks crc file for source of rename when CheckFs or its descendant is used as underlying filesystem. https://issues.apache.org/jira/browse/SPARK-28025 took a workaround via removing crc file manually, and we hope to get rid of workaround eventually.
This PR is ported version of #1388 for branch-2.