-
Notifications
You must be signed in to change notification settings - Fork 8.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HADOOP-17458. S3A to treat "SdkClientException: Data read has a different length than the expected" as EOFException #3040
HADOOP-17458. S3A to treat "SdkClientException: Data read has a different length than the expected" as EOFException #3040
Conversation
…ent length than the expected" as EOFException Some network exceptions would cause `SdkClientException` to fail with a `Data read has a different length than the expected` message. These should be recoverable. - Modifying `translateException` to recover from the above exception;
Added a test to check if the errors which we're translating by looking at the message are actually doing what we're expecting.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
code looks good.
we really some unit test for this, don't we? (we should have had one from the outset....)
Could you add one in org.apache.hadoop.fs.s3a.TestInvoker ? That'll stop us breaking this in future
Adding new unit tests to make sure that the new translatable exceptions are being translated and retried appropriately.
Sure thing 👍 ! Added them. |
🎊 +1 overall
This message was automatically generated. |
@steveloughran let me know if the tests are ok, it's been a while for this PR and wanted to get it closed :D |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
checkstyle is all line length. I'm Ok with that if it's not too wide and cutting down would actually hurt readability
(oh, two commits on trunk earlier: https://issues.apache.org/jira/browse/HADOOP-17813 , line length is now 100 chars!) |
…rent length than the expected" as EOFException (#3040) Some network exceptions can raise SdkClientException with message `Data read has a different length than the expected`. These should be recoverable. Contributed by Bogdan Stolojan Change-Id: Ia22fd77d90971e9e02b4f947398a4749eebe5909
…rent length than the expected" as EOFException (apache#3040) Some network exceptions can raise SdkClientException with message `Data read has a different length than the expected`. These should be recoverable. Contributed by Bogdan Stolojan
HADOOP-17458
Some network exceptions would cause
SdkClientException
to fail with aData read has a different length than the expected
message. Theseshould be recoverable.
translateException
to recover from the above exception;Tested with
eu-west-1
andmvn -Dparallel-tests -DtestsThreadCount=32 clean verify