Skip to content

HDFS-15219. DFS Client will stuck when ResponseProcessor.run throw Error #1902

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 24, 2020

Conversation

isahekmat
Copy link
Contributor

No description provided.

@ayushtkn
Copy link
Member

Can we extend a UT for the issue?

@isahekmat
Copy link
Contributor Author

I tried, but as ResponseProcessor class is a private inner class it's almost impossible to mock its behavior ( throw Error in the middle of run() method)
even I tried to mock block field to throw an Error when calling setNumBytes(), but as it's final and ExtendedBlock and BlockToWrite classes are very cautious to not expose their internal object it is impossible to inject my mock instead of their Block object.
can you help me to find a way to write its unit test?

@ayushtkn
Copy link
Member

If there is no easy way out for the UT. I think it is Ok then.
Yetus isn't behaving the way it should, Let me check, if there aren't any complains for Jenkins.
This should be fair enough..

@ayushtkn
Copy link
Member

-1 overall
04:08:57
04:08:57 | Vote | Subsystem | Runtime | Comment
04:08:57 ============================================================================
04:08:57 | 0 | reexec | 0m 34s | Docker mode activated.
04:08:57 | | | | Prechecks
04:08:57 | +1 | dupname | 0m 0s | No case conflicting files found.
04:08:57 | +1 | @author | 0m 0s | The patch does not contain any @author
04:08:57 | | | | tags.
04:08:57 | -1 | test4tests | 0m 0s | The patch doesn't appear to include
04:08:57 | | | | any new or modified tests. Please
04:08:57 | | | | justify why no new tests are needed for
04:08:57 | | | | this patch. Also please list what
04:08:57 | | | | manual steps were performed to verify
04:08:57 | | | | this patch.
04:08:57 | | | | trunk Compile Tests
04:08:57 | +1 | mvninstall | 20m 13s | trunk passed
04:08:57 | +1 | compile | 0m 48s | trunk passed
04:08:57 | +1 | checkstyle | 0m 25s | trunk passed
04:08:57 | +1 | mvnsite | 0m 51s | trunk passed
04:08:57 | +1 | shadedclient | 14m 24s | branch has no errors when building and
04:08:57 | | | | testing our client artifacts.
04:08:57 | +1 | javadoc | 0m 35s | trunk passed
04:08:57 | 0 | spotbugs | 2m 20s | Used deprecated FindBugs config;
04:08:57 | | | | considering switching to SpotBugs.
04:08:57 | +1 | findbugs | 2m 18s | trunk passed
04:08:57 | | | | Patch Compile Tests
04:08:57 | +1 | mvninstall | 0m 46s | the patch passed
04:08:57 | +1 | compile | 0m 43s | the patch passed
04:08:57 | +1 | javac | 0m 43s | the patch passed
04:08:57 | +1 | checkstyle | 0m 19s | the patch passed
04:08:57 | +1 | mvnsite | 0m 45s | the patch passed
04:08:57 | +1 | whitespace | 0m 0s | The patch has no whitespace issues.
04:08:57 | +1 | shadedclient | 13m 9s | patch has no errors when building and
04:08:57 | | | | testing our client artifacts.
04:08:57 | +1 | javadoc | 0m 29s | the patch passed
04:08:57 | +1 | findbugs | 2m 17s | the patch passed
04:08:57 | | | | Other Tests
04:08:57 | +1 | unit | 2m 2s | hadoop-hdfs-client in the patch
04:08:57 | | | | passed.
04:08:57 | +1 | asflicense | 0m 29s | The patch does not generate ASF
04:08:57 | | | | License warnings.
04:08:57 | | | 63m 11s |
04:08:57
04:08:57
04:08:57 || Subsystem || Report/Notes ||
04:08:57 ============================================================================
04:08:57 | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1902/1/artifact/out/Dockerfile |
04:08:57 | GITHUB PR | #1902 |
04:08:57 | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
04:08:57 | uname | Linux 24c38dc308fc 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
04:08:57 | Build tool | maven |
04:08:57 | Personality | personality/hadoop.sh |
04:08:57 | git revision | trunk / f9bb2a8 |
04:08:57 | Default Java | 1.8.0_242 |
04:08:57 | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1902/1/testReport/ |
04:08:57 | Max. process+thread count | 424 (vs. ulimit of 5500) |
04:08:57 | modules | C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client |
04:08:57 | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1902/1/console |
04:08:57 | versions | git=2.17.1 maven=3.6.0�[m findbugs=3.1.0-RC1 |
04:08:57 | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |

Copy link
Member

@ayushtkn ayushtkn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No Complains for yetus, Have manually pasted the result.
+1
WIll merge in couple of days, if no objections

@ayushtkn ayushtkn changed the title HDFS-15219. fix ResponseProcessor.run to catch Throwable instead of Exception HDFS-15219. DFS Client will stuck when ResponseProcessor.run throw Error Mar 24, 2020
@ayushtkn ayushtkn merged commit d9c4f11 into apache:trunk Mar 24, 2020
@isahekmat isahekmat deleted the HDFS-15219 branch March 24, 2020 18:01
RogPodge pushed a commit to RogPodge/hadoop that referenced this pull request Mar 25, 2020
asfgit pushed a commit that referenced this pull request Mar 26, 2020
…ror (#1902). Contributed by  zhengchenyu.

(cherry picked from commit d9c4f11)
asfgit pushed a commit that referenced this pull request Mar 26, 2020
…ror (#1902). Contributed by  zhengchenyu.

(cherry picked from commit d9c4f11)
bentito pushed a commit to bentito/hadoop that referenced this pull request Dec 2, 2020
…ror (apache#1902). Contributed by  zhengchenyu.

(cherry picked from commit d9c4f11)
bentito pushed a commit to bentito/hadoop that referenced this pull request Dec 3, 2020
…ror (apache#1902). Contributed by  zhengchenyu.

(cherry picked from commit d9c4f11)
zhangxiping1 pushed a commit to zhangxiping1/hadoop that referenced this pull request Dec 13, 2022
jojochuang added a commit to jojochuang/hadoop that referenced this pull request May 23, 2023
…ror (apache#1902). Contributed by  zhengchenyu.

(cherry picked from commit d9c4f11)
(cherry picked from commit d64f688)
Change-Id: I8945bafc655130f187afb8cd833ffc37bbe76eb0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants