Skip to content

HDDS-1178. Healthy pipeline Chill Mode Rule. #518

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from

Conversation

bharatviswa504
Copy link
Contributor

No description provided.

@hadoop-yetus

This comment has been minimized.

@anuengineer
Copy link
Contributor

A couple of comments:

  1. Why is it 10% isn't that too low?
  2. I see that for each pipeline report arrival, we check the pipeline manager for the state -- to check if the pipeline is in healthy. Isn't there a race condition here? How do we guarantee that this check and the pipeline report update does not race each other?

@bharatviswa504
Copy link
Contributor Author

bharatviswa504 commented Feb 26, 2019

Thank You @anuengineer for the review.

  1. Low 10% is, as this rule main purpose is once we are out of chill mode, we have atleast few pipelines for writes to succeed. (As other rules like container chill mode rule, pipeline rule with at least one datanode reported by the time these completed, we might have more pipelines, this rule is more like a conservative side.) Let me know if you want to change it to any other default value or any other suggestion for the default value.
  2. Thanks for catching it. done.

@hadoop-yetus

This comment has been minimized.

@hadoop-yetus

This comment has been minimized.

@elek
Copy link
Member

elek commented Feb 26, 2019

One test failure seems to be related:

java.lang.AssertionError: class org.apache.hadoop.ozone.OzoneConfigKeys class org.apache.hadoop.hdds.scm.ScmConfigKeys class org.apache.hadoop.ozone.om.OMConfigKeys class org.apache.hadoop.hdds.HddsConfigKeys class org.apache.hadoop.ozone.s3.S3GatewayConfigKeys has 1 variables missing in ozone-default.xml Entries:   hdds.scm.chillmode.healthy.pipelie.pct expected:<0> but was:<1>

ozone-default.xml should be updated with the new configuration.

@anuengineer
Copy link
Contributor

+1, Thanks for the update. Looks good to me.

@anuengineer
Copy link
Contributor

sorry, is this failure related to this patch?
hdds.scm.chillmode.healthy.pipelie.pct expected:<0> but was:<1>
if so, can we please add this field to the ozone-default.xml before we commit this? thanks

@bharatviswa504
Copy link
Contributor Author

Thank You @anuengineer and @elek for the review.
I have fixed the test failure.

@anuengineer
Copy link
Contributor

👍 , feel free to commit once we get a Jenkins run. Thanks for taking care of this.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 23 Docker mode activated.
_ Prechecks _
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 3 new or modified test files.
_ trunk Compile Tests _
0 mvndep 22 Maven dependency ordering for branch
+1 mvninstall 968 trunk passed
+1 compile 949 trunk passed
+1 checkstyle 233 trunk passed
+1 mvnsite 157 trunk passed
+1 shadedclient 1062 branch has no errors when building and testing our client artifacts.
0 findbugs 0 Skipped patched modules with no Java source: hadoop-ozone/integration-test
+1 findbugs 113 trunk passed
+1 javadoc 90 trunk passed
_ Patch Compile Tests _
0 mvndep 21 Maven dependency ordering for patch
+1 mvninstall 101 the patch passed
+1 compile 884 the patch passed
+1 javac 884 the patch passed
+1 checkstyle 188 the patch passed
+1 mvnsite 123 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 xml 1 The patch has no ill-formed XML file.
+1 shadedclient 704 patch has no errors when building and testing our client artifacts.
0 findbugs 0 Skipped patched modules with no Java source: hadoop-ozone/integration-test
+1 findbugs 122 the patch passed
+1 javadoc 83 the patch passed
_ Other Tests _
-1 unit 69 common in the patch failed.
+1 unit 97 server-scm in the patch passed.
-1 unit 537 integration-test in the patch failed.
+1 asflicense 38 The patch does not generate ASF License warnings.
6442
Reason Tests
Failed junit tests hadoop.hdds.security.x509.certificate.client.TestDefaultCertificateClient
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-518/4/artifact/out/Dockerfile
GITHUB PR #518
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname Linux ae75be440e0a 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 9192f71
maven version: Apache Maven 3.3.9
Default Java 1.8.0_191
findbugs v3.1.0-RC1
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-518/4/artifact/out/patch-unit-hadoop-hdds_common.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-518/4/artifact/out/patch-unit-hadoop-ozone_integration-test.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-518/4/testReport/
Max. process+thread count 3556 (vs. ulimit of 5500)
modules C: hadoop-hdds/common hadoop-hdds/server-scm hadoop-ozone/integration-test U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-518/4/console
Powered by Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

@elek
Copy link
Member

elek commented Feb 27, 2019

Thanks @anuengineer the review and @bharatviswa504 the PR. I am pushing it to the trunk.

I checked the remaining unit tests and they are not related.

@elek elek closed this in 6c8c422 Feb 27, 2019
@bharatviswa504 bharatviswa504 deleted the HDDS-1178 branch February 27, 2019 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants