Skip to content

HDDS-1788. Fix kerberos principal error in Ozone Recon #1201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 3, 2019

Conversation

vivekratnavel
Copy link
Contributor

Recon fails to come up in a secure cluster with the following error:

Failed startup of context o.e.j.w.WebAppContext@2009f9b0{/,file:///tmp/jetty-0.0.0.0-9888-recon-_-any-2565178148822292652.dir/webapp/,UNAVAILABLE}{/recon} javax.servlet.ServletException: javax.servlet.ServletException: Principal not defined in configuration at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:188) at

This patch addresses this issue and enables Recon to come up in clusters secured by kerberos. I have manually tested the patch by creating the recon jar and replacing an old jar in a live secure CM deployed cluster and verified that Recon starts successfully and is able to login successfully with the kerberos ticket. Also updated ozonesecure docker-compose file to add recon and verified that recon is able to come up successfully. This patch also fixes various typos found in other parts of the source code not related to the title of this JIRA.

@vivekratnavel
Copy link
Contributor Author

/label ozone

@vivekratnavel
Copy link
Contributor Author

@swagle @avijayanhwx @anuengineer Please review when you find time. Thanks!

@elek elek added the ozone label Aug 1, 2019
Copy link
Contributor

@swagle swagle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 LGTM

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 98 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
0 shelldocs 0 Shelldocs was not available.
0 yamllint 0 yamllint was not available.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
0 mvndep 73 Maven dependency ordering for branch
+1 mvninstall 659 trunk passed
+1 compile 431 trunk passed
+1 checkstyle 81 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 968 branch has no errors when building and testing our client artifacts.
+1 javadoc 208 trunk passed
0 spotbugs 454 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 691 trunk passed
_ Patch Compile Tests _
0 mvndep 53 Maven dependency ordering for patch
+1 mvninstall 600 the patch passed
+1 compile 409 the patch passed
+1 javac 409 the patch passed
+1 checkstyle 94 the patch passed
+1 hadolint 3 There were no new hadolint issues.
+1 mvnsite 0 the patch passed
+1 shellcheck 0 There were no new shellcheck issues.
+1 whitespace 0 The patch has no whitespace issues.
+1 xml 2 The patch has no ill-formed XML file.
+1 shadedclient 745 patch has no errors when building and testing our client artifacts.
+1 javadoc 186 the patch passed
+1 findbugs 702 the patch passed
_ Other Tests _
-1 unit 357 hadoop-hdds in the patch failed.
-1 unit 1968 hadoop-ozone in the patch failed.
+1 asflicense 49 The patch does not generate ASF License warnings.
8657
Reason Tests
Failed junit tests hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis
hadoop.ozone.client.rpc.Test2WayCommitInRatis
hadoop.hdds.scm.pipeline.TestRatisPipelineCreateAndDestory
hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures
hadoop.ozone.client.rpc.TestMultiBlockWritesWithDnFailures
hadoop.ozone.TestContainerStateMachineIdempotency
hadoop.ozone.om.TestScmSafeMode
hadoop.ozone.client.rpc.TestSecureOzoneRpcClient
hadoop.ozone.client.rpc.TestOzoneRpcClient
hadoop.ozone.client.rpc.TestOzoneAtRestEncryption
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1201/1/artifact/out/Dockerfile
GITHUB PR #1201
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml hadolint shellcheck shelldocs yamllint findbugs checkstyle
uname Linux ccaf957e3cb3 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / b008072
Default Java 1.8.0_222
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1201/1/artifact/out/patch-unit-hadoop-hdds.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1201/1/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1201/1/testReport/
Max. process+thread count 4257 (vs. ulimit of 5500)
modules C: hadoop-hdds/common hadoop-ozone/dist hadoop-ozone/ozone-recon U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1201/1/console
versions git=2.7.4 maven=3.3.9 shellcheck=0.4.6 findbugs=3.1.0-RC1 hadolint=1.11.1-0-g0e692dd
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@anuengineer
Copy link
Contributor

anuengineer commented Aug 1, 2019

I am not sure I understand the patch well enough. @xiaoyuyao @arp7 @elek can you guys please take a look ?

Copy link
Contributor

@bharatviswa504 bharatviswa504 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think some time back we debugged this issue and see that configuration object in recon http server is not having correct configuration values. But one question I have is previously also there is a provider for OzoneConfiguration, and now we have changed it to Configuration. How this is solving that issue by changing to Configuration I have not got that part, care to explain.

@vivekratnavel
Copy link
Contributor Author

Hi @bharatviswa504,

OzoneConfigurationProvider is changed to ConfigurationProvider in this patch to have the correct Configuration object passed to the constructor of BaseHttpServer here - https://github.com/apache/hadoop/blob/trunk/hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/ReconHttpServer.java#L35

I understand that its not very clear because this class is not part of this PR. The issue was that injection was not working in ReconHttpServer - https://github.com/apache/hadoop/blob/trunk/hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/ReconHttpServer.java#L34 because of the use of Configuration object. Now, with refactoring OzoneConfigurationProvider to ConfigurationProvider and changing this line for bind - https://github.com/apache/hadoop/pull/1201/files#diff-4a219a4368f56aa118b6ba4afb981244R57 fixes the injection in ReconHttpServer and thus Recon web server is able to get access to the correct principal name and authenticate with kerberos ticket.

Please let me know if you have any more questions.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 41 Docker mode activated.
_ Prechecks _
+1 dupname 1 No case conflicting files found.
0 shelldocs 0 Shelldocs was not available.
0 yamllint 0 yamllint was not available.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
0 mvndep 80 Maven dependency ordering for branch
+1 mvninstall 613 trunk passed
+1 compile 399 trunk passed
+1 checkstyle 78 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 849 branch has no errors when building and testing our client artifacts.
+1 javadoc 172 trunk passed
0 spotbugs 467 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 676 trunk passed
_ Patch Compile Tests _
0 mvndep 40 Maven dependency ordering for patch
+1 mvninstall 567 the patch passed
+1 compile 394 the patch passed
+1 javac 394 the patch passed
+1 checkstyle 87 the patch passed
+1 hadolint 4 There were no new hadolint issues.
+1 mvnsite 0 the patch passed
+1 shellcheck 0 There were no new shellcheck issues.
+1 whitespace 0 The patch has no whitespace issues.
+1 xml 1 The patch has no ill-formed XML file.
+1 shadedclient 742 patch has no errors when building and testing our client artifacts.
+1 javadoc 176 the patch passed
+1 findbugs 687 the patch passed
_ Other Tests _
+1 unit 340 hadoop-hdds in the patch passed.
-1 unit 212 hadoop-ozone in the patch failed.
+1 asflicense 45 The patch does not generate ASF License warnings.
6456
Reason Tests
Failed junit tests hadoop.ozone.om.ratis.TestOzoneManagerDoubleBufferWithDummyResponse
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1201/2/artifact/out/Dockerfile
GITHUB PR #1201
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml hadolint shellcheck shelldocs yamllint findbugs checkstyle
uname Linux 2b59c8fcad27 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / f86de6f
Default Java 1.8.0_212
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1201/2/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1201/2/testReport/
Max. process+thread count 984 (vs. ulimit of 5500)
modules C: hadoop-hdds/common hadoop-ozone/dist hadoop-ozone/ozone-recon U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1201/2/console
versions git=2.7.4 maven=3.3.9 shellcheck=0.4.6 findbugs=3.1.0-RC1 hadolint=1.11.1-0-g0e692dd
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@vivekratnavel
Copy link
Contributor Author

The unit test, integration test and checkstyle failures are not related to this patch.

@bharatviswa504
Copy link
Contributor

bharatviswa504 commented Aug 2, 2019

Thank You @vivekratnavel for the fix and the explanation.
+1 LGTM. Will wait till tomorrow for others to review, If no more comments, will commit it.

I think, we need to fix the title for this jira, as this is not adding Kerberos support to Ozone. It is fixing ConfigurationProvider, so that injection works. And fixing this will fix ReconServer start. As this will fix passing correct configuration to ReconHttpServer. So, this means it fix Kerberos/Non-Kerberos and picking correct configuration in ReconHttpServer.

Can you change the title accordingly, and update the Jira. It will be helpful for future reference.

@vivekratnavel vivekratnavel changed the title HDDS-1788. Add kerberos support to Ozone Recon HDDS-1788. Fix kerberos principal error in Ozone Recon Aug 2, 2019
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 139 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
0 shelldocs 1 Shelldocs was not available.
0 yamllint 1 yamllint was not available.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
0 mvndep 40 Maven dependency ordering for branch
+1 mvninstall 762 trunk passed
+1 compile 458 trunk passed
+1 checkstyle 101 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 894 branch has no errors when building and testing our client artifacts.
+1 javadoc 175 trunk passed
0 spotbugs 446 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 651 trunk passed
_ Patch Compile Tests _
0 mvndep 39 Maven dependency ordering for patch
+1 mvninstall 565 the patch passed
+1 compile 392 the patch passed
+1 javac 392 the patch passed
+1 checkstyle 87 the patch passed
+1 hadolint 3 There were no new hadolint issues.
+1 mvnsite 0 the patch passed
+1 shellcheck 0 There were no new shellcheck issues.
+1 whitespace 0 The patch has no whitespace issues.
+1 xml 1 The patch has no ill-formed XML file.
+1 shadedclient 748 patch has no errors when building and testing our client artifacts.
+1 javadoc 174 the patch passed
+1 findbugs 657 the patch passed
_ Other Tests _
+1 unit 339 hadoop-hdds in the patch passed.
-1 unit 2183 hadoop-ozone in the patch failed.
+1 asflicense 51 The patch does not generate ASF License warnings.
8740
Reason Tests
Failed junit tests hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis
hadoop.hdds.scm.pipeline.TestPipelineClose
hadoop.hdds.scm.pipeline.TestRatisPipelineCreateAndDestory
hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures
hadoop.ozone.client.rpc.TestMultiBlockWritesWithDnFailures
hadoop.ozone.om.TestScmSafeMode
hadoop.ozone.client.rpc.TestSecureOzoneRpcClient
hadoop.ozone.client.rpc.TestOzoneRpcClient
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1201/3/artifact/out/Dockerfile
GITHUB PR #1201
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml hadolint shellcheck shelldocs yamllint findbugs checkstyle
uname Linux 3f8e6986510a 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / b01efe5
Default Java 1.8.0_222
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1201/3/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1201/3/testReport/
Max. process+thread count 5413 (vs. ulimit of 5500)
modules C: hadoop-hdds/common hadoop-ozone/dist hadoop-ozone/ozone-recon U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1201/3/console
versions git=2.7.4 maven=3.3.9 shellcheck=0.4.6 findbugs=3.1.0-RC1 hadolint=1.11.1-0-g0e692dd
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@bharatviswa504 bharatviswa504 merged commit ec1d453 into apache:trunk Aug 3, 2019
@bharatviswa504
Copy link
Contributor

Thank You @vivekratnavel for the contribution.
I have committed this to the trunk.

amahussein pushed a commit to amahussein/hadoop that referenced this pull request Oct 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants