Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/openshift cli and summon tls error #144

Merged
merged 1 commit into from
Sep 1, 2021

Conversation

codihuston
Copy link

Hello, this PR resolves the following:

Issues with Openshift (Next)

  • ImagePullBackOff error when pulling secretless-broker. This was resolved by explicitly specifying the full docker domain name. See 7_app_deploy.sh
  • Specify OCP CLI version 4.8.2 for the Openshift (Next) environment. This resolves some exceptions thrown when describing pods, and using the deprecated secrets api to generate and use image pull secrets
  • The new secrets API is conditionally used based on the OCP CLI version, in efforts to preserve any older functionality from older versions of the CLI
  • The MySQL image for OCP dropped support for an earlier version of TLS. Simply updating the MySQL image resolved this issue with no code changes. Thanks Kumbi!

References regarding MySQL TLS support:

All tests are green now =) (previously OpenShift (Next) and the MySQL tests were failing across all of OpenShift).

@codihuston
Copy link
Author

I have disabled the OCP_ NEXT param in the jenkins file. That will be handled by this repo: https://github.com/conjurinc/openshift-next-suite-test

See the results of my full run of the test suite: https://jenkins.conjur.net/blue/organizations/jenkins/conjurdemos--kubernetes-conjur-demo/detail/fix%2Fopenshift-cli-and-summon-tls-error/1/pipeline/

@codihuston codihuston requested a review from rpothier August 19, 2021 18:16
Copy link
Contributor

@doodlesbykumbi doodlesbykumbi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR looking good. Left some comments and suggestions.

I understand that the earlier commits were experimental. As the PR is nearing approval I think it's worth revisiting the commit history and potentially reducing the commits into logical wholes. Happy to chat git rebase -i ...

@codihuston codihuston force-pushed the fix/openshift-cli-and-summon-tls-error branch 3 times, most recently from 91df3e0 to 925768a Compare August 19, 2021 22:21
@codihuston
Copy link
Author

codihuston commented Aug 19, 2021

@rpothier Sorry for the superfluous review request, Kumbi is on it--I didn't see if my original request had gone through.

Hey @doodlesbykumbi, I went ahead and cleaned up the commits a bit. Please let me know if that is adequate. Tests are running now. Thanks!

@codihuston codihuston force-pushed the fix/openshift-cli-and-summon-tls-error branch from 925768a to c56170e Compare August 20, 2021 15:49
@codihuston
Copy link
Author

I went ahead and re-confirmed that tests against openshift (next) were still green. Disabling those tests now via TEST_OCP_NEXT since it seems that no projects are actively testing against this cluster by default.

See: https://jenkins.conjur.net/blue/organizations/jenkins/conjurdemos--kubernetes-conjur-demo/detail/fix%2Fopenshift-cli-and-summon-tls-error/9/pipeline

The test currently running now will not test against this cluster.

Copy link
Contributor

@doodlesbykumbi doodlesbykumbi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@codihuston Thanks for addressing those earlier comments. Left 2 comments. The PR is currently at 5 commits, I think we could potentially reduce it all to a single (if not 2) logical commit/s.

@codihuston codihuston force-pushed the fix/openshift-cli-and-summon-tls-error branch from c56170e to d03483d Compare August 20, 2021 16:06
@codihuston
Copy link
Author

Added a comment on the mysql template, see d03483d. Ready for re-review =).

Copy link
Contributor

@doodlesbykumbi doodlesbykumbi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more comments

@codihuston codihuston force-pushed the fix/openshift-cli-and-summon-tls-error branch 2 times, most recently from 1abf9ae to 8ac2ebd Compare August 20, 2021 18:07
@codihuston
Copy link
Author

Comments revisited and resolved. One of the test stages is failing due to a rate-limit from docker for this image: centos/mysql-80-centos7. I can restart this later this afternoon to see if this kicks off. Until then please review once more. =)

@codihuston codihuston force-pushed the fix/openshift-cli-and-summon-tls-error branch 2 times, most recently from 0af1f1a to d7eece8 Compare August 24, 2021 19:25
@codihuston
Copy link
Author

@doodlesbykumbi Hi Kumbi. As we discussed, I've added some logic to build out a custom mysql image based on centos/mysql-80-centos7. In the pipeline, it is built and stored in the openshift image registry only when running against openshift clusters.

I have also reduced everything into one commit.

Looks like all of the tests have green lights. While these tests include the openshift (next) cluster, jenkins should not include that cluster in future instances. See Jenkinsfile.

Copy link
Contributor

@doodlesbykumbi doodlesbykumbi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @codihuston. This is looking great. We just need to make sure the Secretless backend doesn't also hit the Docker pull limits. Otherwise, everything looks good.

@codihuston codihuston force-pushed the fix/openshift-cli-and-summon-tls-error branch from d7eece8 to 2520300 Compare August 30, 2021 15:54
Testing in openshift (next) led to discovery of a change in TLS version support between RHEL, mysql, and Connector/J (spring boot in test apps). We are using a newer mysql image for the test apps so that there is no discrepancy between TLS versions for direct mysql connections.

- for openshift only, the public centos/mysql-80-centos7 image is rebuilt, retagged, and pushed to our openshift registry in the ci pipeline
- bump test summon apps to centos/mysql-80-centos7 for direct connections because the latest version of the test app. This is because to carry on supporting `centos/mysql-57-centos7` it would require special changes to the DB_URL secret stored in Conjur only for MySQL (which we're opting against due to complexity), in order to avoid the TLS version mismatch issues.
- still use centos/mysql-57-centos7 for connections through secretless because it doesn't have the TLS version mismatch issue, but has an issue with centos/mysql-80-centos7 due to lack of support for new authentication mechanism from MySQL 8.0.
- secretless image path pointed to docker.io explicitly; openshift is unable to pull without this

OpenShift CLI issues resolved:

- openshift cli 3.* threw errors when testing in the oc (next) cluster. This has been resolved by updating the cli to 4.8.2 when testing against this cluster

    > Note: the openshift cli v4.8.2 does not support old secrets API any longer
@codihuston codihuston force-pushed the fix/openshift-cli-and-summon-tls-error branch from 2520300 to 677ceaa Compare August 30, 2021 19:33
@codihuston
Copy link
Author

Mysql v5.7 is tagged and pushed to our openshift registry in attempt to circumvent docker hub rate-limits as well. See this pipeline. Note that I just re-ran the pipeline while enabling the TEST_OCP_NEXT flag.

Please verify my approach to injecting images into the k8s/openshift templates here: injection of the templated values, particularly for k8s.

Thanks!

@doodlesbykumbi doodlesbykumbi merged commit 815d327 into main Sep 1, 2021
@doodlesbykumbi doodlesbykumbi deleted the fix/openshift-cli-and-summon-tls-error branch September 1, 2021 11:26
@doodlesbykumbi
Copy link
Contributor

Thanks @codihuston. Approved and merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants