Addons: add OLM support

[blues-man]

Adding OLM support as addon, fixes #6419

Using latest version 0.14.1 as per latest yamls available, I'm wondering if there is any way to avoid hardcoding versions in addons in cases where versions are in the code like in olm.yaml

Simulating OLM installation script

Example of usage:

$ minikube addons enable olm

Expected output:

🌟  The 'olm' addon is enabled

Install your favourite Operator, you can pick one from OperatorHub and follow Operator documentation to install it, skipping the OLM installation since you did it already from Minikube una tantum for all new operators you would like to install.

Example for etcd:

$ kubectl create -f https://operatorhub.io/install/etcd.yaml
$ kubectl get csv -n my-etcd

By default the olm addon is disabled:

./out/minikube addons list
|-----------------------------|----------|--------------|
|         ADDON NAME          | PROFILE  |    STATUS    |
|-----------------------------|----------|--------------|
| dashboard                   | minikube | disabled     |
| default-storageclass        | minikube | enabled ✅   |
| efk                         | minikube | disabled     |
| freshpod                    | minikube | disabled     |
| gvisor                      | minikube | disabled     |
| helm-tiller                 | minikube | disabled     |
| ingress                     | minikube | disabled     |
| ingress-dns                 | minikube | disabled     |
| istio                       | minikube | disabled     |
| istio-provisioner           | minikube | disabled     |
| logviewer                   | minikube | disabled     |
| metallb                     | minikube | disabled     |
| metrics-server              | minikube | disabled     |
| nvidia-driver-installer     | minikube | disabled     |
| nvidia-gpu-device-plugin    | minikube | disabled     |
| olm                         | minikube | disabled     |
| registry                    | minikube | disabled     |
| registry-aliases            | minikube | disabled     |
| registry-creds              | minikube | disabled     |
| storage-provisioner         | minikube | enabled ✅   |
| storage-provisioner-gluster | minikube | disabled     |
|-----------------------------|----------|--------------|

[k8s-ci-robot]

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

---

• If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
• If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
• If you have done the above and are still having issues with the CLA being reported as unsigned, please log a ticket with the Linux Foundation Helpdesk: https://support.linuxfoundation.org/
• Should you encounter any issues with the Linux Foundation Helpdesk, send a message to the backup e-mail support address at: login-issues@jira.linuxfoundation.org

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[k8s-ci-robot]

Welcome @blues-man!

It looks like this is your first PR to kubernetes/minikube 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/minikube has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @blues-man. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: blues-man
To complete the pull request process, please assign afbjorklund
You can assign the PR to them by writing /assign @afbjorklund in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[minikube-bot]

Can one of the admins verify this patch?

[blues-man]

/check-cla

[medyagh]

@blues-man thank you for this contribution.
do you mind sharing an example of using this addon in the PR description ?

and also make sure by default this addon is not enabled.
you can paste the output of minikube addon list in the PR description to verify

[medyagh]

this seems like a good addon to add. please resolve the comments

[blues-man]

Hi @medyagh thanks for review, I updated the comment on the PR description. I also believe it could be a good addon, because it could also optimize the number of addon request, if they are present on OperatorHub.io they can be installed directly from this addon.

[priyawadhwa]

Hey @blues-man thanks for submitting this PR. I think this change is big enough to warrant an integration test. Would you mind writing one? The test would:

1. Start a minikube cluster, and enable the olm addon by adding the --addons=olm flag (Sample code here)
2. Install some simple operator, ideally something that is pinned so that it doesn't change
3. Makes sure the installation was successful (as in, some expected pod came up as Running) (sample code here)

You can add TestOlmAddon to our addons_test.go file.

A lot of the code from TestAddons can be copied into TestOlmAddon (I linked some key lines above).

Docs for quickly running an integration test locally can be found here. Please let me know if you have any questions!

[blues-man]

Hi @priyawadhwa, right, it makes sense to add those, thanks for the documentation!
I'll add one and add to the PR.

[priyawadhwa]

/ok-to-test

[blues-man]

Hello @priyawadhwa , I've added a TestOlmAddon using a validateOlmAddon function.

I'm checkin that all deployments and pods are up and running into olm namespace after enabling the addon, and then I'm installing latest etcd operator checking that installation is successful from kubectl get csv -m my-etcd output.

Let me know what do you think about, this is output I got from test I run:

$ make integration -e TEST_ARGS="-test.run TestOlmAddon/parallel/Olm --profile=minikube --cleanup=true"
go test -v -test.timeout=60m ./test/integration --tags="integration container_image_ostree_stub containers_image_openpgp go_getter_nos3 go_getter_nogcs" -test.run TestOlmAddon/parallel/Olm --profile=minikube --cleanup=true
=== RUN   TestOlmAddon
=== RUN   TestOlmAddon/parallel
=== RUN   TestOlmAddon/parallel/Olm
=== PAUSE TestOlmAddon/parallel/Olm
=== CONT  TestOlmAddon/parallel/Olm
--- PASS: TestOlmAddon (100.59s)
    addons_test.go:338: (dbg) Run:  out/minikube start -p minikube --wait=false --memory=2600 --alsologtostderr --addons=olm 
    addons_test.go:338: (dbg) Done: out/minikube start -p minikube --wait=false --memory=2600 --alsologtostderr --addons=olm : (45.225074859s)
    --- PASS: TestOlmAddon/parallel (0.00s)
        --- PASS: TestOlmAddon/parallel/Olm (51.61s)
            addons_test.go:387: catalog-operator stabilized in 1.705376713s
            addons_test.go:391: olm-operator stabilized in 1.712894262s
            addons_test.go:395: packageserver stabilized in 12.427525518s
            addons_test.go:397: (dbg) TestOlmAddon/parallel/Olm: waiting 6m0s for pods matching "app=catalog-operator" in namespace "olm" ...
            helpers.go:314: "catalog-operator-86f777cc59-sn8t4" [dd3ac2e3-1980-4d38-819c-d966d08637ff] Running / Ready:ContainersNotReady (containers with unready status: [catalog-operator]) / ContainersReady:ContainersNotReady (containers with unready status: [catalog-operator])
            helpers.go:314: "catalog-operator-86f777cc59-sn8t4" [dd3ac2e3-1980-4d38-819c-d966d08637ff] Running
            addons_test.go:397: (dbg) TestOlmAddon/parallel/Olm: app=catalog-operator healthy within 5.087491495s
            addons_test.go:400: (dbg) TestOlmAddon/parallel/Olm: waiting 6m0s for pods matching "app=olm-operator" in namespace "olm" ...
            helpers.go:314: "olm-operator-5fd48d8cd4-krbwn" [7fd8f7fa-11b7-4563-96c9-25acb993fabb] Running
            addons_test.go:400: (dbg) TestOlmAddon/parallel/Olm: app=olm-operator healthy within 5.007438185s
            addons_test.go:403: (dbg) TestOlmAddon/parallel/Olm: waiting 6m0s for pods matching "app=packageserver" in namespace "olm" ...
            helpers.go:314: "packageserver-7d5b9db999-dhkln" [137a5986-4ad4-4bda-8551-63082f93f71f] Running / Ready:ContainersNotReady (containers with unready status: [packageserver]) / ContainersReady:ContainersNotReady (containers with unready status: [packageserver])
            helpers.go:314: "packageserver-7d5b9db999-lx5qd" [c3a4988b-40bb-46ef-b26a-ef602de8406b] Running
            helpers.go:314: "packageserver-7d5b9db999-dhkln" [137a5986-4ad4-4bda-8551-63082f93f71f] Running / Ready:ContainersNotReady (containers with unready status: [packageserver]) / ContainersReady:ContainersNotReady (containers with unready status: [packageserver])
            helpers.go:314: "packageserver-7d5b9db999-lx5qd" [c3a4988b-40bb-46ef-b26a-ef602de8406b] Running
            helpers.go:314: "packageserver-7d5b9db999-dhkln" [137a5986-4ad4-4bda-8551-63082f93f71f] Running
            helpers.go:314: "packageserver-7d5b9db999-lx5qd" [c3a4988b-40bb-46ef-b26a-ef602de8406b] Running
            helpers.go:314: "packageserver-7d5b9db999-dhkln" [137a5986-4ad4-4bda-8551-63082f93f71f] Running
            helpers.go:314: "packageserver-7d5b9db999-lx5qd" [c3a4988b-40bb-46ef-b26a-ef602de8406b] Running
            helpers.go:314: "packageserver-7d5b9db999-dhkln" [137a5986-4ad4-4bda-8551-63082f93f71f] Running
            helpers.go:314: "packageserver-7d5b9db999-lx5qd" [c3a4988b-40bb-46ef-b26a-ef602de8406b] Running
            helpers.go:314: "packageserver-7d5b9db999-dhkln" [137a5986-4ad4-4bda-8551-63082f93f71f] Running
            addons_test.go:403: (dbg) TestOlmAddon/parallel/Olm: app=packageserver healthy within 5.007613888s
            addons_test.go:406: (dbg) TestOlmAddon/parallel/Olm: waiting 6m0s for pods matching "olm.catalogSource=operatorhubio-catalog" in namespace "olm" ...
            helpers.go:314: "operatorhubio-catalog-w2mhl" [d7889a22-0ea9-47f1-9e00-7280da75b6c2] Running
            addons_test.go:406: (dbg) TestOlmAddon/parallel/Olm: olm.catalogSource=operatorhubio-catalog healthy within 5.010433197s
            addons_test.go:411: (dbg) Run:  kubectl --context minikube create -f https://operatorhub.io/install/etcd.yaml
            addons_test.go:411: (dbg) Done: kubectl --context minikube create -f https://operatorhub.io/install/etcd.yaml: (1.128865055s)
            addons_test.go:418: (dbg) Run:  kubectl --context minikube get csv -n my-etcd
            addons_test.go:423: kubectl --context minikube get csv -n my-etcd: unexpected stderr: No resources found in my-etcd namespace.
            addons_test.go:418: (dbg) Run:  kubectl --context minikube get csv -n my-etcd
            addons_test.go:418: (dbg) Run:  kubectl --context minikube get csv -n my-etcd
            addons_test.go:418: (dbg) Run:  kubectl --context minikube get csv -n my-etcd
    addons_test.go:361: (dbg) Run:  out/minikube stop -p minikube
    addons_test.go:361: (dbg) Done: out/minikube stop -p minikube: (3.174289013s)
    addons_test.go:365: (dbg) Run:  out/minikube addons enable dashboard -p minikube
    addons_test.go:369: (dbg) Run:  out/minikube addons disable dashboard -p minikube
    helpers.go:170: (dbg) Run:  out/minikube delete -p minikube
PASS
ok      k8s.io/minikube/test/integration        100.604s

[minikube-pr-bot]

kvm2 Driver
docker Driver

[blues-man]

Is there any way to define an order or a custom sequence when disabling an addon?

When I try to disable it I got error because kubectl delete loops in the dir deleting CRDs first and then CR, so the olm is uninstalled but minikube reports it as enabled.

[codecov-commenter]

Codecov Report

Merging #8129 into master will decrease coverage by 0.76%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #8129      +/-   ##
==========================================
- Coverage   35.27%   34.50%   -0.77%     
==========================================
  Files         146      147       +1     
  Lines        9326     9412      +86     
==========================================
- Hits         3290     3248      -42     
- Misses       5637     5765     +128     
  Partials      399      399              

Impacted Files	Coverage Δ
pkg/minikube/bootstrapper/certs.go	44.26% <0.00%> (-27.05%)	⬇️
pkg/minikube/proxy/proxy.go	77.10% <0.00%> (-19.87%)	⬇️
cmd/minikube/cmd/node_delete.go	13.33% <0.00%> (-4.85%)	⬇️
cmd/minikube/cmd/start.go	13.73% <0.00%> (-1.67%)	⬇️
cmd/minikube/cmd/node_stop.go	13.33% <0.00%> (-0.96%)	⬇️
cmd/minikube/cmd/node_start.go	9.37% <0.00%> (-0.63%)	⬇️
cmd/minikube/cmd/start_flags.go	50.80% <0.00%> (-0.17%)	⬇️
cmd/minikube/cmd/delete.go	21.45% <0.00%> (-0.16%)	⬇️
pkg/minikube/perf/binary.go	78.94% <0.00%> (ø)
cmd/minikube/cmd/config/config.go	10.41% <0.00%> (ø)
... and 3 more

[minikube-pr-bot]

kvm2 Driver
Times for minikube: [64.92485875899999 64.702864945 65.122621748]
Average time for minikube: 64.91678181733333

Times for Minikube (PR 8129): [66.657461135 67.29643341100001 63.399033846]
Average time for Minikube (PR 8129): 65.784309464

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 8129) |
+--------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian   |  0.059889 |           0.058716 |
|                           9.11 |           |                    |
| * Using the kvm2 driver based  |  0.019799 |           0.018767 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.003043 |           0.003011 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 41.110987 |          41.255857 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.2 | 21.695693 |          22.231546 |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  1.376612 |           1.370654 |
| components...                  |           |                    |
| * Enabled addons:              |  0.566835 |           0.757649 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.080830 |           0.082317 |
| configured to use "minikube"   |           |                    |
|                                |  0.003093 |           0.005792 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [27.180954368000002 26.522926222 25.412623299999996]
Average time for minikube: 26.37216796333333

Times for Minikube (PR 8129): [27.900685975000002 25.515968625 25.970571450999998]
Average time for Minikube (PR 8129): 26.462408683666666

Averages Time Per Log

+----------------------------------------+-----------+--------------------+
|                  LOG                   | MINIKUBE  | MINIKUBE (PR 8129) |
+----------------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian           |  0.073014 |           0.071890 |
|                                   9.11 |           |                    |
| * Using the docker driver              |  0.002531 |           0.002413 |
| based on existing profile              |           |                    |
| * Starting control plane node          |  0.057844 |           0.057874 |
| minikube in cluster minikube           |           |                    |
| * Creating docker container            |  7.620281 |           7.433590 |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Preparing Kubernetes v1.18.2         |  0.115457 |           0.119494 |
| on Docker 19.03.2 ...                  |           |                    |
|   -                                    | 16.983011 |          17.593701 |
| kubeadm.pod-network-cidr=10.244.0.0/16 |           |                    |
| * Verifying Kubernetes                 |  1.440367 |           0.940156 |
| components...                          |           |                    |
| * Enabled addons:                      |  0.002557 |           0.173942 |
| default-storageclass,                  |           |                    |
| storage-provisioner                    |           |                    |
| * Done! kubectl is now                 |  0.073939 |           0.063827 |
| configured to use "minikube"           |           |                    |
|                                        |  0.003169 |           0.005522 |
+----------------------------------------+-----------+--------------------+

[medyagh]

thank you for adding this addon and adding integration tests for it