Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix nginx-ingress-controller on k8s 1.22 #94

Merged
merged 3 commits into from
Aug 16, 2021
Merged

Fix nginx-ingress-controller on k8s 1.22 #94

merged 3 commits into from
Aug 16, 2021

Conversation

Cynerva
Copy link
Contributor

@Cynerva Cynerva commented Aug 13, 2021 •
edited
Loading

This fixes two issues with nginx-ingress-controller.

RBAC v1beta1 API no longer works

nginx-ingress-controller pods were in a crashloop:

$ kubectl get po -n ingress-nginx-kubernetes-worker
NAME                                               READY   STATUS             RESTARTS         AGE
nginx-ingress-controller-kubernetes-worker-2dngn   0/1     CrashLoopBackOff   29 (4m58s ago)   88m
nginx-ingress-controller-kubernetes-worker-527nf   0/1     CrashLoopBackOff   31 (4m50s ago)   89m
nginx-ingress-controller-kubernetes-worker-fzgdm   0/1     CrashLoopBackOff   30 (3s ago)      87m

With a permissions issue:

$ kubectl logs -n ingress-nginx-kubernetes-worker nginx-ingress-controller-kubernetes-worker-2dngn
F0813 18:23:47.585675       6 main.go:143] Unexpected error obtaining ingress-nginx pod: unable to get POD information: pods "nginx-ingress-controller-kubernetes-worker-2dngn" is forbidden: User "system:serviceaccount:ingress-nginx-kubernetes-worker:nginx-ingress-serviceaccount-kubernetes-worker" cannot get resource "pods" in API group "" in the namespace "ingress-nginx-kubernetes-worker"

Caused by a failure to apply RBAC rules:

unit-kubernetes-worker-0: 13:33:11 WARNING unit.kubernetes-worker/0.update-status unable to recognize "/root/cdk/addons/ingress-daemon-set.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unit-kubernetes-worker-0: 13:33:11 WARNING unit.kubernetes-worker/0.update-status unable to recognize "/root/cdk/addons/ingress-daemon-set.yaml": no matches for kind "Role" in version "rbac.authorization.k8s.io/v1beta1"
unit-kubernetes-worker-0: 13:33:11 WARNING unit.kubernetes-worker/0.update-status unable to recognize "/root/cdk/addons/ingress-daemon-set.yaml": no matches for kind "RoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unit-kubernetes-worker-0: 13:33:11 WARNING unit.kubernetes-worker/0.update-status unable to recognize "/root/cdk/addons/ingress-daemon-set.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"

The fix is to update RBAC apiVersions to rbac.authorization.k8s.io/v1.

ingress v1beta1 API is deprecated, nginx-ingress-controller v0.45.0 still uses it

I don't have logs handy, but nginx-ingress-controller v0.45.0 fails to watch ingress resources via the v1beta1 API, which no longer exists in k8s 1.22. It's the same issue seen here.

I also tried nginx-ingress-controller v0.48.1 and it has the same issue.

The fix is to upgrade to nginx-ingress-controller v1.0.0-beta.1, which uses the ingress v1 API instead. There is no stable release that supports ingress v1.

@kwmonroe kwmonroe mentioned this pull request Aug 16, 2021
Merged
@Cynerva
Copy link
Contributor Author

Cynerva commented Aug 16, 2021

Quick test with microbot looks good:

$ kubectl get po -n ingress-nginx-kubernetes-worker
NAME                                                     READY   STATUS    RESTARTS   AGE
default-http-backend-kubernetes-worker-cd9b77777-dbdsj   1/1     Running   0          56m
nginx-ingress-controller-kubernetes-worker-4x85j         1/1     Running   0          55m
nginx-ingress-controller-kubernetes-worker-gbfzd         1/1     Running   0          56m
nginx-ingress-controller-kubernetes-worker-rz9f2         1/1     Running   0          55m

$ kubectl get po -n ingress-nginx-kubernetes-worker nginx-ingress-controller-kubernetes-worker-4x85j -o jsonpath='{.spec.containers[0].image}'
rocks.canonical.com:443/cdk/k8s-artifacts-prod/ingress-nginx/controller:v1.0.0-beta.3

$ kubectl get ingress
NAME               CLASS                      HOSTS                            ADDRESS                                        PORTS   AGE
microbot-ingress   nginx-ingress-controller   microbot.10.246.154.138.nip.io   10.246.154.121,10.246.154.138,10.246.154.246   80      7m48s

$ curl microbot.10.246.154.138.nip.io
<!DOCTYPE html>
<html>
  <style type="text/css">
    .centered
      {
      text-align:center;
      margin-top:0px;
      margin-bottom:0px;
      padding:0px;
      }
  </style>
  <body>
    <p class="centered"><img src="microbot.png" alt="microbot"/></p>
    <p class="centered">Container hostname: microbot-75cfc77b98-99nhc</p>
  </body>
</html>

kwmonroe
kwmonroe approved these changes Aug 16, 2021
View reviewed changes
Copy link
Contributor

@kwmonroe kwmonroe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick turnaround to beta.3 and re-test. LGTM.

@kwmonroe kwmonroe merged commit fd7367d into master Aug 16, 2021
@kwmonroe kwmonroe deleted the gkk/1.22-ingress branch August 16, 2021 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kwmonroe kwmonroe kwmonroe approved these changes

Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Cynerva @kwmonroe