k8s.io/client-go@v0.20.2/tools/cache/reflector.go:167: Failed to watch v1beta1.Ingress: failed to list v1beta1.Ingress #7448

sysnet4admin · 2021-08-06T07:38:39Z

NGINX Ingress controller version:

NGINX Ingress controller
  Release:       v0.47.0
  Build:         7201e37633485d1f14dbe9cd7b22dd380df00a07
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.20.1

Kubernetes version (use kubectl version):

[root@m-k8s ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.0", GitCommit:"c2b5237ccd9c0f1d600d3072634ca66cefdf272f", GitTreeState:"clean", BuildDate:"2021-08-04T18:03:20Z", GoVersion:"go1.16.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.0", GitCommit:"c2b5237ccd9c0f1d600d3072634ca66cefdf272f", GitTreeState:"clean", BuildDate:"2021-08-04T17:57:25Z", GoVersion:"go1.16.6", Compiler:"gc", Platform:"linux/amd64"}

Environment:

Cloud provider or hardware configuration:
- BareMetal(VM Enviroment)
OS (e.g. from /etc/os-release):

[root@m-k8s ~]# cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

Kernel (e.g. uname -a):

[root@m-k8s ~]# uname -a
Linux m-k8s 3.10.0-1127.19.1.el7.x86_64 #1 SMP Tue Aug 25 17:23:54 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Install tools:
- Not special
Others:
- None

What happened:
I have encountered reflector still point out v1beta1.ingress even I follow compatible procedure for newest 1.22.

Support Versions table

Ingress-nginx version	k8s supported version	Alpine Version	Nginx Version
v1.0.0-alpha.2 <<	1.22, 1.21, 1.20, 1.19	3.13.5	1.20.1
v1.0.0-alpha.1	1.21, 1.20, 1.19	3.13.5	1.20.1
v0.48.1	1.21, 1.20, 1.19	3.13.5	1.20.1
v0.47.0	1.21, 1.20, 1.19	3.13.5	1.20.1
v0.46.0	1.21, 1.20, 1.19	3.13.2	1.19.6

[root@m-k8s ~]# k get po -n ingress-nginx
NAME                                        READY   STATUS              RESTARTS   AGE
ingress-nginx-admission-create--1-zh5fn     0/1     ContainerCreating   0          4s
ingress-nginx-admission-patch--1-zdftk      0/1     Error               0          4s
ingress-nginx-controller-86db694fd4-kd22m   0/1     ContainerCreating   0          4s
[root@m-k8s ~]# k logs -n ingress-nginx ingress-nginx-controller-86db694fd4-kd22m
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v0.47.0
  Build:         7201e37633485d1f14dbe9cd7b22dd380df00a07
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.20.1

-------------------------------------------------------------------------------

I0806 07:23:28.379942       6 flags.go:208] "Watching for Ingress" class="nginx"
W0806 07:23:28.380175       6 flags.go:213] Ingresses with an empty class will also be processed by this Ingress controller
W0806 07:23:28.380901       6 client_config.go:614] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0806 07:23:28.381299       6 main.go:241] "Creating API client" host="https://10.96.0.1:443"
I0806 07:23:28.401118       6 main.go:285] "Running in Kubernetes cluster" major="1" minor="22" git="v1.22.0" state="clean" commit="c2b5237ccd9c0f1d600d3072634ca66cefdf272f" platform="linux/amd64"
I0806 07:23:28.937819       6 main.go:105] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
I0806 07:23:28.945494       6 main.go:115] "Enabling new Ingress features available since Kubernetes v1.18"
W0806 07:23:28.949544       6 main.go:127] No IngressClass resource with name nginx found. Only annotation will be used.
I0806 07:23:29.046851       6 ssl.go:532] "loading tls certificate" path="/usr/local/certificates/cert" key="/usr/local/certificates/key"
I0806 07:23:29.089552       6 nginx.go:254] "Starting NGINX Ingress controller"
I0806 07:23:29.130956       6 event.go:282] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"dcaac7a0-a91f-4aa8-9a31-f5f90ccfc6d8", APIVersion:"v1", ResourceVersion:"74715", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
E0806 07:23:30.227849       6 reflector.go:138] k8s.io/client-go@v0.20.2/tools/cache/reflector.go:167: Failed to watch *v1beta1.Ingress: failed to list *v1beta1.Ingress: the server could not find the requested resource
E0806 07:23:31.581614       6 reflector.go:138] k8s.io/client-go@v0.20.2/tools/cache/reflector.go:167: Failed to watch *v1beta1.Ingress: failed to list *v1beta1.Ingress: the server could not find the requested resource
E0806 07:23:33.264973       6 reflector.go:138] k8s.io/client-go@v0.20.2/tools/cache/reflector.go:167: Failed to watch *v1beta1.Ingress: failed to list *v1beta1.Ingress: the server could not find the requested resource
E0806 07:23:38.431993       6 reflector.go:138] k8s.io/client-go@v0.20.2/tools/cache/reflector.go:167: Failed to watch *v1beta1.Ingress: failed to list *v1beta1.Ingress: the server could not find the requested resource
E0806 07:23:45.072547       6 reflector.go:138] k8s.io/client-go@v0.20.2/tools/cache/reflector.go:167: Failed to watch *v1beta1.Ingress: failed to list *v1beta1.Ingress: the server could not find the requested resource

What you expected to happen:
properly deployed without error :)

How to reproduce it:

Download this bundle pack
https://github.com/kubernetes/ingress-nginx/releases/tag/controller-v1.0.0-alpha.2
deploy this location's manifest deploy/static/provider/baremetal/deploy.yaml
And check above

/kind bug

The text was updated successfully, but these errors were encountered:

k8s-ci-robot · 2021-08-06T07:38:45Z

@sysnet4admin: This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

cleveritcz · 2021-08-06T09:18:37Z

I see this in version 0.43 also

longwuyuan · 2021-08-06T09:30:26Z

If your kubernetes version is 1.22, can you try the beta release of the controller and provide feedback https://github.com/kubernetes/ingress-nginx/releases/download/helm-chart-4.0.0/ingress-nginx-4.0.0.tgz

/remove-kind bug
/kind feature

sysnet4admin · 2021-08-06T09:41:24Z

@longwuyuan Thank you for your advice. I am afraid to say that I can't use helm in my environment.
In my humble view, it is in the fixed code in reflector related. so it could fix easy to do.

longwuyuan · 2021-08-06T09:46:34Z

Ok. Can you try this one and give feedback. https://github.com/kubernetes/ingress-nginx/blob/3c0bfc1ca3eb48246b12e77d40bde1162633efae/deploy/static/provider/baremetal/deploy.yaml

sysnet4admin · 2021-08-06T10:01:21Z

Oh...it fixed and a little remained error(due to probably uptime) but it looks good.

[root@m-k8s ~]# k get po -n ingress-nginx
NAME                                        READY   STATUS      RESTARTS      AGE
ingress-nginx-admission-create--1-ggdgq     0/1     Completed   0             27s
ingress-nginx-admission-patch--1-5v9k9      0/1     Error       2 (21s ago)   27s
ingress-nginx-controller-7f8cb86456-6bxsl   0/1     Running     0             27s
[root@m-k8s ~]# k logs -n ingress-nginx ingress-nginx-controller-7f8cb86456-6bxsl
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v1.0.0-beta.1
  Build:         da790570bd8d07d4980b175719f16c194301950d
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.20.1

-------------------------------------------------------------------------------

W0806 09:57:33.651154       6 client_config.go:615] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0806 09:57:33.652807       6 main.go:221] "Creating API client" host="https://10.96.0.1:443"
I0806 09:57:33.691838       6 main.go:265] "Running in Kubernetes cluster" major="1" minor="22" git="v1.22.0" state="clean" commit="c2b5237ccd9c0f1d600d3072634ca66cefdf272f" platform="linux/amd64"
I0806 09:57:35.067851       6 main.go:104] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
I0806 09:57:35.132605       6 ssl.go:532] "loading tls certificate" path="/usr/local/certificates/cert" key="/usr/local/certificates/key"
I0806 09:57:35.191669       6 nginx.go:254] "Starting NGINX Ingress controller"
I0806 09:57:35.282129       6 event.go:282] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"0292ea6f-3800-4816-9972-5d1dcd729033", APIVersion:"v1", ResourceVersion:"87397", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I0806 09:57:36.395092       6 nginx.go:296] "Starting NGINX process"
I0806 09:57:36.395634       6 nginx.go:316] "Starting validation webhook" address=":8443" certPath="/usr/local/certificates/cert" keyPath="/usr/local/certificates/key"
I0806 09:57:36.395751       6 leaderelection.go:243] attempting to acquire leader lease ingress-nginx/ingress-controller-leader...
I0806 09:57:36.397089       6 controller.go:150] "Configuration changes detected, backend reload required"
I0806 09:57:36.426012       6 leaderelection.go:253] successfully acquired lease ingress-nginx/ingress-controller-leader
I0806 09:57:36.427361       6 status.go:84] "New leader elected" identity="ingress-nginx-controller-7f8cb86456-6bxsl"
I0806 09:57:36.543654       6 status.go:204] "POD is not ready" pod="ingress-nginx/ingress-nginx-controller-7f8cb86456-6bxsl" node="w2-k8s"
I0806 09:57:36.867743       6 controller.go:167] "Backend successfully reloaded"
I0806 09:57:36.868060       6 controller.go:178] "Initial sync, sleeping for 1 second"
I0806 09:57:36.870359       6 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-7f8cb86456-6bxsl", UID:"d94365af-9b5f-4979-9465-deed849c520b", APIVersion:"v1", ResourceVersion:"87538", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
[root@m-k8s ~]# k get po -n ingress-nginx
NAME                                        READY   STATUS             RESTARTS      AGE
ingress-nginx-admission-create--1-ggdgq     0/1     Completed          0             40s
ingress-nginx-admission-patch--1-5v9k9      0/1     CrashLoopBackOff   2 (16s ago)   40s
ingress-nginx-controller-7f8cb86456-6bxsl   1/1     Running            0             40s

For your reference (webhook)

[root@m-k8s ~]# k get po -n ingress-nginx
NAME                                        READY   STATUS             RESTARTS      AGE
ingress-nginx-admission-create--1-ggdgq     0/1     Completed          0             2m46s
ingress-nginx-admission-patch--1-5v9k9      0/1     CrashLoopBackOff   4 (73s ago)   2m46s
ingress-nginx-controller-7f8cb86456-6bxsl   1/1     Running            0             2m46s
[root@m-k8s ~]# ^C
[root@m-k8s ~]# k logs -n ingress-nginx ingress-nginx-admission-patch--1-5v9k9
W0806 10:00:04.106894       1 client_config.go:608] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
{"level":"info","msg":"patching webhook configurations 'ingress-nginx-admission' mutating=false, validating=true, failurePolicy=Fail","source":"k8s/k8s.go:39","time":"2021-08-06T10:00:04Z"}
{"err":"the server could not find the requested resource","level":"fatal","msg":"failed getting validating webhook","source":"k8s/k8s.go:48","time":"2021-08-06T10:00:04Z"}

sysnet4admin · 2021-08-06T10:24:47Z

And ingrss.yaml is worked after removing ValidatingWebhookConfiguration

[root@m-k8s ~]# ka  ingress.yaml
Error from server (InternalError): error when creating "_Lecture_k8s_learning.kit/ch4/4.8/ingress.yaml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": Post "https://ingress-nginx-controller-admission.ingress-nginx.svc:443/networking/v1/ingresses?timeout=10s": dial tcp 10.101.69.76:443: connect: connection refused

[root@m-k8s ~]# kubectl delete ValidatingWebhookConfiguration ingress-nginx-admission
validatingwebhookconfiguration.admissionregistration.k8s.io "ingress-nginx-admission" deleted
[root@m-k8s ~]#
[root@m-k8s ~]#
[root@m-k8s ~]# k get po -n ingress-nginx
NAME                                       READY   STATUS             RESTARTS      AGE
ingress-nginx-admission-create--1-nc6zq    0/1     Completed          0             3m55s
ingress-nginx-admission-patch--1-jjlp5     0/1     CrashLoopBackOff   5 (55s ago)   3m55s
ingress-nginx-controller-98c5d9f78-2fwdn   1/1     Running            0             3m55s
[root@m-k8s ~]# ka  ingress.yaml
ingress.networking.k8s.io/nginx-ingress created

#7418 is related issue in my view.

**File: ingress.yaml **

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
<snipped>

sysnet4admin · 2021-08-06T10:32:29Z

And then it won't work..
Probably due to bottom logs.

[root@m-k8s ~]# k get svc
NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
ing-default   ClusterIP   10.97.68.215    <none>        80/TCP    5h11m
ing-ip        ClusterIP   10.108.232.58   <none>        80/TCP    5h13m
kubernetes    ClusterIP   10.96.0.1       <none>        443/TCP   31h
[root@m-k8s ~]# k get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.96.120.26   <none>        80:32255/TCP,443:30783/TCP   16m
ingress-nginx-controller-admission   ClusterIP   10.101.69.76   <none>        443/TCP                      16m
[root@m-k8s ~]# curl 192.168.1.101:32255
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
[root@m-k8s ~]# k get po -n ingress-nginx
NAME                                       READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create--1-nc6zq    0/1     Completed   0          17m
ingress-nginx-controller-98c5d9f78-2fwdn   1/1     Running     0          17m
[root@m-k8s ~]# k logs -n ingress-nginx ingress-nginx-admission-create--1-nc6zq create
W0806 10:13:03.618640       1 client_config.go:608] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
{"err":"secrets \"ingress-nginx-admission\" not found","level":"info","msg":"no secret found","source":"k8s/k8s.go:106","time":"2021-08-06T10:13:03Z"}
{"level":"info","msg":"creating new secret","source":"cmd/create.go:23","time":"2021-08-06T10:13:03Z"}
[root@m-k8s ~]# k logs -n ingress-nginx ingress-nginx-controller-98c5d9f78-2fwdn
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v1.0.0-beta.1
  Build:         da790570bd8d07d4980b175719f16c194301950d
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.20.1

-------------------------------------------------------------------------------

W0806 10:13:09.664134       6 client_config.go:615] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0806 10:13:09.664602       6 main.go:221] "Creating API client" host="https://10.96.0.1:443"
I0806 10:13:09.721811       6 main.go:265] "Running in Kubernetes cluster" major="1" minor="22" git="v1.22.0" state="clean" commit="c2b5237ccd9c0f1d600d3072634ca66cefdf272f" platform="linux/amd64"
I0806 10:13:10.800619       6 main.go:104] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
I0806 10:13:10.898707       6 nginx.go:254] "Starting NGINX Ingress controller"
I0806 10:13:10.950220       6 event.go:282] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"8324cb08-b757-4ae4-832b-ca4e7e22f3fb", APIVersion:"v1", ResourceVersion:"89132", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I0806 10:13:12.107254       6 nginx.go:296] "Starting NGINX process"
I0806 10:13:12.108160       6 leaderelection.go:243] attempting to acquire leader lease ingress-nginx/ingress-controller-leader...
I0806 10:13:12.110055       6 controller.go:150] "Configuration changes detected, backend reload required"
I0806 10:13:12.251382       6 leaderelection.go:253] successfully acquired lease ingress-nginx/ingress-controller-leader
I0806 10:13:12.252380       6 status.go:84] "New leader elected" identity="ingress-nginx-controller-98c5d9f78-2fwdn"
I0806 10:13:12.370032       6 status.go:204] "POD is not ready" pod="ingress-nginx/ingress-nginx-controller-98c5d9f78-2fwdn" node="w2-k8s"
I0806 10:13:12.485741       6 controller.go:167] "Backend successfully reloaded"
I0806 10:13:12.486978       6 controller.go:178] "Initial sync, sleeping for 1 second"
I0806 10:13:12.488785       6 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-98c5d9f78-2fwdn", UID:"ec472361-e351-457a-a7d2-44868e42b2d3", APIVersion:"v1", ResourceVersion:"89245", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
I0806 10:17:11.013144       6 store.go:361] "Ignoring ingress because of error while validating ingress class" ingress="default/nginx-ingress" error="ingress does not contain a valid IngressClass"

longwuyuan · 2021-08-06T11:19:24Z

Yes, please create/use ingressClass

sysnet4admin · 2021-08-06T11:47:32Z

@longwuyuan Thank you for your very kindness.
However ingressclass already applied by this code bundle..
https://github.com/kubernetes/ingress-nginx/blob/3c0bfc1ca3eb48246b12e77d40bde1162633efae/deploy/static/provider/baremetal/deploy.yaml

[root@m-k8s ~]# k get -n ingress-nginx ingressclasses.networking.k8s.io -o yaml
apiVersion: v1
items:
- apiVersion: networking.k8s.io/v1
  kind: IngressClass
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"networking.k8s.io/v1","kind":"IngressClass","metadata":{"annotations":{},"labels":{"app.kubernetes.io/component":"controller","app.kubernetes.io/instance":"ingress-nginx","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"ingress-nginx","app.kubernetes.io/version":"1.0.0-beta.1","helm.sh/chart":"ingress-nginx-4.0.0"},"name":"nginx"},"spec":{"controller":"k8s.io/ingress-nginx"}}
    creationTimestamp: "2021-08-06T10:12:59Z"
    generation: 1
    labels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/version: 1.0.0-beta.1
      helm.sh/chart: ingress-nginx-4.0.0
    name: nginx
    resourceVersion: "89155"
    uid: 90172d8a-e3be-48bc-8a95-984c6033eab5
  spec:
    controller: k8s.io/ingress-nginx
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

longwuyuan · 2021-08-06T11:54:17Z

From your ingress, can you please remove the annotation "nginx.ingress.kubernetes.io/rewrite-target: /" and add the annotation for the ingressclass. Then update here with result .

kubectl explain ingress.spec.ingressClassName

sysnet4admin · 2021-08-06T12:22:57Z

Remove from Ingress (metadata is remained)

[root@m-k8s ~]# k get ing nginx-ingress -o yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.k8s.io/v1","kind":"Ingress","metadata":{"annotations":{"nginx.ingress.kubernetes.io/rewrite-target":"/"},"name":"nginx-ingress","namespace":"default"},"spec":{"rules":[{"http":{"paths":[{"backend":{"service":{"name":"ing-default","port":{"number":80}}},"path":"/","pathType":"Prefix"},{"backend":{"service":{"name":"ing-hn","port":{"number":80}}},"path":"/hn","pathType":"Prefix"},{"backend":{"service":{"name":"ing-ip","port":{"number":80}}},"path":"/ip","pathType":"Prefix"}]}}]}}
  creationTimestamp: "2021-08-06T12:08:32Z"
  generation: 1
  name: nginx-ingress
  namespace: default
  resourceVersion: "94709"
  uid: 8aca632d-d1ce-43d3-a553-972d3011cb78
spec:
  rules:
  - http:
      paths:
      - backend:
          service:
            name: ing-default
            port:
              number: 80
        path: /
        pathType: Prefix
      - backend:
          service:
            name: ing-hn
            port:
              number: 80
        path: /hn
        pathType: Prefix
      - backend:
          service:
            name: ing-ip
            port:
              number: 80
        path: /ip
        pathType: Prefix
status:
  loadBalancer: {}

Add in ingressclass

[root@m-k8s ~]# k get ingressclasses.networking.k8s.io -o yaml
apiVersion: v1
items:
- apiVersion: networking.k8s.io/v1
  kind: IngressClass
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"networking.k8s.io/v1","kind":"IngressClass","metadata":{"annotations":{},"labels":{"app.kubernetes.io/component":"controller","app.kubernetes.io/instance":"ingress-nginx","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"ingress-nginx","app.kubernetes.io/version":"1.0.0-beta.1","helm.sh/chart":"ingress-nginx-4.0.0"},"name":"nginx"},"spec":{"controller":"k8s.io/ingress-nginx"}}
      nginx.ingress.kubernetes.io/rewrite-target: /
    creationTimestamp: "2021-08-06T12:06:55Z"
    generation: 1
    labels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/version: 1.0.0-beta.1
      helm.sh/chart: ingress-nginx-4.0.0
    name: nginx
    resourceVersion: "94774"
    uid: 4419599d-9fbd-4b07-9db3-6d8961f7bed1
  spec:
    controller: k8s.io/ingress-nginx
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

And revitalize nodeport (i.e. remove and create)

[root@m-k8s ~]# k get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.109.72.171   <none>        80:31464/TCP,443:31425/TCP   5m46s
ingress-nginx-controller-admission   ClusterIP   10.99.32.174    <none>        443/TCP                      12m
[root@m-k8s ~]# k get svc
NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
ing-default   ClusterIP   10.97.68.215    <none>        80/TCP    7h1m
ing-ip        ClusterIP   10.108.232.58   <none>        80/TCP    7h3m
kubernetes    ClusterIP   10.96.0.1       <none>        443/TCP   33h
[root@m-k8s ~]# k get nodes -o wide
NAME     STATUS   ROLES                  AGE   VERSION   INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION                CONTAINER-RUNTIME
m-k8s    Ready    control-plane,master   33h   v1.22.0   192.168.1.10    <none>        CentOS Linux 7 (Core)   3.10.0-1127.19.1.el7.x86_64   docker://20.10.8
w1-k8s   Ready    <none>                 33h   v1.22.0   192.168.1.101   <none>        CentOS Linux 7 (Core)   3.10.0-1127.19.1.el7.x86_64   docker://20.10.8
w2-k8s   Ready    <none>                 33h   v1.22.0   192.168.1.102   <none>        CentOS Linux 7 (Core)   3.10.0-1127.19.1.el7.x86_64   docker://20.10.8
w3-k8s   Ready    <none>                 33h   v1.22.0   192.168.1.103   <none>        CentOS Linux 7 (Core)   3.10.0-1127.19.1.el7.x86_64   docker://20.10.8
[root@m-k8s ~]# curl 192.168.1.101:31464
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>

But it failed.
And controller logs like below:

[root@m-k8s ~]# k logs -n ingress-nginx ingress-nginx-controller-7f8cb86456-pxsq9
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v1.0.0-beta.1
  Build:         da790570bd8d07d4980b175719f16c194301950d
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.20.1

-------------------------------------------------------------------------------

W0806 12:07:05.397303       6 client_config.go:615] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0806 12:07:05.398257       6 main.go:221] "Creating API client" host="https://10.96.0.1:443"
I0806 12:07:05.538857       6 main.go:265] "Running in Kubernetes cluster" major="1" minor="22" git="v1.22.0" state="clean" commit="c2b5237ccd9c0f1d600d3072634ca66cefdf272f" platform="linux/amd64"
I0806 12:07:06.184795       6 main.go:104] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
I0806 12:07:06.292468       6 ssl.go:532] "loading tls certificate" path="/usr/local/certificates/cert" key="/usr/local/certificates/key"
I0806 12:07:06.349039       6 nginx.go:254] "Starting NGINX Ingress controller"
I0806 12:07:06.403404       6 event.go:282] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"684ebaa5-1107-4259-b633-fc5445622dab", APIVersion:"v1", ResourceVersion:"94208", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I0806 12:07:07.552378       6 nginx.go:296] "Starting NGINX process"
I0806 12:07:07.552969       6 nginx.go:316] "Starting validation webhook" address=":8443" certPath="/usr/local/certificates/cert" keyPath="/usr/local/certificates/key"
I0806 12:07:07.553269       6 leaderelection.go:243] attempting to acquire leader lease ingress-nginx/ingress-controller-leader...
I0806 12:07:07.560267       6 controller.go:150] "Configuration changes detected, backend reload required"
I0806 12:07:07.593921       6 leaderelection.go:253] successfully acquired lease ingress-nginx/ingress-controller-leader
I0806 12:07:07.595820       6 status.go:84] "New leader elected" identity="ingress-nginx-controller-7f8cb86456-pxsq9"
I0806 12:07:07.687481       6 status.go:204] "POD is not ready" pod="ingress-nginx/ingress-nginx-controller-7f8cb86456-pxsq9" node="w2-k8s"
I0806 12:07:07.829628       6 controller.go:167] "Backend successfully reloaded"
I0806 12:07:07.829862       6 controller.go:178] "Initial sync, sleeping for 1 second"
I0806 12:07:07.830737       6 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-7f8cb86456-pxsq9", UID:"f599d589-31e9-42de-8848-628d7ef3adde", APIVersion:"v1", ResourceVersion:"94321", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
I0806 12:08:32.135188       6 store.go:361] "Ignoring ingress because of error while validating ingress class" ingress="default/nginx-ingress" error="ingress does not contain a valid IngressClass"

here too.

[root@m-k8s ~]# kubectl explain ingress.spec.ingressClassName
KIND:     Ingress
VERSION:  networking.k8s.io/v1

FIELD:    ingressClassName <string>

DESCRIPTION:
     IngressClassName is the name of the IngressClass cluster resource. The
     associated IngressClass defines which controller will implement the
     resource. This replaces the deprecated `kubernetes.io/ingress.class`
     annotation. For backwards compatibility, when that annotation is set, it
     must be given precedence over this field. The controller may emit a warning
     if the field and annotation have different values. Implementations of this
     API should ignore Ingresses without a class specified. An IngressClass
     resource may be marked as default, which can be used to set a default value
     for this field. For more information, refer to the IngressClass
     documentation.

rikatz · 2021-08-06T13:03:37Z

Hi @sysnet4admin

So for ingress v1, some things have changed with Ingress Class:

If you have only one ingress controller running in your environment and you still want to use ingress class, you should add the annotation "ingressclass.kubernetes.io/is-default-class" in your ingress class, so new Ingress objects will have this one as defaulted
If you don't care about ingress class, or you have a lot of Ingress objects without ingress class configuration, you can run the ingress-controller with the option --watch-ingress-without-class=true
Can you please make a new attempt and let us know?

Thanks!

sysnet4admin · 2021-08-06T13:38:54Z

@rikatz I appropriate your help.
It solve by 2nd option and I aim to try 1st option due to my curiosity :)
Here is the result of 2nd option.

[root@m-k8s ~]# k get po -n ingress-nginx
NAME                                        READY   STATUS      RESTARTS      AGE
ingress-nginx-admission-create--1-6hlqd     0/1     Completed   0             63s
ingress-nginx-admission-patch--1-qjhld      0/1     Error       3 (40s ago)   63s
ingress-nginx-controller-64dd9bfd84-df622   1/1     Running     0             64s
[root@m-k8s ~]# k logs -n ingress-nginx ingress-nginx-controller-64dd9bfd84-df622
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v1.0.0-beta.1
  Build:         da790570bd8d07d4980b175719f16c194301950d
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.20.1

-------------------------------------------------------------------------------

W0806 13:30:24.461379       8 client_config.go:615] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0806 13:30:24.462161       8 main.go:221] "Creating API client" host="https://10.96.0.1:443"
I0806 13:30:24.535573       8 main.go:265] "Running in Kubernetes cluster" major="1" minor="22" git="v1.22.0" state="clean" commit="c2b5237ccd9c0f1d600d3072634ca66cefdf272f" platform="linux/amd64"
I0806 13:30:25.486555       8 main.go:104] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
I0806 13:30:25.557902       8 ssl.go:532] "loading tls certificate" path="/usr/local/certificates/cert" key="/usr/local/certificates/key"
I0806 13:30:25.614048       8 nginx.go:254] "Starting NGINX Ingress controller"
I0806 13:30:25.655365       8 event.go:282] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"b2807e0e-6e43-4eaf-80f2-268629cee9f3", APIVersion:"v1", ResourceVersion:"102630", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I0806 13:30:26.730882       8 store.go:365] "Found valid IngressClass" ingress="default/nginx-ingress" ingressclass="_"
I0806 13:30:26.735768       8 event.go:282] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"nginx-ingress", UID:"8aca632d-d1ce-43d3-a553-972d3011cb78", APIVersion:"networking.k8s.io/v1", ResourceVersion:"94709", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync
I0806 13:30:26.815910       8 nginx.go:296] "Starting NGINX process"
I0806 13:30:26.816824       8 leaderelection.go:243] attempting to acquire leader lease ingress-nginx/ingress-controller-leader...
I0806 13:30:26.817134       8 nginx.go:316] "Starting validation webhook" address=":8443" certPath="/usr/local/certificates/cert" keyPath="/usr/local/certificates/key"
W0806 13:30:26.819656       8 controller.go:912] Error obtaining Endpoints for Service "default/ing-hn": no object matching key "default/ing-hn" in local store
I0806 13:30:26.820225       8 controller.go:150] "Configuration changes detected, backend reload required"
I0806 13:30:26.861440       8 leaderelection.go:253] successfully acquired lease ingress-nginx/ingress-controller-leader
I0806 13:30:26.884775       8 status.go:84] "New leader elected" identity="ingress-nginx-controller-64dd9bfd84-df622"
I0806 13:30:26.971081       8 status.go:204] "POD is not ready" pod="ingress-nginx/ingress-nginx-controller-64dd9bfd84-df622" node="w2-k8s"
I0806 13:30:27.099884       8 controller.go:167] "Backend successfully reloaded"
I0806 13:30:27.099967       8 controller.go:178] "Initial sync, sleeping for 1 second"
I0806 13:30:27.101717       8 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-64dd9bfd84-df622", UID:"e5a16488-9fb4-406f-97b7-79b52620c506", APIVersion:"v1", ResourceVersion:"102731", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
W0806 13:30:30.639407       8 controller.go:912] Error obtaining Endpoints for Service "default/ing-hn": no object matching key "default/ing-hn" in local store
W0806 13:30:39.472080       8 controller.go:912] Error obtaining Endpoints for Service "default/ing-hn": no object matching key "default/ing-hn" in local store
W0806 13:30:42.805819       8 controller.go:912] Error obtaining Endpoints for Service "default/ing-hn": no object matching key "default/ing-hn" in local store
I0806 13:31:27.031656       8 status.go:284] "updating Ingress status" namespace="default" ingress="nginx-ingress" currentValue=[] newValue=[{IP:192.168.1.102 Hostname: Ports:[]}]
W0806 13:31:27.047295       8 controller.go:912] Error obtaining Endpoints for Service "default/ing-hn": no object matching key "default/ing-hn" in local store
I0806 13:31:27.048271       8 event.go:282] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"nginx-ingress", UID:"8aca632d-d1ce-43d3-a553-972d3011cb78", APIVersion:"networking.k8s.io/v1", ResourceVersion:"102865", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync
[root@m-k8s ~]# k get ingress
NAME            CLASS    HOSTS   ADDRESS         PORTS   AGE
nginx-ingress   <none>   *       192.168.1.102   80      83m
[root@m-k8s ~]# k get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.107.114.168   <none>        80:32242/TCP,443:32725/TCP   2m18s
ingress-nginx-controller-admission   ClusterIP   10.107.66.204    <none>        443/TCP                      2m18s
[root@m-k8s ~]# curl 192.168.1.102:32242
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@m-k8s ~]# curl 192.168.1.101:32242
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@m-k8s ~]# curl 192.168.1.101:32242/ip
request_method : GET | ip_dest: 172.16.103.147

sysnet4admin · 2021-08-06T13:57:27Z

1st options is worked on properly too. but it should(?) remove ValidatingWebhookConfiguration ingress-nginx-admission to apply ingress backend configuration.

Here is the result of 1st option.

Check before run

[root@m-k8s ~]# cat  deploy.yaml | grep ingressclass.kubernetes.io/is-default-class
    ingressclass.kubernetes.io/is-default-class: "true"

Actual run

[root@m-k8s ~]# k get po -n ingress-nginx
NAME                                        READY   STATUS             RESTARTS      AGE
ingress-nginx-admission-create--1-m4f8v     0/1     Completed          0             2m
ingress-nginx-admission-patch--1-5tkp2      0/1     CrashLoopBackOff   4 (15s ago)   2m
ingress-nginx-controller-7f8cb86456-dz6p5   1/1     Running            0             2m
[root@m-k8s ~]# k logs -n ingress-nginx ingress-nginx-controller-7f8cb86456-dz6p5
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v1.0.0-beta.1
  Build:         da790570bd8d07d4980b175719f16c194301950d
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.20.1

-------------------------------------------------------------------------------

W0806 13:44:21.151566       6 client_config.go:615] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0806 13:44:21.156023       6 main.go:221] "Creating API client" host="https://10.96.0.1:443"
I0806 13:44:21.227695       6 main.go:265] "Running in Kubernetes cluster" major="1" minor="22" git="v1.22.0" state="clean" commit="c2b5237ccd9c0f1d600d3072634ca66cefdf272f" platform="linux/amd64"
I0806 13:44:21.795035       6 main.go:104] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
I0806 13:44:21.880159       6 ssl.go:532] "loading tls certificate" path="/usr/local/certificates/cert" key="/usr/local/certificates/key"
I0806 13:44:21.924233       6 nginx.go:254] "Starting NGINX Ingress controller"
I0806 13:44:21.971117       6 event.go:282] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"79f6f68c-5d6a-439b-be4f-a2e3e16fbc49", APIVersion:"v1", ResourceVersion:"104171", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I0806 13:44:23.053408       6 store.go:361] "Ignoring ingress because of error while validating ingress class" ingress="default/nginx-ingress" error="ingress does not contain a valid IngressClass"
I0806 13:44:23.126723       6 nginx.go:296] "Starting NGINX process"
I0806 13:44:23.127932       6 nginx.go:316] "Starting validation webhook" address=":8443" certPath="/usr/local/certificates/cert" keyPath="/usr/local/certificates/key"
I0806 13:44:23.128563       6 leaderelection.go:243] attempting to acquire leader lease ingress-nginx/ingress-controller-leader...
I0806 13:44:23.130649       6 controller.go:150] "Configuration changes detected, backend reload required"
I0806 13:44:23.174171       6 leaderelection.go:253] successfully acquired lease ingress-nginx/ingress-controller-leader
I0806 13:44:23.175632       6 status.go:84] "New leader elected" identity="ingress-nginx-controller-7f8cb86456-dz6p5"
I0806 13:44:23.255121       6 status.go:204] "POD is not ready" pod="ingress-nginx/ingress-nginx-controller-7f8cb86456-dz6p5" node="w2-k8s"
I0806 13:44:23.404692       6 controller.go:167] "Backend successfully reloaded"
I0806 13:44:23.404790       6 controller.go:178] "Initial sync, sleeping for 1 second"
I0806 13:44:23.407392       6 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-7f8cb86456-dz6p5", UID:"9d6059ef-7a9a-4c5c-a897-bbc49b7d3e38", APIVersion:"v1", ResourceVersion:"104261", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
[root@m-k8s ~]# ka  ingress.yaml
Error from server (InternalError): error when creating "_Lecture_k8s_learning.kit/ch4/4.8/ingress.yaml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": Post "https://ingress-nginx-controller-admission.ingress-nginx.svc:443/networking/v1/ingresses?timeout=10s": x509: certificate signed by unknown authority
[root@m-k8s ~]# kubectl delete ValidatingWebhookConfiguration ingress-nginx-admission
validatingwebhookconfiguration.admissionregistration.k8s.io "ingress-nginx-admission" deleted
[root@m-k8s ~]# ka  ingress.yaml
ingress.networking.k8s.io/nginx-ingress created
[root@m-k8s ~]# k logs -n ingress-nginx ingress-nginx-controller-7f8cb86456-dz6p5
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v1.0.0-beta.1
  Build:         da790570bd8d07d4980b175719f16c194301950d
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.20.1

-------------------------------------------------------------------------------

W0806 13:44:21.151566       6 client_config.go:615] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0806 13:44:21.156023       6 main.go:221] "Creating API client" host="https://10.96.0.1:443"
I0806 13:44:21.227695       6 main.go:265] "Running in Kubernetes cluster" major="1" minor="22" git="v1.22.0" state="clean" commit="c2b5237ccd9c0f1d600d3072634ca66cefdf272f" platform="linux/amd64"
I0806 13:44:21.795035       6 main.go:104] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
I0806 13:44:21.880159       6 ssl.go:532] "loading tls certificate" path="/usr/local/certificates/cert" key="/usr/local/certificates/key"
I0806 13:44:21.924233       6 nginx.go:254] "Starting NGINX Ingress controller"
I0806 13:44:21.971117       6 event.go:282] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"79f6f68c-5d6a-439b-be4f-a2e3e16fbc49", APIVersion:"v1", ResourceVersion:"104171", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I0806 13:44:23.053408       6 store.go:361] "Ignoring ingress because of error while validating ingress class" ingress="default/nginx-ingress" error="ingress does not contain a valid IngressClass"
I0806 13:44:23.126723       6 nginx.go:296] "Starting NGINX process"
I0806 13:44:23.127932       6 nginx.go:316] "Starting validation webhook" address=":8443" certPath="/usr/local/certificates/cert" keyPath="/usr/local/certificates/key"
I0806 13:44:23.128563       6 leaderelection.go:243] attempting to acquire leader lease ingress-nginx/ingress-controller-leader...
I0806 13:44:23.130649       6 controller.go:150] "Configuration changes detected, backend reload required"
I0806 13:44:23.174171       6 leaderelection.go:253] successfully acquired lease ingress-nginx/ingress-controller-leader
I0806 13:44:23.175632       6 status.go:84] "New leader elected" identity="ingress-nginx-controller-7f8cb86456-dz6p5"
I0806 13:44:23.255121       6 status.go:204] "POD is not ready" pod="ingress-nginx/ingress-nginx-controller-7f8cb86456-dz6p5" node="w2-k8s"
I0806 13:44:23.404692       6 controller.go:167] "Backend successfully reloaded"
I0806 13:44:23.404790       6 controller.go:178] "Initial sync, sleeping for 1 second"
I0806 13:44:23.407392       6 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-7f8cb86456-dz6p5", UID:"9d6059ef-7a9a-4c5c-a897-bbc49b7d3e38", APIVersion:"v1", ResourceVersion:"104261", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
I0806 13:48:17.343387       6 store.go:336] "Ignoring ingress because of error while validating ingress class" ingress="default/nginx-ingress" error="ingress does not contain a valid IngressClass"
2021/08/06 13:48:26 http: TLS handshake error from 172.16.171.64:32934: remote error: tls: bad certificate
I0806 13:49:11.736044       6 store.go:365] "Found valid IngressClass" ingress="default/nginx-ingress" ingressclass="nginx"
W0806 13:49:11.737919       6 controller.go:912] Error obtaining Endpoints for Service "default/ing-hn": no object matching key "default/ing-hn" in local store
I0806 13:49:11.738426       6 controller.go:150] "Configuration changes detected, backend reload required"
I0806 13:49:11.744970       6 event.go:282] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"nginx-ingress", UID:"4a18b17d-10b9-4704-89be-2259805c26ef", APIVersion:"networking.k8s.io/v1", ResourceVersion:"104781", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync
I0806 13:49:11.847534       6 controller.go:167] "Backend successfully reloaded"
I0806 13:49:11.849659       6 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-7f8cb86456-dz6p5", UID:"9d6059ef-7a9a-4c5c-a897-bbc49b7d3e38", APIVersion:"v1", ResourceVersion:"104261", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
I0806 13:49:23.210850       6 status.go:284] "updating Ingress status" namespace="default" ingress="nginx-ingress" currentValue=[] newValue=[{IP:192.168.1.102 Hostname: Ports:[]}]
W0806 13:49:23.223796       6 controller.go:912] Error obtaining Endpoints for Service "default/ing-hn": no object matching key "default/ing-hn" in local store
I0806 13:49:23.225543       6 event.go:282] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"nginx-ingress", UID:"4a18b17d-10b9-4704-89be-2259805c26ef", APIVersion:"networking.k8s.io/v1", ResourceVersion:"104804", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync
[root@m-k8s ~]# k get ingress
NAME            CLASS   HOSTS   ADDRESS         PORTS   AGE
nginx-ingress   nginx   *       192.168.1.102   80      39s
[root@m-k8s ~]# curl 192.168.1.101:31603
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

muka · 2021-08-06T14:45:15Z

I am following the test here using helm with v1.0.0-beta.2 but helm doesn't return from the install command and in the ingress controller pods I see

Failed to update lock: configmaps "ingress-controller-leader" is forbidden: User "system:serviceaccount:ingress-nginx:ingress-nginx" cannot update resource "configmaps" in API group "" in the namespace "ingress-nginx"

Also the pod ingress-nginx-admission-patch--1-h2694 errors as said above

rikatz · 2021-08-06T18:25:32Z

weird, will test the helm here.

About the migration, we are starting to document, and as you are brave users already helping us on this, wdyt? #7441 (review)

muka · 2021-08-09T08:28:19Z

By disabling the admission hook in the yaml values I got the ingress working again. Will check #7441 if we can help

imranrazakhan · 2021-08-18T10:40:35Z

@muka I tested beta3 and it works without disabling admission hook, I only have to add --watch-ingress-without-class=true

darox · 2021-08-19T13:19:14Z

Has this something to do with ?
8 reflector.go:138] k8s.io/client-go@v0.20.2/tools/cache/reflector.go:167: Failed to watch *v1beta1.Ingress: failed to list *v1beta1.Ingress: the server could not find the requested resource

EDIT: Beta version fixed the issue for me.

jjafrih · 2021-08-20T10:10:29Z

If your kubernetes version is 1.22, can you try the beta release of the controller and provide feedback https://github.com/kubernetes/ingress-nginx/releases/download/helm-chart-4.0.0/ingress-nginx-4.0.0.tgz

/remove-kind bug
/kind feature

I was facing the same issue on Kubernetes version 1.22, this helm release worked for me. However, I had to disable "ValidatingWebhookConfiguration".

rikatz · 2021-08-21T12:36:54Z

we have improved the validation webhook container as well.

I'm closing this, as apparently the v1 beta3 solves what was described here but feel free to reopen!

rikatz · 2021-08-21T12:37:07Z

/close

k8s-ci-robot · 2021-08-21T12:37:19Z

@rikatz: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

gaui · 2021-09-06T15:39:48Z

https://github.com/kubernetes/ingress-nginx/blob/main/charts/ingress-nginx/values.yaml#L67

rundeks · 2021-09-08T11:50:36Z

I am on Minikube version 1.23 and am seeing the same problem. Should I still need to use the beta version?

sysnet4admin added the kind/bug Categorizes issue or PR as related to a bug. label Aug 6, 2021

k8s-ci-robot added needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority labels Aug 6, 2021

k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. and removed kind/bug Categorizes issue or PR as related to a bug. labels Aug 6, 2021

rikatz mentioned this issue Aug 10, 2021

Release new helm chart with certgen fixed #7478

Merged

3 tasks

Cynerva mentioned this issue Aug 13, 2021

Fix nginx-ingress-controller on k8s 1.22 charmed-kubernetes/charm-kubernetes-worker#94

Merged

Travisivart mentioned this issue Aug 18, 2021

nginx-ingress-controller k8s 1.22 - Failed to watch *v1beta1.Ingress: failed to list *v1beta1.Ingress: the server could not find the requested resource bitnami/charts#7264

Closed

rikatz added this to the v1.0.0 milestone Aug 21, 2021

k8s-ci-robot closed this as completed Aug 21, 2021

prezha mentioned this issue Aug 22, 2021

Bump default Kubernetes version to v1.22.1 and update addons to with new API (ingress, gcpauth, olm and cilium) kubernetes/minikube#12325

Merged

matti mentioned this issue Sep 5, 2021

ingressclass.kubernetes.io/is-default-class: "true" does not work #7600

Closed

kingledion mentioned this issue Sep 8, 2021

Leaderelection: Failed to update lock #7612

Closed

This comment has been minimized.

Sign in to view

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

k8s.io/client-go@v0.20.2/tools/cache/reflector.go:167: Failed to watch v1beta1.Ingress: failed to list v1beta1.Ingress #7448

k8s.io/client-go@v0.20.2/tools/cache/reflector.go:167: Failed to watch v1beta1.Ingress: failed to list v1beta1.Ingress #7448

sysnet4admin commented Aug 6, 2021

k8s-ci-robot commented Aug 6, 2021

cleveritcz commented Aug 6, 2021

longwuyuan commented Aug 6, 2021

sysnet4admin commented Aug 6, 2021

longwuyuan commented Aug 6, 2021

sysnet4admin commented Aug 6, 2021

sysnet4admin commented Aug 6, 2021

sysnet4admin commented Aug 6, 2021

longwuyuan commented Aug 6, 2021 •

edited

Loading

sysnet4admin commented Aug 6, 2021

longwuyuan commented Aug 6, 2021 •

edited

Loading

sysnet4admin commented Aug 6, 2021

rikatz commented Aug 6, 2021

sysnet4admin commented Aug 6, 2021

sysnet4admin commented Aug 6, 2021

muka commented Aug 6, 2021 •

edited

Loading

rikatz commented Aug 6, 2021

muka commented Aug 9, 2021

imranrazakhan commented Aug 18, 2021

darox commented Aug 19, 2021 •

edited

Loading

jjafrih commented Aug 20, 2021

rikatz commented Aug 21, 2021

rikatz commented Aug 21, 2021

k8s-ci-robot commented Aug 21, 2021

gaui commented Sep 6, 2021

rundeks commented Sep 8, 2021

This comment has been minimized.

k8s.io/client-go@v0.20.2/tools/cache/reflector.go:167: Failed to watch *v1beta1.Ingress: failed to list *v1beta1.Ingress #7448

k8s.io/client-go@v0.20.2/tools/cache/reflector.go:167: Failed to watch *v1beta1.Ingress: failed to list *v1beta1.Ingress #7448

Comments

sysnet4admin commented Aug 6, 2021

k8s-ci-robot commented Aug 6, 2021

cleveritcz commented Aug 6, 2021

longwuyuan commented Aug 6, 2021

sysnet4admin commented Aug 6, 2021

longwuyuan commented Aug 6, 2021

sysnet4admin commented Aug 6, 2021

sysnet4admin commented Aug 6, 2021

sysnet4admin commented Aug 6, 2021

longwuyuan commented Aug 6, 2021 • edited Loading

sysnet4admin commented Aug 6, 2021

longwuyuan commented Aug 6, 2021 • edited Loading

sysnet4admin commented Aug 6, 2021

rikatz commented Aug 6, 2021

sysnet4admin commented Aug 6, 2021

sysnet4admin commented Aug 6, 2021

Check before run

Actual run

muka commented Aug 6, 2021 • edited Loading

rikatz commented Aug 6, 2021

muka commented Aug 9, 2021

imranrazakhan commented Aug 18, 2021

darox commented Aug 19, 2021 • edited Loading

jjafrih commented Aug 20, 2021

rikatz commented Aug 21, 2021

rikatz commented Aug 21, 2021

k8s-ci-robot commented Aug 21, 2021

gaui commented Sep 6, 2021

rundeks commented Sep 8, 2021

This comment has been minimized.

k8s.io/client-go@v0.20.2/tools/cache/reflector.go:167: Failed to watch v1beta1.Ingress: failed to list v1beta1.Ingress #7448

k8s.io/client-go@v0.20.2/tools/cache/reflector.go:167: Failed to watch v1beta1.Ingress: failed to list v1beta1.Ingress #7448

longwuyuan commented Aug 6, 2021 •

edited

Loading

longwuyuan commented Aug 6, 2021 •

edited

Loading

muka commented Aug 6, 2021 •

edited

Loading

darox commented Aug 19, 2021 •

edited

Loading