k0s cluster - unable to connect with service

[sombriks]

Hello all,

i just installed a k0s cluster and put a sample app on it, then i tried the guide to proper install the API Gateway on it.

the thing is, i must be missing something.

when i port-forward the app i can access it:

[opc@spekkio manifests]$ kubectl port-forward services/simple-knex-koa-service 3000 &
[1] 25624
[opc@spekkio manifests]$ Forwarding from 127.0.0.1:3000 -> 3000
Forwarding from [::1]:3000 -> 3000

[opc@spekkio manifests]$ curl -i http://localhost:3000/books
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Content-Length: 488
Date: Sat, 09 Dec 2023 06:17:17 GMT
Connection: keep-alive
Keep-Alive: timeout=5

however, when i try via NodePort exposed API Gateway i get a 500.

[opc@spekkio manifests]$ curl -i http://localhost:32730
HTTP/1.1 502 Bad Gateway
Server: nginx/1.25.2
Date: Sat, 09 Dec 2023 06:19:09 GMT
Content-Type: text/html
Content-Length: 157
Connection: keep-alive

when this error happens, i get lines like these in gateway logs:

│ nginx-gateway-cd989895f-vlfw8 nginx 2023/12/09 06:32:39 [error] 164#164: *98 connect() failed (113: Host is unreachable) while connecting to upstream, client: 10.244.0.1, server: ~^, request: "GET /books HTTP/1.1", upstream: "http://10.244.0.2:3000/books", host: "localhost:32730"                                                                                           
│ nginx-gateway-cd989895f-vlfw8 nginx 2023/12/09 06:32:39 [warn] 164#164: *98 upstream server temporarily disabled while connecting to upstream, client: 10.244.0.1, server: ~^, request: "GET /books HTTP/1.1", upstream: "http://10.244.0.2:3000/books", host: "localhost:32730"                                                                                                   
│ nginx-gateway-cd989895f-vlfw8 nginx 2023/12/09 06:32:39 [error] 164#164: *98 connect() failed (113: Host is unreachable) while connecting to upstream, client: 10.244.0.1, server: ~^, request: "GET /books HTTP/1.1", upstream: "http://10.244.0.3:3000/books", host: "localhost:32730"                                                                                           
│ nginx-gateway-cd989895f-vlfw8 nginx 2023/12/09 06:32:39 [warn] 164#164: *98 upstream server temporarily disabled while connecting to upstream, client: 10.244.0.1, server: ~^, request: "GET /books HTTP/1.1", upstream: "http://10.244.0.3:3000/books", host: "localhost:32730"                                                                                                   
│ nginx-gateway-cd989895f-vlfw8 nginx 2023/12/09 06:32:39 [info] 164#164: *98 client 10.244.0.1 closed keepalive connection 

i am following those steps in the hope of proper configure the Gateway (i am trying to build a guide for my future self):

Gateway API on k0s

Simplest possible API Gateway on simplest possilbe kubernetes server

Steps

• Install Custom Resource Definitions for Kubernetes Api Gateway
• Install Nginx Controller for Api Gateway (Nginx Gateway Fabric)
• Install a gateway with proper Gatewayclass (nginx)
• Install a http-route linking a path and a service to the parent gateway
• Expose the gateway either via a nodeport service or a load balancer

Reference links

• https://gateway-api.sigs.k8s.io/guides/#installing-gateway-api
• https://docs.nginx.com/nginx-gateway-fabric/installation/installing-ngf/manifests/
• https://gateway-api.sigs.k8s.io/guides/simple-gateway/
• https://docs.nginx.com/nginx-gateway-fabric/installation/expose-nginx-gateway-fabric/#create-a-nodeport-service
• https://docs.nginx.com/nginx-gateway-fabric/how-to/traffic-management/routing-traffic-to-your-app/

This is my current gateway:

apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: local-gw
spec:
  gatewayClassName: nginx
  listeners:
  - protocol: HTTP
    port: 80
    name: local-gw-http
    allowedRoutes:
      namespaces:
        from: All

This is my HTTPRoute:

apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: app-route
spec:
  parentRefs:
    - name: local-gw
  rules:
    - matches:
      - path:
          type: PathPrefix
          value: /
      backendRefs:
        - name: simple-knex-koa-service
          port: 3000

This is the nodeport i am using to expose the Gateway:

[opc@spekkio ~]$ kubectl describe service/nginx-gateway -n nginx-gateway 
Name:                     nginx-gateway
Namespace:                nginx-gateway
Labels:                   app.kubernetes.io/instance=nginx-gateway
                          app.kubernetes.io/name=nginx-gateway
                          app.kubernetes.io/version=1.0.0
Annotations:              <none>
Selector:                 app.kubernetes.io/instance=nginx-gateway,app.kubernetes.io/name=nginx-gateway
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.98.204.51
IPs:                      10.98.204.51
Port:                     http  80/TCP
TargetPort:               80/TCP
NodePort:                 http  32730/TCP
Endpoints:                10.244.0.11:80
Port:                     https  443/TCP
TargetPort:               443/TCP
NodePort:                 https  31199/TCP
Endpoints:                10.244.0.11:443
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

Please let me know if there is any cluster information i am missing that could help to troubleshoot this and thanks in advance for any guidance.

[sombriks]

For anyone wondering te outcome of this i ended up using nginx ingress with k0s, way simpler to configure
https://docs.k0sproject.io/v1.28.4+k0s.0/examples/nginx-ingress/#install-nginx-using-host-network

i just applied the manifest and done, i was able to create a sample application+ingress. i guess i need to wait a few more releases to try api gateway again.