virtual-kubelet has different DNS resolution behaviour to standard kubelet

[kalioz]

The DNS configuration is not applied similarly between the AKS cluster and the ACI connector.

Due to the issue linked with https://github.com/virtual-kubelet/virtual-kubelet/issues/641, I added a dnsConfig bloc to my pods.

It worked for a while, but it is now failing to apply the dns change randomly; It didn't work at all since the start of the week.

Do you have any status on if #8 will be merged ? I could work with using a dnsConfig block, but I can't rely on it if it is only randomly applied.

In the ACI :

CLICK ME

yaml :

apiVersion: v1
kind: Pod
metadata:
  name: simple-pod-test-aci
spec:
  dnsConfig:
    options:
    - name: ndots
      value: "5"
    searches:
    - default.svc.cluster.local
    - svc.cluster.local
    - cluster.local
  dnsPolicy: ClusterFirst
  imagePullSecrets:
    - name: regcred
  containers:
  - image: ubuntu:latest
    imagePullPolicy: Always
    name: valo-back
    command: ["sleep"]
    args: ["1h"]
    ports:
    - containerPort: 8080
      name: http
      protocol: TCP
    resources:
      requests:
        cpu: "1"
        memory: "4G"
      limits:
        cpu: "1"
        memory: "4G"
  nodeSelector:
    kubernetes.io/role: agent
    beta.kubernetes.io/os: linux
    type: virtual-kubelet
  tolerations:
  - key: virtual-kubelet.io/provider
    operator: Exists
  - key: azure.com/aci
    effect: NoSchedule

/etc/resolv.conf :

nameserver 168.63.129.16
search wgomt5qrb0pevnpjyx4ypnwxvh.ax.internal.cloudapp.net

In the AKS:

CLICK ME

yaml :

apiVersion: v1
kind: Pod
metadata:
  name: simple-pod-test-not-aci
spec:
  dnsConfig:
    options:
    - name: ndots
      value: "5"
    searches:
    - default.svc.cluster.local
    - svc.cluster.local
    - cluster.local
  dnsPolicy: ClusterFirst
  imagePullSecrets:
    - name: regcred
  containers:
  - image: ubuntu:latest
    imagePullPolicy: Always
    name: valo-back
    command: ["sleep"]
    args: ["1h"]

/etc/resolv.conf :

nameserver 10.3.0.10
search default.svc.cluster.local svc.cluster.local cluster.local cuftdwxpgr1e3oqz1ew1iil53e.ax.internal.cloudapp.net
options ndots:5

Virtual-kubelet version : 1.2.1.1

Thanks !

[mgsnuno]

+1 for this, any updates on what is blocking #8 ?

[kalioz]

For information I found out the DNS configuration stopped working once we scaled the cluster with the azure portal.
When we reapply the configuration with terraform, the DNS configuration is reapplied (I don't have the change here, but from memory it was the network associated with the ACI that was removed silently by the manual scaling)

Still a bug, but it might not be associated with this project

[macolso]

hey @mgsnuno - we're reviewing #8 now

[mgsnuno]

@macolso thank you for the feedback. Any news on the progress or an eta? thanks again

[gossion]

#8 is already merged, @mgsnuno is the issue solved for you?

With your pod yaml example, I saw the dnsConfig was applied in my env

root@wk-caas-44333e90743f4866952460a8064ace09-3542fcc883eac9c9edcd28:/# cat /etc/resolv.conf
nameserver 10.0.0.10
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5

[mgsnuno]

stopped using virtual kubelet so if you got those results all fine by me to close this.