Error (timeout) when resolving internal ".local" domains

[mcanalesmayo]

Actual Behavior

RD is failing to retrieve docker images from an internal repository (in our case Artifactory). The repository domain is something like artifactory.{{company}}.com.
Images from other public repositories are pulled with no issues, so only the internal repository is failing.

However we found a workaround, please read Additional Information section.

Steps to Reproduce

docker pull artifactory.{{company}}.com/{{any_image}}

Or anything that tries to pull Docker images from an internal Docker repository.

Result

Error response from daemon: Get "artifactory.{{company}}.com": dial tcp: lookup artifactory.{{company}}.com on 192.168.5.3:53: read udp 192.168.5.15:35829->192.168.5.3:53: i/o timeout

Expected Behavior

Be able to pull the images from the internal repository

Additional Information

As discussed in Slack (https://rancher-users.slack.com/archives/C0200L1N1MM/p1652402759907389), we debugged this issue and found the root cause and workaround.
When running dig against the domain that is failing, we found that the internal company domain is resolved to a CNAME with .local domain, which then resolves it to its corresponding IP (see the ANSWER SECTION):

$ dig artifactory.{{company}}.com

; <<>> DiG 9.10.6 <<>> artifactory.{{company}}.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29767
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;artifactory.{{company}}.com.	IN	A

;; ANSWER SECTION:
artifactory.{{company}}.com. 2842 IN	CNAME	artifactory.{{another-domain}}.local.
artifactory.{{another-domain}}.local. 2716	IN	A	{{masked-ip}}

;; Query time: 47 msec
;; SERVER: {{masked}}
;; WHEN: Tue May 17 09:35:57 JST 2022
;; MSG SIZE  rcvd: 110

This is a common misconfiguration in the company network/DNS setup, as .local domains should not be used for regular domains. It is reserved for mDNS.

However, we found a workaround. First we need to stop Rancher Desktop. Then edit (or create if not there) the lima overrides ~/Library/Application\ Support/rancher-desktop/lima/_config/override.yaml:

hostResolver:
  hosts:
    {{internal-domain}}: {{masked-ip}}

Where {{internal-domain}} in our case was artifactory.{{company}}.com and {{masked-ip}} is the resolved IP from the record A in the ASNWER SECTION in the response above. This will bypass the .local domain, which is the one causing the issue. Then start RD again and now it's able to resolve the internal domain.

Some points to note about this workaround:

• The lima override.yaml will get deleted in case of RD factory reset.
• This file needs to be maintained, i.e. if this IP address changes then we need to again stop RD, edit the IP and start RD.

Rancher Desktop Version

1.3.0

Rancher Desktop K8s Version

1.23.6

Which container runtime are you using?

moby (docker cli)

What operating system are you using?

macOS

Operating System / Build Version

macOS Monterey 12.3.1

What CPU architecture are you using?

x64

Linux only: what package format did you use to install Rancher Desktop?

No response

Windows User Only

No response