Bug: systemd-resolved issues

[ratcashdev]

What is Happening

starting dns-proxy from within a compose file, like:

services:
  dns-proxy:
    image: defreitas/dns-proxy-server
    hostname: dns
    volumes:
      - type: bind
        source: /var/run/docker.sock
        target: /var/run/docker.sock
      - type: bind
        source: /etc
        target: /host/etc
    ports:
      - "5380:5380"

ping dns executed from the host says Name or service not known
The logs, in addition contain the entry: Not possible to restart resolved service: service systemd-resolved restart

Actually, the command service is not present on my machine. Only systemctl restart systemd-resolved works.
However, even if I restart resolved manually, DNS lookup does not work through dns-proxy.

What is Expected

systemd-resolved is restarted and DNS lookup works.

Specs

• Docker Version: docker version 27.1.2
• DPS Version:
  • When using docker: the docker image version or run the following if you are using the latest docker run --rm defreitas/dns-proxy-server --version 3.25.10-snapshot
• OS: Linux mymachine 6.10.6-10-MANJARO 1 SMP PREEMPT_DYNAMIC Sat Aug 24 16:17:40 UTC 2024 x86_64 GNU/Linux
• Attach DPS Log file

12:03:31.664 [main           ] DEB c.m.d.config.dataprovider.ConfigPathBuilder       l=37   m=build                           status=configPathBuilt, path=/app/conf/config.json
12:03:31.675 [main           ] INF c.m.dnsproxyserver.config.dataprovider.JsonConfigsl=69   m=createDefaultConfigOnPath       status=createdDefaultConfigFile, path=/app/conf/config.json
12:03:31.676 [main           ] DEB c.m.d.config.dataprovider.ConfigDAOJson           l=32   m=find                            configPath=/app/conf/config.json
12:03:31.698 [main           ] INF c.m.d.s.d.a.DpsDockerEnvironmentSetupService      l=32   m=setup                           status=binding-docker-events, connectedToDocker=true
12:03:31.698 [main           ] INF c.m.d.s.d.a.DpsDockerEnvironmentSetupService      l=44   m=setupNetwork                    status=dpsNetwork, active=false
12:03:31.698 [main           ] INF c.m.d.s.docker.application.DpsContainerService    l=116  m=tRunningContainersToDpsNetwork  status=autoConnectDpsNetworkDisabled, dpsNetwork=false, dpsNetworkAutoConnect=false
12:03:31.698 [main           ] INF c.m.d.solver.docker.entrypoint.EventListener      l=32   m=onStart                         status=containerAutoConnectToDpsNetworkDisabled
12:03:31.698 [main           ] INF com.mageddo.dnsserver.UDPServerPool               l=31   m=start                           Starting UDP server, addresses=/0.0.0.0:53
12:03:31.698 [main           ] INF c.mageddo.dnsproxyserver.server.dns.ServerStarter l=30   m=start                           status=startingDnsServer, protocol=UDP_TCP, port=53
12:03:31.701 [virtual-44     ] INF com.mageddo.dnsserver.TCPServer                   l=56   m=start0                          status=tcpServerStarting, port=53
12:03:31.706 [main           ] INF com.mageddo.http.WebServer                        l=134  m=start                           status=startingWebServer, port=5380
12:03:36.733 [Thread-3       ] INF c.m.d.dnsconfigurator.DnsConfigurators            l=113  m=getInstance0                    usingDnsConfigurator=DnsConfiguratorLinux
12:03:36.733 [Thread-3       ] INF c.m.d.dnsconfigurator.linux.DnsConfiguratorLinux  l=122  m=init                            status=using, configFile=ResolvFile(path=/host/etc/systemd/resolved.conf, type=SYSTEMD_RESOLVED)
12:03:36.738 [Thread-3       ] WAR c.m.d.dnsconfigurator.linux.DnsConfiguratorLinux  l=145  m=tryRestartResolved              status=can't restart resolved service, please run: 'service systemd-resolved restart' to apply DPS as default DNS.
Not possible to restart resolved service: 1 : systemd-resolved: unrecognized service

[mageddo]

Hey @ratcashdev , DPS won´t be able to configure itself as the default DNS when running inside a docker container when the host is using system-resolved.

Yep, you will do need to restart systemd-resolved using service or systemctl command, see more details about the limitation.

If you are using DPS as your primary DNS solver I would recommend follow theses steps to configure it as a service and don't have to reconfigure systemd-resolved at every restart.

In alternative you can bypass systemd-resolved and use resolv.conf without the need to restart any service by following these instructions:

services:
  dns-proxy:
    image: defreitas/dns-proxy-server
    hostname: dns
    volumes:
      - type: bind
        source: /var/run/docker.sock
        target: /var/run/docker.sock
      - type: bind
        source: /etc
        target: /host/etc
   environment: 
     - MG_RESOLVCONF=/host/etc/resolv.conf
    ports:
      - "5380:5380"

[ratcashdev]

HI @mageddo thanks for the quick replay and solution.

DPS won´t be able to configure itself as the default DNS when running inside a docker container when the host is using system-resolved.

can you plz elaborate what is the fundamental issue here?

[mageddo]

Basically systemd-resolved need to be restarted and it's a host program, a host program can't be executed from the docker container as expected by the isolation principle, DPS is running inside a docker container so it can´t restart the host system-resolved at the host.

[ratcashdev]

@mageddo I have tried your suggested setup above. Unfortunately, it seems it still does not work.
here are the logs:

dns-proxy-1  | 06:41:24.864 [main           ] DEB c.m.d.config.dataprovider.ConfigPathBuilder       l=37   m=build                           status=configPathBuilt, path=/app/conf/config.json
dns-proxy-1  | 06:41:24.874 [main           ] INF c.m.dnsproxyserver.config.dataprovider.JsonConfigsl=69   m=createDefaultConfigOnPath       status=createdDefaultConfigFile, path=/app/conf/config.json
dns-proxy-1  | 06:41:24.874 [main           ] DEB c.m.d.config.dataprovider.ConfigDAOJson           l=32   m=find                            configPath=/app/conf/config.json
dns-proxy-1  | 06:41:24.890 [main           ] INF c.m.d.s.d.a.DpsDockerEnvironmentSetupService      l=32   m=setup                           status=binding-docker-events, connectedToDocker=true
dns-proxy-1  | 06:41:24.890 [main           ] INF c.m.d.s.d.a.DpsDockerEnvironmentSetupService      l=44   m=setupNetwork                    status=dpsNetwork, active=false
dns-proxy-1  | 06:41:24.890 [main           ] INF c.m.d.s.docker.application.DpsContainerService    l=116  m=tRunningContainersToDpsNetwork  status=autoConnectDpsNetworkDisabled, dpsNetwork=false, dpsNetworkAutoConnect=false
dns-proxy-1  | 06:41:24.890 [main           ] INF c.m.d.solver.docker.entrypoint.EventListener      l=32   m=onStart                         status=containerAutoConnectToDpsNetworkDisabled
dns-proxy-1  | 06:41:24.890 [main           ] INF com.mageddo.dnsserver.UDPServerPool               l=31   m=start                           Starting UDP server, addresses=/0.0.0.0:53
dns-proxy-1  | 06:41:24.890 [main           ] INF c.mageddo.dnsproxyserver.server.dns.ServerStarter l=30   m=start                           status=startingDnsServer, protocol=UDP_TCP, port=53
dns-proxy-1  | 06:41:24.892 [virtual-44     ] INF com.mageddo.dnsserver.TCPServer                   l=56   m=start0                          status=tcpServerStarting, port=53
dns-proxy-1  | 06:41:24.897 [main           ] INF com.mageddo.http.WebServer                        l=134  m=start                           status=startingWebServer, port=5380
dns-proxy-1  | 06:41:29.911 [Thread-2       ] INF c.m.d.dnsconfigurator.DnsConfigurators            l=113  m=getInstance0                    usingDnsConfigurator=DnsConfiguratorLinux
dns-proxy-1  | 06:41:29.911 [Thread-2       ] INF c.m.d.dnsconfigurator.linux.DnsConfiguratorLinux  l=91   m=lambda$findBestConfFile$0       status=noValidConfFile, file=/host/etc/resolv.conf
dns-proxy-1  | 06:41:29.911 [Thread-2       ] INF c.m.d.dnsconfigurator.linux.DnsConfiguratorLinux  l=122  m=init                            status=using, configFile=null

/etc/resolv.conf still stays the same, containing nameserver 127.0.0.53

[mageddo]

I will do a check, looks like DPS hasn't permission to edit resolv.conf

[ratcashdev]

yeah, pretty much:

-rw-r--r-- 1 systemd-resolve systemd-resolve 920 sep 30 16:23 /run/systemd/resolve/stub-resolv.conf

[mageddo]

@ratcashdev

Got it.

/host/etc/resolv.conf is a symlink

# ls -lha /host/etc/resolv*
lrwxrwxrwx 1 root root   39 Sep 15  2022 /host/etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf

So you need to run like below:

services:
  dns-proxy:
    image: defreitas/dns-proxy-server
    hostname: dns
    volumes:
      - /run/systemd/resolve/:/host/run/
    environment:
      MG_RESOLVCONF: /host/run/stub-resolv.conf
    ports:
      - "5380:5380"

dns-proxy-1  | 13:00:31.189 [Thread-3       ] INF c.m.d.dnsconfigurator.linux.DnsConfiguratorLinux  l=122  m=init                            status=using, configFile=ResolvFile(path=/host/run/stub-resolv.conf, type=RESOLVCONF)

Improving the logs to make the failing condition explicit at #585 .