ver 1.4.0 connect local host?

[pyama86]

Hi team!

I found the following log.

hackmd-5444ccb8bb-vqwf6 app dial tcp 127.0.0.1:80: connect: connection refused
hackmd-5444ccb8bb-vqwf6 app dial tcp 127.0.0.1:80: connect: connection refused
hackmd-5444ccb8bb-vqwf6 app dial tcp 127.0.0.1:80: connect: connection refused
hackmd-5444ccb8bb-vqwf6 app dial tcp 127.0.0.1:80: connect: connection refused

We use hackmd with kubernetes and apply configure below.

          - name: CMD_DOMAIN
            value: "hackmd.xxxxx"
          - name: CMD_DB_URL
            valueFrom:
              secretKeyRef:
                name: hackmd-secret
                key: mysql_url
          - name: CMD_PROTOCOL_USESSL
            value: "true"
          - name: CMD_URL_ADDPORT
            value: "false"
          - name: CMD_ALLOW_ANONYMOUS
            value: "false"
          - name: CMD_ALLOW_ANONYMOUS_EDITS
            value: "false"
          - name: CMD_ALLOW_FREEURL
            value: "false"
          - name: CMD_ALLOW_EMAIL_REGISTER
            value: "false"
          - name: CMD_EMAIL
            value: "false"
          - name: CMD_DEFAULT_PERMISSION
            value: "limited"
          - name: CMD_ALLOW_ANONYMOUS_VIEWS
            value: "false"
          - name: CMD_PORT
            value: "3000"
          - name: DEBUG
            value: "true"
          - name: CMD_LOG_LEVEL
            value: "debug"

What log is this connection connected to?

[a60814billy]

please provide more details config

1. what docker image does you uses
2. Is CMD_DB_URL environment variable setup correctly? (the format of CMD_DB_URL is mysql://username:password@hostname:3306/hackmd_db_name)

[a60814billy]

By the way, official docker image (nabo.codimd.dev/hackmdio/hackmd:1.4.0) will check the connectivity of CMD_DB_URL, It seems your CMD_DB_URL is incorrect

[pyama86]

Hi @a60814billy

what docker image does you uses

I have used hackmdio/hackmd:1.4.0.

Is CMD_DB_URL environment variable setup correctly? (the format of CMD_DB_URL is mysql://username:password@hostname:3306/hackmd_db_name)

The item was set correctly, and it worked fine with the 1.3 series.

mysql://hackmd:password@host:3306/hackmd?charset=utf8mb4

[dsprenkels]

I got also bit by this issue. Just like @pyama86, this regressed in version 1.4.0 in my production setup.

Sequelize still resolves correctly, but it seems that there is something going with the host resolving in the pcheck binary.

In my case, I have the database at postgres://hackmd:[password]@codimd_db:5432/hackmd. The error says:

# docker-compose logs codimd
codimd_1_f2179ed6361c  | dial tcp 172.21.0.3:5432: connect: connection refused

However, when I resolve the host from inside the container, I get a different IP address:

root@66ee50e8e98a:/home/hackmd/app# host codimd_db
codimd_db has address 192.168.16.2

[a60814billy]

portchecker not working properly caused by golang not correct use /etc/host in docker container,
related issues: golang/go#22846

we will fix it in 1.4.1

[a60814billy]

Hi @dsprenkels and @pyama86,
We found the issue isn't the one that I attached on the above comment, that /etc/hosts should be parsed and resolved properly in our docker image.

This has been tested in the Docker CE 1.13, 17 on Debian Jessie, below is the docker-compose.yml file:

version: "3"
services:
  database:
    image: postgres:11.5
    environment:
      - POSTGRES_USER=codimd
      - POSTGRES_PASSWORD=change_password
      - POSTGRES_DB=codimd
    volumes:
      - "database-data:/var/lib/postgresql/data"
    restart: always
  codimd:
    image: nabo.codimd.dev/hackmdio/hackmd:1.4.0
    environment:
      - CMD_DB_URL=postgres://codimd:change_password@database/codimd
      - CMD_USECDN=false
    depends_on:
      - database
    ports:
      - "3000:3000"
    volumes:
      - upload-data:/home/hackmd/app/public/uploads
    restart: always
volumes:
  database-data: {}
  upload-data: {}

Also, this has been tested in Kubernetes on GCP of version 1.12, config file as follows:

apiVersion: v1
kind: Secret
metadata:
  name: db
  namespace: codimd-dev
type: Opaque
stringData:
  constr: mysql://hackmd:hackmdpass@mysql-svc.codimd-dev.svc:3306/hackmd?charset=utf8mb4
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: codi-mysql
  namespace: codimd-dev
spec:
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
        - name: mysql
          image: mysql:5.6
          env:
            - name: MYSQL_ROOT_PASSWORD
              value: hackmdpass
            - name: MYSQL_USER
              value: hackmd
            - name: MYSQL_PASSWORD
              value: hackmdpass
            - name: MYSQL_DATABASE
              value: hackmd
          ports:
            - containerPort: 3306
---
apiVersion: v1
kind: Service
metadata:
  name: mysql-svc
  namespace: codimd-dev
spec:
  selector:
    app: mysql
  type: ClusterIP
  ports:
    - port: 3306
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: codimd
  namespace: codimd-dev
spec:
  selector:
    matchLabels:
      app: codimd
  template:
    metadata:
      labels:
        app: codimd
        version: v1
    spec:
      containers:
        - name: codi
          image: nabo.codimd.dev/hackmdio/hackmd:1.4.0
          env:
            - name: CMD_DB_URL
              valueFrom:
                secretKeyRef:
                  name: db
                  key: constr

But all of above seems fine to me, it all can resolve the DB connection and the port checker (pchecker) is working.

Could you let me know what's your environment like the OS, k8s, docker versions and your config file? We're trying hard to reproduce this condition and to lure down this issue carefully.

Thank you!

[dsprenkels]

I am not completely sure what kind of info would be helpful, as the docker-container itself sure is identical to yours (except for the patch that disables pcheck).

I myself use docker-compose with docker, to spin up a bunch of different containers that run my webservices, of which CodiMD is one. They are all behind a single nginx reverse-proxy.

My docker-compose.yml file for the deployment version that had this issue can be found at: https://git.dsprenkels.com/?p=jati.git;a=blob_plain;f=roles/webserver/templates/docker-compose.yml.j2;hb=e69a11c8bdbc68788bd4f6ae8da96c50046ffac4
You can browse around in the general configuration, if you like.

Then there's a bunch of info that may be of use to you. An important detail could be that the container has 2 external network interfaces (one that has the nginx container and one that has the database container).

Host

daan@jati:~$ uname -a
Linux jati 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u6 (2018-10-08) x86_64 GNU/Linux

daan@jati:~$ sudo docker --version
Docker version 17.05.0-ce, build 89658be

daan@jati:~$ sudo docker-compose --version
docker-compose version 1.23.1, build b02f1306

daan@jati:/srv/webserver$ sudo docker network ls
NETWORK ID          NAME                            DRIVER              SCOPE
817f0fbdb31f        webserver_codimd                bridge              local
6b0602723fb5        webserver_codimd_db             bridge              local
# [more lines redacted]

Container

root@f5df4b8f969c:/home/hackmd/app# uname -a
Linux f5df4b8f969c 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u6 (2018-10-08) x86_64 GNU/Linux
root@f5df4b8f969c:/home/hackmd/app# cat /etc/hosts
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
192.168.16.3	f5df4b8f969c
172.21.0.3	f5df4b8f969c

root@f5df4b8f969c:/home/hackmd/app# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
39015: eth1@if39016: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:c0:a8:10:03 brd ff:ff:ff:ff:ff:ff
    inet 192.168.16.3/20 scope global eth1
       valid_lft forever preferred_lft forever
39017: eth0@if39018: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:ac:15:00:03 brd ff:ff:ff:ff:ff:ff
    inet 172.21.0.3/16 scope global eth0
       valid_lft forever preferred_lft forever

root@f5df4b8f969c:/home/hackmd/app# ip r
default via 172.21.0.1 dev eth0 
172.21.0.0/16 dev eth0  proto kernel  scope link  src 172.21.0.3 
192.168.16.0/20 dev eth1  proto kernel  scope link  src 192.168.16.3 

root@f5df4b8f969c:/home/hackmd/app# host codimd_db
codimd_db has address 192.168.16.2
Host codimd_db not found: 3(NXDOMAIN)

root@f5df4b8f969c:/home/hackmd/app# ping -c5 192.168.16.2
PING 192.168.16.2 (192.168.16.2) 56(84) bytes of data.
64 bytes from 192.168.16.2: icmp_seq=1 ttl=64 time=0.057 ms
64 bytes from 192.168.16.2: icmp_seq=2 ttl=64 time=0.049 ms
64 bytes from 192.168.16.2: icmp_seq=3 ttl=64 time=0.070 ms
64 bytes from 192.168.16.2: icmp_seq=4 ttl=64 time=0.049 ms
64 bytes from 192.168.16.2: icmp_seq=5 ttl=64 time=0.052 ms

--- 192.168.16.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4095ms
rtt min/avg/max/mdev = 0.049/0.055/0.070/0.010 ms

root@f5df4b8f969c:/home/hackmd/app# pcheck -constr postgres://hackmd@codimd_db:5432/hackmd
dial tcp 172.21.0.3:5432: connect: connection refused

# Hardcoding the ip address
root@f5df4b8f969c:/home/hackmd/app# pcheck -constr postgres://hackmd@192.168.16.2:5432/hackmd
dial tcp 192.168.16.2:5432: connect: connection success

[dsprenkels]

I think I may have a clue. I straced the pcheck command. Here is the output:

`strace` output

``` md_db:5432/hackmdc:/home/hackmd/app# strace pcheck -constr postgres://hackmd@codim execve("/usr/local/bin/pcheck", ["pcheck", "-constr", "postgres://hackmd@codimd_db:5432"...], [/* 32 vars */]) = 0 arch_prctl(ARCH_SET_FS, 0x645950) = 0 sched_getaffinity(0, 8192, {3, 0, 0, 0, 0, 0, 0, 0}) = 64 mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f12407b8000 mmap(0xc000000000, 67108864, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xc000000000 mmap(0xc000000000, 67108864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xc000000000 mmap(NULL, 33554432, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f123e7b8000 mmap(NULL, 2164736, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f123e5a7000 mmap(NULL, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f123e597000 clock_gettime(CLOCK_MONOTONIC, {7290960, 737130545}) = 0 mmap(NULL, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f123e587000 clock_gettime(CLOCK_MONOTONIC, {7290960, 737342223}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 737399740}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 737454618}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 737498804}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 737562801}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 737636045}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 737685402}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 737730468}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 737774911}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 737814732}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 737880230}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 737925969}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 737973626}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 738015653}) = 0 rt_sigprocmask(SIG_SETMASK, NULL, [], 8) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 738254308}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 738301400}) = 0 sigaltstack(NULL, {ss_sp=0, ss_flags=SS_DISABLE, ss_size=0}) = 0 sigaltstack({ss_sp=0xc000002000, ss_flags=0, ss_size=32768}, NULL) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 gettid() = 338 rt_sigaction(SIGHUP, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGHUP, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGINT, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGINT, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGQUIT, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGQUIT, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGILL, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGILL, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGTRAP, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGTRAP, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGABRT, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGABRT, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGBUS, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGBUS, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGFPE, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGFPE, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGUSR1, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGUSR1, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGSEGV, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGSEGV, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGUSR2, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGUSR2, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGPIPE, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGPIPE, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGALRM, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGALRM, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGTERM, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGTERM, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGSTKFLT, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGSTKFLT, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGCHLD, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGURG, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGURG, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGXCPU, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGXCPU, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGXFSZ, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGXFSZ, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGVTALRM, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGVTALRM, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGPROF, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGPROF, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGWINCH, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGWINCH, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGIO, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGIO, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGPWR, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGPWR, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGSYS, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGSYS, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRTMIN, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_1, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_2, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_2, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_3, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_3, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_4, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_4, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_5, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_5, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_6, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_6, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_7, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_7, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_8, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_8, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_9, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_9, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_10, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_10, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_11, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_11, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_12, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_12, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_13, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_13, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_14, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_14, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_15, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_15, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_16, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_16, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_17, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_17, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_18, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_18, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_19, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_19, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_20, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_20, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_21, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_21, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_22, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_22, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_23, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_23, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_24, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_24, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_25, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_25, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_26, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_26, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_27, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_27, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_28, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_28, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_29, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_29, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_30, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_30, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_31, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_31, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigaction(SIGRT_32, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGRT_32, {0x4577c0, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x4578f0}, NULL, 8) = 0 rt_sigprocmask(SIG_SETMASK, ~[], [], 8) = 0 clone(child_stack=0xc000046000, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM) = 339 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 rt_sigprocmask(SIG_SETMASK, ~[], [], 8) = 0 clone(child_stack=0xc000048000, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM) = 340 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 744744403}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 744800435}) = 0 rt_sigprocmask(SIG_SETMASK, ~[], [], 8) = 0 clone(child_stack=0xc000042000, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM) = 341 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0xc000036848, FUTEX_WAKE_PRIVATE, 1) = 1 futex(0xc0000364c8, FUTEX_WAKE_PRIVATE, 1) = 1 clock_gettime(CLOCK_MONOTONIC, {7290960, 745451781}) = 0 readlinkat(AT_FDCWD, "/proc/self/exe", "/usr/local/bin/pcheck", 128) = 21 fcntl(0, F_GETFL) = 0x8002 (flags O_RDWR|O_LARGEFILE) futex(0xc000036848, FUTEX_WAKE_PRIVATE, 1) = 1 mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f123e547000 fcntl(1, F_GETFL) = 0x8002 (flags O_RDWR|O_LARGEFILE) fcntl(2, F_GETFL) = 0x8002 (flags O_RDWR|O_LARGEFILE) mmap(NULL, 1439992, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f123e3e7000 clock_gettime(CLOCK_REALTIME, {1574891506, 583382377}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 746625066}) = 0 futex(0xc000036848, FUTEX_WAKE_PRIVATE, 1) = 1 openat(AT_FDCWD, "/etc/nsswitch.conf", O_RDONLY|O_CLOEXEC) = 3 epoll_create1(EPOLL_CLOEXEC) = 4 epoll_ctl(4, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=1045758008, u64=139716331896888}}) = -1 EPERM (Operation not permitted) epoll_ctl(4, EPOLL_CTL_DEL, 3, c00007c9fc) = -1 EPERM (Operation not permitted) read(3, "# /etc/nsswitch.conf\n#\n# Example"..., 1024) = 497 read(3, "", 1024) = 0 close(3) = 0 openat(AT_FDCWD, "/etc/resolv.conf", O_RDONLY|O_CLOEXEC) = 3 epoll_ctl(4, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=1045758008, u64=139716331896888}}) = -1 EPERM (Operation not permitted) epoll_ctl(4, EPOLL_CTL_DEL, 3, c00007c8cc) = -1 EPERM (Operation not permitted) fstat(3, {st_mode=S_IFREG|0644, st_size=38, ...}) = 0 read(3, "nameserver 127.0.0.11\noptions nd"..., 65536) = 38 read(3, "", 65498) = 0 read(3, "", 65536) = 0 uname({sys="Linux", node="f5df4b8f969c", ...}) = 0 close(3) = 0 newfstatat(AT_FDCWD, "/etc/mdns.allow", 0xc00006cc68, 0) = -1 ENOENT (No such file or directory) clock_gettime(CLOCK_REALTIME, {1574891506, 585226733}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 748430382}) = 0 newfstatat(AT_FDCWD, "/etc/hosts", {st_mode=S_IFREG|0644, st_size=200, ...}, 0) = 0 openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 3 epoll_ctl(4, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=1045758008, u64=139716331896888}}) = -1 EPERM (Operation not permitted) epoll_ctl(4, EPOLL_CTL_DEL, 3, c00009ce1c) = -1 EPERM (Operation not permitted) read(3, "127.0.0.1\tlocalhost\n::1\tlocalhos"..., 65536) = 200 read(3, "", 65336) = 0 read(3, "", 65536) = 0 close(3) = 0 clock_gettime(CLOCK_REALTIME, {1574891506, 586038554}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 749260662}) = 0 clock_gettime(CLOCK_REALTIME, {1574891506, 586188363}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 749368291}) = 0 futex(0xc000036848, FUTEX_WAKE_PRIVATE, 1) = 1 clock_gettime(CLOCK_REALTIME, {1574891506, 586388406}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 749592663}) = 0 clock_gettime(CLOCK_REALTIME, {1574891506, 586519613}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 749713465}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 749780963}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 749829330}) = 0 clock_gettime(CLOCK_REALTIME, {1574891506, 586749781}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 749926876}) = 0 clock_gettime(CLOCK_REALTIME, {1574891506, 586853628}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 750032661}) = 0 socket(PF_INET, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 3 setsockopt(3, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0 connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.11")}, 16) = 0 epoll_ctl(4, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=1045758008, u64=139716331896888}}) = 0 getsockname(3, {sa_family=AF_INET, sin_port=htons(38974), sin_addr=inet_addr("127.0.0.1")}, [16]) = 0 getpeername(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.11")}, [16]) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 750439578}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 750496319}) = 0 write(3, "z\344\1\0\0\1\0\0\0\0\0\0\6codimd\0\0\34\0\1", 24) = 24 read(3, 0xc0000c0200, 512) = -1 EAGAIN (Resource temporarily unavailable) clock_gettime(CLOCK_MONOTONIC, {7290960, 750709777}) = 0 clock_gettime(CLOCK_MONOTONIC, {7290960, 750758211}) = 0 futex(0x647c40, FUTEX_WAIT_PRIVATE, 0, {4, 999052055}dial tcp 172.21.0.3:5432: connect: connection refused dial tcp 172.21.0.3:5432: connect: connection refused ^CProcess 338 detached ```

Close to the end, we see that the DNS request seems to be write(3, "z\344\1\0\0\1\0\0\0\0\0\0\6codimd\0\0\34\0\1", 24) = 24. I.e. the IP for codimd is requested, not codimd_db. It kinda looks like everything after the underscore is being dropped!

If we request the IP for the nginx name instead, we do get the correct answer:

hackmd5df4b8f969c:/home/hackmd/app# pcheck -constr postgres://hackmd@nginx:5432/h
dial tcp 172.21.0.2:5432: connect: connection refused
dial tcp 172.21.0.2:5432: connect: connection refused
dial tcp 172.21.0.2:5432: connect: connection refused
dial tcp 172.21.0.2:5432: connect: connection refused
dial tcp 172.21.0.2:5432: connect: connection refused

root@f5df4b8f969c:/home/hackmd/app# host nginx
nginx has address 172.21.0.2
Host nginx not found: 3(NXDOMAIN)

So I guess I should fix the incorrect hostname of my codimd_db container.

@pyama86, is this consistent with your configurations?

[dsprenkels]

Small update: Removing the underscore from the hostname fixed the issue for me.