ClamAV Docker health check always fail in Docker Engine 26.0.0+

[sammyhk]

As title, ClamAV Docker health check always fail in Docker Engine 26.0.0+ due to Docker Engine since 26.0.0 enabled IPv6 on loopback address (https://docs.docker.com/engine/release-notes/26.0/#bug-fixes-and-enhancements-2)

Always attempt to enable IPv6 on a container's loopback interface, and only include IPv6 in /etc/hosts if successful. moby/moby#47062

Which caused localhost resolved as IPv6 [::1] address and nc command in busybox fail to handle IPv6 address, causing the health check always fail:

clamav-docker/clamav/1.3/alpine/scripts/clamdcheck.sh

Line 6 in 5eb477e

if [ "$(echo "PING" | nc localhost 3310)" != "PONG" ]; then

[sammyhk]

Suggest to fix by hard-coding the the loopback IPv4 address 127.0.0.1 instead of using localhost.

[punkyard]

hi
i hope this is the right thread to publish this log emited by the dear szaimen Simon L. of Nextcloud AiO team in nextcloud/all-in-one#4987.

regards ✌️

2024-07-22T01:56:03.692353872Z Mon Jul 22 01:56:03 2024 -> SelfCheck: Database status OK.
2024-07-22T02:01:00.810800955Z Received signal: wake up
2024-07-22T02:01:00.811240021Z ClamAV update process started at Mon Jul 22 02:01:00 2024
2024-07-22T02:01:00.819865754Z daily database available for update (local version: 27342, remote version: 27343)
2024-07-22T02:01:02.455843305Z Testing database: '/var/lib/clamav/tmp.f04c0b4ba2/clamav-2214c2d0c700dff0ac40a399ae85eca2.tmp-daily.cld' ...
2024-07-22T02:01:16.117431721Z Database test passed.
2024-07-22T02:01:16.133739564Z daily.cld updated (version: 27343, sigs: 2064568, f-level: 90, builder: raynman)
2024-07-22T02:01:16.148758750Z main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
2024-07-22T02:01:16.149564207Z bytecode.cvd database is up-to-date (version: 335, sigs: 86, f-level: 90, builder: raynman)
2024-07-22T02:01:16.151591907Z Clamd successfully notified about the update.
2024-07-22T02:01:16.152758055Z Mon Jul 22 02:01:16 2024 -> Reading databases from /var/lib/clamav
2024-07-23T02:01:16.160314210Z Received signal: wake up
2024-07-23T02:01:16.161674945Z ClamAV update process started at Tue Jul 23 02:01:16 2024
2024-07-23T02:01:16.172695408Z daily database available for update (local version: 27343, remote version: 27344)
2024-07-23T02:01:17.516696561Z Testing database: '/var/lib/clamav/tmp.f04c0b4ba2/clamav-f27be4c5dc4a7a8a3c8db61e0ed15d76.tmp-daily.cld' ...
2024-07-23T02:01:28.720846976Z Database test passed.
2024-07-23T02:01:28.740170813Z daily.cld updated (version: 27344, sigs: 2064568, f-level: 90, builder: raynman)
2024-07-23T02:01:28.771097800Z main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
2024-07-23T02:01:28.771428242Z bytecode.cvd database is up-to-date (version: 335, sigs: 86, f-level: 90, builder: raynman)
2024-07-23T02:01:28.773908657Z WARNING: Clamd was NOT notified: Can't connect to clamd through /tmp/clamd.sock: Connection refused
2024-07-24T02:01:28.774485680Z Received signal: wake up
2024-07-24T02:01:28.774628646Z ClamAV update process started at Wed Jul 24 02:01:28 2024
2024-07-24T02:01:28.780848828Z daily database available for update (local version: 27344, remote version: 27345)
2024-07-24T02:01:30.035826136Z Testing database: '/var/lib/clamav/tmp.f04c0b4ba2/clamav-0d9e7de10713d7987fd2f83dca94ac3a.tmp-daily.cld' ...
2024-07-24T02:01:41.245038567Z Database test passed.
2024-07-24T02:01:41.298852677Z daily.cld updated (version: 27345, sigs: 2064656, f-level: 90, builder: raynman)
2024-07-24T02:01:41.310087002Z main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
2024-07-24T02:01:41.311091305Z bytecode.cvd database is up-to-date (version: 335, sigs: 86, f-level: 90, builder: raynman)
2024-07-24T02:01:41.312276796Z WARNING: Clamd was NOT notified: Can't connect to clamd through /tmp/clamd.sock: Connection refused
2024-07-25T02:01:41.313835241Z Received signal: wake up
2024-07-25T02:01:41.314245512Z ClamAV update process started at Thu Jul 25 02:01:41 2024
2024-07-25T02:01:41.318551774Z daily database available for update (local version: 27345, remote version: 27346)
2024-07-25T02:01:42.646589631Z Testing database: '/var/lib/clamav/tmp.f04c0b4ba2/clamav-f745418abdf1cc7e0635c2b6a23f0873.tmp-daily.cld' ...
2024-07-25T02:01:54.398083455Z Database test passed.
2024-07-25T02:01:54.443370652Z daily.cld updated (version: 27346, sigs: 2064675, f-level: 90, builder: raynman)
2024-07-25T02:01:54.450774064Z main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
2024-07-25T02:01:54.450874011Z bytecode.cvd database is up-to-date (version: 335, sigs: 86, f-level: 90, builder: raynman)
2024-07-25T02:01:54.451248940Z WARNING: Clamd was NOT notified: Can't connect to clamd through /tmp/clamd.sock: Connection refused
2024-07-26T02:01:54.451964831Z Received signal: wake up
2024-07-26T02:01:54.452099090Z ClamAV update process started at Fri Jul 26 02:01:54 2024
2024-07-26T02:01:54.456390219Z daily database available for update (local version: 27346, remote version: 27347)
2024-07-26T02:01:55.500021760Z Testing database: '/var/lib/clamav/tmp.f04c0b4ba2/clamav-35032e2383a10d7569ec4dbf10363b1a.tmp-daily.cld' ...
2024-07-26T02:02:07.421016887Z Database test passed.
2024-07-26T02:02:07.454167337Z daily.cld updated (version: 27347, sigs: 2064765, f-level: 90, builder: raynman)
2024-07-26T02:02:07.460691759Z main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
2024-07-26T02:02:07.460821452Z bytecode.cvd database is up-to-date (version: 335, sigs: 86, f-level: 90, builder: raynman)
2024-07-26T02:02:07.460897035Z WARNING: Clamd was NOT notified: Can't connect to clamd through /tmp/clamd.sock: Connection refused

[val-ms]

@punkyard the issue you're facing is unrelated.

The original complaint is that nc from Alpine's busybox package doesn't support IPv6, and since localhost resolves to an IPv6 address in Docker Engine 26.0.0, the status check, which uses nc won't work. The proposed solution here is to have the status check use an IPv4 address. I think that's ... okay. If someone wants to put in a PR, I'll review it.

Regarding your issue, @punkyard:

From your logs it looks like clamd started up okay. It has a log message "SelfCheck: Database status OK." which shows that it is monitoring if the database directory has any changes.

After freshclam runs the first time, it tells clamd to reload: Clamd successfully notified about the update.
After that, we don't hear from clamd again. All subsequent updates end with the warning: "WARNING: Clamd was NOT notified: Can't connect to clamd through /tmp/clamd.sock: Connection refused"
That means clamd isn't listening to the socket anymore.

My guess is you don't have enough RAM assigned to your container for clamd to do a concurrent reload - and so the OS killed the clamd process. Give it more RAM, and it should succeed. If you can't afford to do that, then you can override the clamd.conf with this option ConcurrentDatabaseReload no. That will cause clamd to pause scanning while it unloads the old database and reloads the new one. It means a gap in scanning, but will not require as much RAM to reload.

[ollioddi]

@micahsnyder i have the exact same issue as @punkyard as a Nextcloud user, however i have allocated almost 10 GB to the VM itself.

free -h
               total        used        free      shared  buff/cache   available
Mem:           9.8Gi       3.3Gi       516Mi       5.7Gi       6.0Gi       823Mi
Swap:             0B          0B          0B

From my usage, not having enough RAM does not seem to be the issue.

Logs:

ClamAV update process started at Mon Dec 30 19:35:17 2024
daily database available for update (local version: 27502, remote version: 27503)
Testing database: '/var/lib/clamav/tmp.804494b8eb/clamav-5c7322336445310d2be348c0abaf636e.tmp-daily.cld' ...
Database test passed.
daily.cld updated (version: 27503, sigs: 2071674, f-level: 90, builder: raynman)
main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
bytecode.cld database is up-to-date (version: 335, sigs: 86, f-level: 90, builder: raynman)
WARNING: Clamd was NOT notified: Can't connect to clamd through /tmp/clamd.sock: Connection refused

Any pointers?

[punkyard]

hi @micahsnyder
thanks for you kind reply

I have peaks at 5.3G of RAM usage on my 8G server

As i'm using ClamAV in Nextcloud All in One and Portainer
i've tried this:

$ docker update --memory="2g" --memory-swap="4g" nextcloud-aio-clamav
nextcloud-aio-clamav
$ docker inspect nextcloud-aio-clamav | grep -i memory
            "Memory": 2147483648,
            "MemoryReservation": 0,
            "MemorySwap": 4294967296,
            "MemorySwappiness": null,

now i'll see in a few days as generally ClamAV gets unhealthy after a few hours

Thanks for your concern