Issue with the default Clair configuration and not reporting vulnerabilities

[mfosterrox]

Hi Clair team,

Clair is being used to scan a container for vulnerabilities alongside ACS scannerv4. I have a situation where scanner v4 is reporting vulnerabilities and Clair is showing them as false. We've altered the configuration on for Clair to ignore_unpatched: false

introspection_addr: :8089
http_listen_addr: :8080
log_level: debug
indexer:
  connstring: '<omitted>'
  scanlock_retry: 10
  layer_scan_concurrency: 5
  migrations: True
matcher:
  connstring: '<omitted>'
  migrations: True
  indexer_addr: http://clair-indexer:8080/
  update_retention: 10
notifier:
  indexer_addr: http://clair-indexer:8080/
  matcher_addr: http://clair-matcher:8080/
  connstring: '<omitted>'
  migrations: True
  delivery_interval: 2m
  disable_summary: False
  poll_interval: 2m
updaters:
  config:
    rhel:
      ignore_unpatched: false
metrics:
  name: "prometheus"

Even with this configuration clair is not reporting vulnerabilities that Scannerv4 is picking up.

[crozzy]

Thanks for the issue. What is the version of Clair that you are using?

[hdonnay]

What are the differences?

Please post a JSON-format vulnerability report of an image where you see this behavior, also.

[willianrampazzo]

ACS is using 4.7.4, and Clair is using 4.8. I added both scan results for image registry.access.redhat.com/ubi9/ubi-micro@sha256:ac3f662eb885c37197c60c41042ee27abca8121d7f44671b8c1d14cb7db5eeaa. ACS reports vulnerabilities, while Clair does not.

acs.txt
clair.txt

[willianrampazzo]

I'm not sure if it is related, but looking at the logs, I can see recurring messages like:

{"level":"error","component":"libvuln/updates/Manager.Start","error":"updating errors:\nrhel-vex: context deadline exceeded\n","message":"errors encountered during updater run","environment":"prod"}

{"level":"error","component":"libvuln/updates/Manager.Start","error":"updating errors:\nrhel-vex: error reading tar contents: context deadline exceeded\n","message":"errors encountered during updater run","environment":"prod"}

[willianrampazzo]

To provide additional information, we had Clair version 4.7.4 running in combo mode. When I attempted to update to Clair version 4.8.0, I encountered some issues. It took some time to investigate, so I decided to discard the existing database and start a brand-new instance of Clair 4.8.0. The Clair instance that is not reporting vulnerabilities has been set up from an empty database.

[crozzy]

The errors that you pasted would certainly account for the lack of vulnerabilities, i.e. something failed during an updater run so none of the vulnerabilities were persisted to the DB.

Are there any more specific errors in the logs? It looks from this like a timeout is being triggered (possibly when downloading the fully archive of VEX data). This is currently set to 2 minutes and the archive is only pulled down when the application is first init'ed.

Recently RH prodsec made a change to include non-RH related CVEs in the VEX data, it is possible that this archive has grown a lot.

[willianrampazzo]

I see a lot of updater messages like:

{"level":"debug","updater":"rhel-vex","component":"rhel/vex/Updater.Fetch","url":"https://security.access.redhat.com/data/csaf/v2/vex/2023/cve-2023-50859.json","time":"2025-04-11T14:56:27Z","message":"copying body to file"}

With messages from other sources, and at the end:

{"level":"error","component":"libvuln/updates/Manager.Start","error":"updating errors:\nrhel-vex: context deadline exceeded\n","time":"2025-04-11T14:58:46Z","message":"errors encountered during updater run"}

This is the only error message I see. It happened today, so it is continuously trying without success.

Can we increase the timeout through any configuration option?

[crozzy]

The first log line is Clair pulling everything that has changed since the last archive was created and the final step of that is then pulling down the archive.

There is not currently a way to change the timeout but I made this after seeing this issue, so potentially in the near future.

[thomoe]

We see a similar effect using clair:4.7.4 (images marked green, Security scan passed), but with a slightly more verbose log message.

{
  "level": "error",
  "component": "libvuln/updates/Manager.Start",
  "error": "updating errors:\nRHEL8-advanced-virtualization-including-unpatched: failed to update: failed to queue vulnerability: failed to flush batch when queueing vulnerability: timeout: context deadline exceeded\nRHEL8-rhel-8.6-eus: failed to update: failed to queue vulnerability: failed to flush batch when queueing vulnerability: timeout: context deadline exceeded\nRHEL8-rhel-8-including-unpatched: failed to update: failed to queue vulnerability: failed to flush batch when queueing vulnerability: timeout: context deadline exceeded\n",
  "time": "2025-06-30T11:43:14Z",
  "message": "errors encountered during updater run"
}

This is our redacted configuration:

introspection_addr: :8089
http_listen_addr: :8080
log_level: debug
indexer:
  connstring: host=clair-rds.[redacted] port=5432 dbname=ClairBackend user=[redacted] password=[redacted]
  scanlock_retry: 10
  layer_scan_concurrency: 5
  migrations: true
matcher:
  connstring: host=clair-rds.[redacted] port=5432 dbname=ClairBackend user=[redacted] password=[redacted]
  migrations: true
  update_retention: 2
  period: 
matchers:
  names:
  - alpine
  - aws
  - debian
  - oracle
  - photon
  - python
  - rhel
  - suse
  - ubuntu
updaters:
  sets:
  - alpine
  - aws
  - debian
  - oracle
  - photon
  - pyupio
  - rhel
  - suse
  - ubuntu
notifier:
  connstring: host=clair-rds.[redacted] port=5432 dbname=ClairBackend user=[redacted] password=[redacted]
  delivery_interval: 1m
  poll_interval: 5m
  migrations: true
auth:
  psk:
    key: [redacted]
    iss: ["quay"]
# tracing and metrics
trace:
  name: "jaeger"
  probability: 1
  jaeger:
    agent:
      endpoint: "localhost:6831"
    service_name: "clair"
metrics:
  name: "prometheus"