Trivy java-db not updated automatically

[workerus]

Description

Trivy java-db not updated automatically.
When i open java-db/metadata.json on February 27th i see NextUpdate":"2023-02-26 which didn't happen for some reason.

cat java-db/metadata.json
{"Version":1,"NextUpdate":"2023-02-26T00:52:15.931018255Z","UpdatedAt":"2023-02-23T00:52:15.931018755Z","DownloadedAt":"2023-02-23T23:53:32.46565187Z"}

To update it i run the following command and it's working fine:

$ docker exec -it containerID trivy image --download-java-db-only

What did you expect to happen?

I expect java-db to be updated automatically every three day interval. Please note we have this issue only with java-db. The db is being updated automatically regularly without issues.

What happened instead?

We run Trivy Docker image and scan our images with --server argument.

./trivy image -s UNKNOWN,MEDIUM,HIGH,CRITICAL --timeout 600s --server http://ourTrivyServer ImageToScan

The problem is that Trivy not updating java-db automatically on ourTrivyServer and thus when i launch scan with --server argument since the db is not updated on our Trivy server this DB is updating locally and sometimes it takes very long time to download this DB.
For that reason it's important for us to have this DB updated on our Trivy server.

Output of run with -debug:

On February 27th i get this output:

cat java-db/metadata.json
{"Version":1,"NextUpdate":"2023-02-26T00:52:15.931018255Z","UpdatedAt":"2023-02-23T00:52:15.931018755Z","DownloadedAt":"2023-02-23T23:53:32.46565187Z"}

Output of trivy -v:

We use latest Trivy client and server.

# ./trivy -v
Version: 0.37.3

Additional details (base image name, container registry info...):

[DmitriyLewen]

Hello @workerus
Thanks for your report!

I have been playing with metadata.json over time and have not been able to reproduce your issue.
Can you reproduce issue on your PC and send me metadata.json files and your actual time(keep in mind we use UTC).

---

For that reason it's important for us to have this DB updated on our Trivy server.

We already have #3560
You can check changes in this issue.

Best Regards, Dmitriy

[workerus]

I wonder how do you reproduce it? Do you change date and time on your server? Yesterday we have updated trivy to 0.38.0 and updated manually java-db next update is scheduled for March 4th let's see if trivy will be able to updated the db without my intervention.

sudo cat /home/sre/.cache/trivy/java-db/metadata.json
{"Version":1,"NextUpdate":"2023-03-04T00:51:59.699061559Z","UpdatedAt":"2023-03-01T00:51:59.699062059Z","DownloadedAt":"2023-03-01T15:45:27.122697794Z"}

[DmitriyLewen]

Do you change date and time on your server?

Trivy-java-db saved on client.

I wonder how do you reproduce it?

Trivy checks NextUpdate field. You can just change this date.

e.g.:
i downloaded java-db:

➜  cat metadata.json 
{"Version":1,"NextUpdate":"2023-03-06T02:28:25.681664393Z","UpdatedAt":"2023-03-03T02:28:25.681665193Z","DownloadedAt":"2023-03-03T05:03:37.305657376Z"}

after that I changed day in NextUpdate to 2023-03-03T02:28:25.681664393Z

➜  cat metadata.json
{"Version":1,"NextUpdate":"2023-03-03T02:28:25.681664393Z","UpdatedAt":"2023-03-03T02:28:25.681665193Z","DownloadedAt":"2023-03-03T05:03:37.305657376Z"}

and tried to scan image with jar file:

➜  ~ trivy image --server http://0.0.0.0:10000 tomcat 
2023-03-03T11:07:44.686+0600	INFO	Vulnerability scanning is enabled
2023-03-03T11:07:44.686+0600	INFO	Secret scanning is enabled
2023-03-03T11:07:44.686+0600	INFO	If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2023-03-03T11:07:44.686+0600	INFO	Please see also https://aquasecurity.github.io/trivy/v0.37/docs/secret/scanning/#recommendation for faster secret detection
2023-03-03T11:07:48.807+0600	INFO	JAR files found
2023-03-03T11:07:48.807+0600	INFO	Downloading the Java DB...
853.72 KiB / 410.00 MiB [>_____________________] 0.20% 474.96 KiB p/s ETA 14m42s

[workerus]

Thank you for your reply @DmitriyLewen. I observe that if on my server java-db is updated then when i launch scan no java-db is saved on the client but when java-db is not updated on the server then client is downloading and saving java-db. I just tested it a few times and i can confirm this behaviour. Also this morning i see again on server side:

date
Mon Mar  6 16:27:48 CET 2023
# here you can see the db was updated all is good
cat /home/sre/.cache/trivy/db/metadata.json
{"Version":2,"NextUpdate":"2023-03-06T18:07:32.180603274Z","UpdatedAt":"2023-03-06T12:07:32.180603674Z","DownloadedAt":"2023-03-06T12:21:27.446961471Z"}

# here you can see the db wasn't updated yesterday
cat /home/sre/.cache/trivy/java-db/metadata.json
{"Version":1,"NextUpdate":"2023-03-05T00:52:38.564736814Z","UpdatedAt":"2023-03-02T00:52:38.564737714Z","DownloadedAt":"2023-03-02T21:18:48.663585772Z"}

Do you know why java-db is not updating automatically the same way as db?
Here is client side:

# Here you can see it's downloading java-db on the client because java-db wasn't updated on the server

 ./trivy image --server http://trivy.server.com:8080 zap2docker > test
2023-03-06T16:16:41.865+0100    INFO    Vulnerability scanning is enabled
2023-03-06T16:16:41.865+0100    INFO    Secret scanning is enabled
2023-03-06T16:16:41.865+0100    INFO    If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2023-03-06T16:16:41.865+0100    INFO    Please see also https://aquasecurity.github.io/trivy/v0.38/docs/secret/scanning/#recommendation for faster secret detection
2023-03-06T16:16:44.347+0100    INFO    JAR files found
2023-03-06T16:16:44.347+0100    INFO    Java DB Repository: ghcr.io/aquasecurity/trivy-java-db:1
2023-03-06T16:16:44.347+0100    INFO    Downloading the Java DB...
410.43 MiB / 410.43 MiB [---------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 39.86 MiB p/s 10s
2023-03-06T16:16:55.934+0100    INFO    The Java DB is cached for 3 days. If you want to update the database more frequently, the '--reset' flag clears the DB cache.
2023-03-06T16:16:55.934+0100    INFO    Analyzing JAR files takes a while...
2023-03-06T16:17:02.377+0100    INFO    Table result includes only package filenames. Use '--format json' option to get the full path to the package file.

ls -la .cache/trivy/
total 12
drwxr-xr-x 3 root root 4096 Mar  6 16:16 .
drwxr-xr-x 3 root root 4096 Mar  6 16:16 ..
drwxr-xr-x 2 root root 4096 Mar  6 16:16 java-db

# i run download java-db on server
docker exec -it e70e3e77b266 trivy image --download-java-db-only


# i scan on some other client and here you can see no java-db download on the client since it was updated on the server 

 ./trivy image --server http://trivy.server.com:8080 armdocker.rnd.ericsson.se/proj-ews/security/zap2docker > test
2023-03-06T16:19:18.654+0100    INFO    Vulnerability scanning is enabled
2023-03-06T16:19:18.654+0100    INFO    Secret scanning is enabled
2023-03-06T16:19:18.654+0100    INFO    If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2023-03-06T16:19:18.654+0100    INFO    Please see also https://aquasecurity.github.io/trivy/v0.38/docs/secret/scanning/#recommendation for faster secret detection
2023-03-06T16:19:19.671+0100    INFO    Table result includes only package filenames. Use '--format json' option to get the full path to the package file.
ls -la .cache/
total 12
drwxr-xr-x 3 root root 4096 Feb  9 22:48 .
drwx------ 5 root root 4096 Mar  6 16:19 ..

The point here is to make sure we always have updated db on the server so no client will ever download this db from ghcr.io/aquasecurity/trivy-java-db:1 because sometimes the download is stuck and it's failing our CI/CD.

[DmitriyLewen]

Hello @workerus
Thanks for your invetigation!

Currently Trivy parses files on client. Trivy-java-db is needed to parse jar files and is used on client.
As i said before - we have issue(#3560) to move trivy-java-db to server, but we currently don't have time to do this right now.
If you have time and desire - you can make changes and create new PR, we are always glad to new contributors!

i run download java-db on server
docker exec -it e70e3e77b266 trivy image --download-java-db-only

In this case Trivy download trivy-java-db, but doesn't use this. This logic needed to Trivy work in default(not server-client) mode.

[workerus]

@DmitriyLewen thank you for your support!

[DmitriyLewen]

i close this issue, feel free to reopen this issue, if you still have questions.