Getting 403 or 404 exception with NVD API key #6816

1-mrudulaahire · 2024-07-08T10:07:17Z

Hello, All!! After jeremylong suggetion moved to 10.0.1. But using this with NVD API key getting 403/404 exception. Though using api key came through email only. How to get know that my API activated or not?

jeremylong · 2024-07-08T11:27:54Z

See https://github.com/jeremylong/Open-Vulnerability-Project/tree/main/vulnz#api-key-is-used-and-a-403-or-404-error-occurs

1-mrudulaahire · 2024-07-08T12:13:58Z

@jeremylong Didn't get JSON with curl commad, tried commad from CMD. but already requested NVD API key three times. Getting exception every time.

lgolubenkobit · 2024-07-08T21:21:56Z

See https://github.com/jeremylong/Open-Vulnerability-Project/tree/main/vulnz#api-key-is-used-and-a-403-or-404-error-occurs

We are having this error when update using apiKey. I've tried doing curl from cmd and works ok.
On pom we have

                <nvdApiDelay>16000</nvdApiDelay>
                <nvdValidForHours>8</nvdValidForHours>
                <connectionTimeout>120000</connectionTimeout>

adam-siklosi · 2024-07-09T05:41:51Z

Same here, just checked that I have a valid api key. It started happening this night. Yesterday it was all good, didn't change anything in my setup. Version 10.0.1 cli / gradle plugin, default settings apart from the credentials.

1-mrudulaahire · 2024-07-09T05:49:38Z

See https://github.com/jeremylong/Open-Vulnerability-Project/tree/main/vulnz#api-key-is-used-and-a-403-or-404-error-occurs

We are having this error when update using apiKey. I've tried doing curl from cmd and works ok. On pom we have
                <nvdApiDelay>16000</nvdApiDelay>
                <nvdValidForHours>8</nvdValidForHours>
                <connectionTimeout>120000</connectionTimeout>

Hi, @lgolubenkobit. Did we have any log or action anything. That's shows our NVD API key is valid? It would be really helpful

mdzt-axi · 2024-07-09T06:13:53Z

I started getting this error last night too.

MidasBOC · 2024-07-09T06:27:00Z

We've experienced the same problem. We found this information in the log:

2024-07-09 08:15:32,287 io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient:359
DEBUG - Response: <html><body><h1>403 Forbidden</h1>
Request forbidden by administrative rules.
</body></html>

adam-siklosi · 2024-07-09T06:50:11Z

I think the issue is this: https://github.com/jeremylong/DependencyCheck?tab=readme-ov-file#mandatory-upgrade-notice
Using 10.0.2 locally seems to solved my problem.

Edit: just updated to 10.0.2 in our repositories and now all work properly.

aggelos-gamvrinos-sympower · 2024-07-09T07:15:37Z

Having the same issue with the gradle plugin. Checked the Nvd api key and it is valid. Upgrading the plugin to 10.0.2 didn't solve it

MidasBOC · 2024-07-09T07:54:05Z

I think the issue is this: https://github.com/jeremylong/DependencyCheck?tab=readme-ov-file#mandatory-upgrade-notice Using 10.0.2 locally seems to solved my problem.

Edit: just updated to 10.0.2 in our repositories and now all work properly.

Bumping the version worked for us. Thank you very much.

jeremylong · 2024-07-09T10:31:36Z

@aggelos-gamvrinos-sympower did you validate that the API key is still valid? See the documentation on how to validate the key is correct.

somera · 2024-07-09T10:34:33Z

Works fine with 10.0.1 and 10.0.2 for mee.

aggelos-gamvrinos-sympower · 2024-07-09T10:50:42Z

Edit: Sorry ended up working with 10.0.2. We're using libs and we hadn't properly bumped the version

david-pulkowski · 2024-07-09T12:50:24Z

Just making a note.

Our API key is valid.
We were also getting the 403 error in our Jenkins CI/CD nightly job after our jenkins plugin brought in the latest OWASP version, 10.0.2.

Troubleshooting: if I download the zip from this project and run it manually on the server, (running the same command as we have in our job, where it fails) the command is successful and updates the DB properly.

So for us, I think its something with how the jenkins plugin caches old versions of the plugin?

I will have to let the job rerun in a few hours, or let the nightly job run again.

jeremylong · 2024-07-09T12:52:53Z

As I just found out - it might be how the apiKey is being provided. I'll release an update soon that will improve the error reporting around the API key.

nico-mcalley · 2024-07-09T15:09:24Z

I updated org.owasp:dependency-check-gradle from 10.0.0 -> 10.0.2 and it worked again.

chadlwilson · 2024-07-09T15:12:51Z

Yes, as noted as mandatory at #6817 :-)

phs-glenfordham · 2024-07-10T03:45:17Z

hi @jeremylong
eagerly awaiting that API key update as we're still getting that generic error after upgrading to 10.0.2.

also, is anyone else getting a 400 error when using the CURL command provided here? even if I remove the API key entirely I get a 400 error, seems like the request is malformed?
edit: looks like Windows CURL through the command prompt doesn't like the command, but it works fine in Bash, interesting!

C:\Users\glenfordham>curl -H "Accept: application/json" -H "apiKey: redacted" -v https://services.nvd.nist.gov/rest/json/cves/2.0\?cpeName\=cpe:2.3:o:microsoft:windows_10:1607:\*:\*:\*:\*:\*:\*:\*
* Host services.nvd.nist.gov:443 was resolved.
* IPv6: (none)
* IPv4: 54.85.30.225
*   Trying 54.85.30.225:443...
* Connected to services.nvd.nist.gov (54.85.30.225) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* ALPN: server accepted http/1.1
* using HTTP/1.x
> GET /rest/json/cves/2.0\?cpeName\=cpe:2.3:o:microsoft:windows_10:1607:\*:\*:\*:\*:\*:\*:\* HTTP/1.1
> Host: services.nvd.nist.gov
> User-Agent: curl/8.7.1
> Accept: application/json
> apiKey: redacted
>
* Request completely sent off
* schannel: remote party requests renegotiation
* schannel: renegotiating SSL/TLS connection
* schannel: SSL/TLS connection renegotiated
* schannel: remote party requests renegotiation
* schannel: renegotiating SSL/TLS connection
* schannel: SSL/TLS connection renegotiated
< HTTP/1.1 400 Bad Request
< x-frame-options: SAMEORIGIN
< access-control-allow-origin: *
< access-control-allow-headers: accept, apiKey, content-type, origin, x-requested-with
< access-control-allow-methods: GET, HEAD, OPTIONS
< access-control-allow-credentials: false
< date: Wed, 10 Jul 2024 03:43:12 GMT
< content-length: 0
< apikey: Yes
< strict-transport-security: max-age=31536000
<
* Connection #0 to host services.nvd.nist.gov left intact

chadlwilson · 2024-07-10T04:18:53Z

@phs-glenfordham I believe it usually means there is a problem with your API key, or it is getting corrupted/lost somehow between how you supply it and the library at the other end. The change really just improves the logging so you can tell generally what was supplied on the API call, so you can figure out if you supplied it correctly to the CLI, Maven, Gradle plugin etc.

As for your curl issue, likely the way the URL is being escaped isn't appropriate for Windows/CMD. With the exact same command on MacOS I get 404 message: Invalid apiKey as expected.

You'll probably need to quote the URL properly, not rely on backslash escaping:

curl -H "Accept: application/json" -H "apiKey: redacted" -v "https://services.nvd.nist.gov/rest/json/cves/2.0?cpeName=cpe:2.3:o:microsoft:windows_10:1607:*:*:*:*:*:*:*"

phs-glenfordham · 2024-07-10T04:32:16Z

Thanks for the quick response Jeremy, and apologies for hijacking the thread.
Yes, it makes sense that cmd is messing with the input, thanks for the thought.

The peace of mind that the key is working was helpful, and we may have solved our issue with the 403/404 error when using the gradle plugin now. The new release will be useful if we ever need to implement it for another project though, still keen :)

chadlwilson · 2024-07-10T04:56:59Z

@phs-glenfordham Would you mind sharing what your issue was getting the API key through the Gradle Plugin correctly? Perhaps we can do something to give faster feedback within the Gradle plugin itself that something looks wrong in some cases. I commonly see problems when people are using Gradle project properties or system properties and trying to supply them from outside, and issues with multi-project Gradle setups.

zsbee · 2024-07-10T05:27:00Z

FYI, we had the same issue, and we have 2 api keys to NVD and both reported valid jsons. Even tried running it without an NVD API Key (that warns us that it will be slow) and that was also throwing errors.

upgrading to 10.0.2 from 10.0.1 fixed it for us.

chadlwilson · 2024-07-10T05:49:22Z

Yes, the upgrade to 10.0.2+ is mandatory and the source of most people's problems as NVD are blocking earlier (or unidentified) versions of ODC due to mitigating retry DDoS (mainly from 9.x versions).

But some folks are also having to supply API keys for the first time, as they are moving from 8.x to using the API, OR trying to reduce the chance of getting throttled given the load the API is under. Some of those are probably having issues supplying their keys properly through the plugins/CLI to the library.

Curl will tell you your API key is valid, but not that it is being picked up by ODC correctly.

phs-glenfordham · 2024-07-10T06:03:31Z

@phs-glenfordham Would you mind sharing what your issue was getting the API key through the Gradle Plugin correctly? Perhaps we can do something to give faster feedback within the Gradle plugin itself that something looks wrong in some cases. I commonly see problems when people are using Gradle project properties or system properties and trying to supply them from outside, and issues with multi-project Gradle setups.

we were using a custom environment variable to load it into the gradle plugin, something like this

dependencyCheck {
    nvd {
        apiKey = System.getenv("NVD_API_KEY")
    }
}

The issue was simply that the environment variable value we needed to use hadn't refreshed properly yet, because we had re-requested one several times trying to make sense of the issue before upgrading. The issue we had with curl didn't help, as then we thought it was just a general issue.
I think it would certainly help if the contents of the NVD request could be optionally dumped to the console, especially if gradle is running in debug mode - would've realised our stupidity sooner :)

chadlwilson · 2024-07-10T06:08:37Z

Here's my summary given the current load. Hope it helps.

>= 9.x, <= 10.0.1: Will get 403/404 due to NVD rejecting old clients Please Read: Mandatory Upgrade to 10.0.2 or later #6817
1. Upgrade to 10.0.2.
>= 10.0.2 Without API key: Quite likely to be getting 503s due to load per https://services.nvd.nist.gov/ endopint is giving 503 Service Unavailable #6758
1. Consider getting an API key. :-)
>= 10.0.2 With valid API key: Should be working, but
- If getting 503: The retries should get you through eventually as load should be improving.
  1. But you might want to look at https://jeremylong.github.io/DependencyCheck/data/cachenvd.html or https://jeremylong.github.io/DependencyCheck/data/cacheh2.html to reduce your dependency on the NVD by reducing # of calls.
  2. Consider adjusting the ODC retry delay to be longer.
- if getting 403/404:
  1. Double check your API key is valid using curl or similar.
  2. Double check the same API key is going through correctly from your plugin/CLI. The logging @jeremylong is adding may help improve this, but if using Gradle you can try logging it temporarily within Gradle to make sure it is being read correctly from your environment.
    - If you have been re-generating API keys, check you are using the correct one. Old keys are invalidated when you regenerate from the same email address.
  3. Turn on Maven debug logging, Gradle --info logging, or gradle --stacktrace and see if there is some other connectivity issue to the NVD API other than a 403/404/503. (especially if you have recently moved from ODC 8.x)

chadlwilson · 2024-07-10T06:10:47Z

The issue was simply that the environment variable value we needed to use hadn't refreshed properly yet, because we had re-requested one several times trying to make sense of the issue before upgrading.

Ahh OK, yes, the invalidation of the old keys when re-generating new keys is a real pain, as there is no distinction between the two types of failures on NVD and if working in a shared environment where multiple people have access to regenerate the key you can end up in a really confusing mess. Let me add that to my summary.

paksydavid · 2024-07-10T06:46:13Z

On our case upgrading to 10.0.2 conmpletelysolved this issue. Many thanks @jeremylong 🙏

1-mrudulaahire · 2024-07-10T08:56:52Z

@phs-glenfordham I believe it usually means there is a problem with your API key, or it is getting corrupted/lost somehow between how you supply it and the library at the other end. The change really just improves the logging so you can tell generally what was supplied on the API call, so you can figure out if you supplied it correctly to the CLI, Maven, Gradle plugin etc.

As for your curl issue, likely the way the URL is being escaped isn't appropriate for Windows/CMD. With the exact same command on MacOS I get 404 message: Invalid apiKey as expected.

You'll probably need to quote the URL properly, not rely on backslash escaping:
curl -H "Accept: application/json" -H "apiKey: redacted" -v "https://services.nvd.nist.gov/rest/json/cves/2.0?cpeName=cpe:2.3:o:microsoft:windows_10:1607:*:*:*:*:*:*:*"

Thanks for curl commad it's help in knowing about api key validation. It always showing as Invalid. Do I need to wait for some to get activated or what once i get mail to check key validation

chadlwilson · 2024-07-10T09:18:08Z

@1-mrudulaahire If it has been some time since you activated (hours), I would try again with a new key. Be careful not to double-click or refresh pages when you activate the key, request new keys etc; or you can end up overwriting your own key and getting confused about which one is valid.

Other than that we can't really help you as this is the American NIST NVD's process/system and I don't believe they publish openly how long it takes for an activated key to be usable within the systems (might be immediate? might not?)

This is all documented at https://nvd.nist.gov/developers/start-here

Each API Key is associated with a single email address. If an email address is used to request an additional API key, clicking the single-use hyperlink will invalidate the key previously associated with that email address. The key will not be invalidated if the email address is used to request another key, but the hyperlink is not opened. There is no process for retrieving a forgotten key or confirming whether a key has been requested or activated by any email address.

1-mrudulaahire closed this as completed Jul 8, 2024

1-mrudulaahire reopened this Jul 8, 2024

chadlwilson mentioned this issue Jul 9, 2024

Getting 403 or 404 exception with NVD API key #6818

Closed

gicig mentioned this issue Jul 9, 2024

Upgrade dependency-check-maven Netcentric/accesscontroltool#740

Closed

chadlwilson mentioned this issue Jul 9, 2024

https://services.nvd.nist.gov/ endopint is giving 503 Service Unavailable #6758

Open

jeremylong added the question label Jul 9, 2024

chadlwilson mentioned this issue Jul 10, 2024

Please Read: Mandatory Upgrade to 10.0.2 or later #6817

Open

jeremylong closed this as completed Jul 29, 2024

ArvinEsmailzadeh mentioned this issue Aug 20, 2024

As a system admin, I want dependency (library, package, image, configuration) updates for security and bug fixes for WRES 6.25 NOAA-OWP/wres#67

Closed

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Getting 403 or 404 exception with NVD API key #6816

Getting 403 or 404 exception with NVD API key #6816

1-mrudulaahire commented Jul 8, 2024 •

edited

Loading

jeremylong commented Jul 8, 2024

1-mrudulaahire commented Jul 8, 2024 •

edited

Loading

lgolubenkobit commented Jul 8, 2024

adam-siklosi commented Jul 9, 2024

1-mrudulaahire commented Jul 9, 2024 •

edited

Loading

mdzt-axi commented Jul 9, 2024

MidasBOC commented Jul 9, 2024

adam-siklosi commented Jul 9, 2024 •

edited

Loading

aggelos-gamvrinos-sympower commented Jul 9, 2024 •

edited

Loading

MidasBOC commented Jul 9, 2024

jeremylong commented Jul 9, 2024

somera commented Jul 9, 2024

aggelos-gamvrinos-sympower commented Jul 9, 2024 •

edited

Loading

david-pulkowski commented Jul 9, 2024

jeremylong commented Jul 9, 2024

nico-mcalley commented Jul 9, 2024

chadlwilson commented Jul 9, 2024

phs-glenfordham commented Jul 10, 2024 •

edited

Loading

chadlwilson commented Jul 10, 2024

phs-glenfordham commented Jul 10, 2024

chadlwilson commented Jul 10, 2024

zsbee commented Jul 10, 2024

chadlwilson commented Jul 10, 2024 •

edited

Loading

phs-glenfordham commented Jul 10, 2024 •

edited

Loading

chadlwilson commented Jul 10, 2024 •

edited

Loading

chadlwilson commented Jul 10, 2024

paksydavid commented Jul 10, 2024

1-mrudulaahire commented Jul 10, 2024 •

edited

Loading

chadlwilson commented Jul 10, 2024

Getting 403 or 404 exception with NVD API key #6816

Getting 403 or 404 exception with NVD API key #6816

Comments

1-mrudulaahire commented Jul 8, 2024 • edited Loading

jeremylong commented Jul 8, 2024

1-mrudulaahire commented Jul 8, 2024 • edited Loading

lgolubenkobit commented Jul 8, 2024

adam-siklosi commented Jul 9, 2024

1-mrudulaahire commented Jul 9, 2024 • edited Loading

mdzt-axi commented Jul 9, 2024

MidasBOC commented Jul 9, 2024

adam-siklosi commented Jul 9, 2024 • edited Loading

aggelos-gamvrinos-sympower commented Jul 9, 2024 • edited Loading

MidasBOC commented Jul 9, 2024

jeremylong commented Jul 9, 2024

somera commented Jul 9, 2024

aggelos-gamvrinos-sympower commented Jul 9, 2024 • edited Loading

david-pulkowski commented Jul 9, 2024

jeremylong commented Jul 9, 2024

nico-mcalley commented Jul 9, 2024

chadlwilson commented Jul 9, 2024

phs-glenfordham commented Jul 10, 2024 • edited Loading

chadlwilson commented Jul 10, 2024

phs-glenfordham commented Jul 10, 2024

chadlwilson commented Jul 10, 2024

zsbee commented Jul 10, 2024

chadlwilson commented Jul 10, 2024 • edited Loading

phs-glenfordham commented Jul 10, 2024 • edited Loading

chadlwilson commented Jul 10, 2024 • edited Loading

chadlwilson commented Jul 10, 2024

paksydavid commented Jul 10, 2024

1-mrudulaahire commented Jul 10, 2024 • edited Loading

chadlwilson commented Jul 10, 2024

1-mrudulaahire commented Jul 8, 2024 •

edited

Loading

1-mrudulaahire commented Jul 8, 2024 •

edited

Loading

1-mrudulaahire commented Jul 9, 2024 •

edited

Loading

adam-siklosi commented Jul 9, 2024 •

edited

Loading

aggelos-gamvrinos-sympower commented Jul 9, 2024 •

edited

Loading

aggelos-gamvrinos-sympower commented Jul 9, 2024 •

edited

Loading

phs-glenfordham commented Jul 10, 2024 •

edited

Loading

chadlwilson commented Jul 10, 2024 •

edited

Loading

phs-glenfordham commented Jul 10, 2024 •

edited

Loading

chadlwilson commented Jul 10, 2024 •

edited

Loading

1-mrudulaahire commented Jul 10, 2024 •

edited

Loading