Skip to content

2.0.0 #118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Dec 23, 2021
Merged

2.0.0 #118

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/dogfooding_suppressions.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<suppressions xmlns="https://jeremylong.github.io/DependencyCheck/dependency-suppression.1.3.xsd">
<!-- You can find examples in https://jeremylong.github.io/DependencyCheck/general/suppression.html -->
<suppress>
<filePath regex="true">.*\bh2-1\.4\.199\.jar</filePath>
<cve>CVE-2021-23463</cve>
</suppress>
</suppressions>
68 changes: 8 additions & 60 deletions .github/integration_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ cd "${BASH_SOURCE%/*}/.." || exit 1

PROJECT_DIR="$PWD"
CONFIG_FILE="$PROJECT_DIR/.github/nvd-config.json"
DOGFOODING_CONFIG_FILE="$PROJECT_DIR/.github/nvd-dogfooding-config.json"
SUCCESS_REGEX="[1-9][0-9] vulnerabilities detected\. Severity: "

if ! lein with-profile -user,-dev,+ci install; then
Expand All @@ -15,49 +16,9 @@ if ! clojure -Ttools install nvd-clojure/nvd-clojure '{:mvn/version "RELEASE"}'
exit 1
fi

cd "$PROJECT_DIR/plugin" || exit 1

if ! lein with-profile -user,-dev,+ci install; then
exit 1
fi

cd "$PROJECT_DIR/example" || exit 1

# 1.- Exercise Lein plugin

if lein with-profile -user nvd check > example-lein-output; then
echo "Should have failed with non-zero code!"
exit 1
fi

if ! grep --silent "$SUCCESS_REGEX" example-lein-output; then
echo "Should have found vulnerabilities!"
exit 1
fi

# 2.- Exercise Lein plugin, with :throw-if-check-unsuccessful? option

if lein with-profile -user,+nvd-throw-on-exit nvd check > example-lein-output 2>&1; then
echo "Should have failed with non-zero code!"
exit 1
fi

if ! grep --silent "$SUCCESS_REGEX" example-lein-output; then
echo "Should have found vulnerabilities!"
exit 1
fi

if ! grep --silent "Error encountered performing task 'nvd'" example-lein-output; then
echo "Should have thrown an exception!"
exit 1
fi

if ! grep --silent "clojure.lang.ExceptionInfo: nvd-clojure failed / found vulnerabilities" example-lein-output; then
echo "Should have thrown an exception with a specific message!"
exit 1
fi

# 3.- Exercise `main` program
# 1.- Exercise `main` program

example_classpath="$(lein with-profile -user,-dev,-test classpath)"

Expand All @@ -74,7 +35,7 @@ if ! grep --silent "$SUCCESS_REGEX" example-lein-output; then
exit 1
fi

# 4.- Exercise `tools.deps` integration
# 2.- Exercise `tools.deps` integration

cd "$PROJECT_DIR/example" || exit 1

Expand All @@ -83,7 +44,7 @@ example_classpath="$(clojure -Spath)"
# cd to the root dir, so that one runs `defproject nvd-clojure` which is the most clean and realistic way to run `main`:
cd "$PROJECT_DIR" || exit 1

if clojure -M -m nvd.task.check "$CONFIG_FILE" "$example_classpath" > example-lein-output; then
if clojure -J-Dclojure.main.report=stderr -M -m nvd.task.check "$CONFIG_FILE" "$example_classpath" > example-lein-output; then
echo "Should have failed with non-zero code!"
exit 1
fi
Expand All @@ -93,7 +54,7 @@ if ! grep --silent "$SUCCESS_REGEX" example-lein-output; then
exit 1
fi

# 5.- Exercise Clojure CLI Tools integration
# 3.- Exercise Clojure CLI Tools integration

cd "$PROJECT_DIR/example" || exit 1

Expand All @@ -102,7 +63,7 @@ example_classpath="$(clojure -Spath)"
# cd to $HOME, to demonstrate that the Tool does not depend on a deps.edn file:
cd || exit 1

if clojure -Tnvd nvd.task/check :classpath '"'"$example_classpath"'"' > example-lein-output; then
if clojure -J-Dclojure.main.report=stderr -Tnvd nvd.task/check :classpath '"'"$example_classpath"'"' > example-lein-output; then
echo "Should have failed with non-zero code!"
exit 1
fi
Expand All @@ -112,28 +73,15 @@ if ! grep --silent "$SUCCESS_REGEX" example-lein-output; then
exit 1
fi

# 6.- Dogfood the `nvd-clojure` project
# 4.- Dogfood the `nvd-clojure` project

cd "$PROJECT_DIR" || exit 1

own_classpath="$(lein with-profile -user,-dev,-test classpath)"

if ! lein with-profile -user,-dev,+ci run -m nvd.task.check "" "$own_classpath"; then
if ! lein with-profile -user,-dev,+ci,+skip-self-check run -m nvd.task.check "$DOGFOODING_CONFIG_FILE" "$own_classpath"; then
echo "nvd-clojure did not pass dogfooding!"
exit 1
fi

# 7.- Dogfood the `lein-nvd` project

cd "$PROJECT_DIR/plugin" || exit 1

plugin_classpath="$(lein with-profile -user,-dev,-test classpath)"

cd "$PROJECT_DIR" || exit 1

if ! lein with-profile -user,-dev,+ci run -m nvd.task.check "" "$plugin_classpath"; then
echo "lein-nvd did not pass dogfooding!"
exit 1
fi

exit 0
2 changes: 2 additions & 0 deletions .github/nvd-dogfooding-config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"delete-config?": false,
"nvd": {"suppression-file": ".github/dogfooding_suppressions.xml"}}
4 changes: 0 additions & 4 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,6 @@ jobs:
uses: actions/checkout@master
with:
ref: ${{ github.ref }}
- name: eclint
uses: snow-actions/eclint@v1.0.1
with:
args: 'check .* * src/**/*.clj test/**/*.clj'
- run: .github/lint.sh
- name: Install leiningen
uses: DeLaGuardo/setup-clojure@master
Expand Down
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
## Changes from 1.9.0 to 2.0.0

* [#113](https://github.com/rm-hull/nvd-clojure/issues/113): Remove all unsafe APIs.
* Please refer to the README for the recommended installation/usage patterns.
* [#117](https://github.com/rm-hull/nvd-clojure/issues/117): Detect when `nvd-clojure` is being used in a likely-incorrect way, and fail the program when that happens.
* Remove deprecated tasks, related to DB management.

## Changes from 1.8.0 to 1.9.0

#### New
Expand Down
154 changes: 63 additions & 91 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,13 @@
> _Formerly known as_ `lein-nvd`

[![Build Status](https://github.com/rm-hull/nvd-clojure/workflows/Continuous%20Integration/badge.svg)](https://github.com/rm-hull/nvd-clojure/actions?query=workflow%3A%22Continuous+Integration%22)
[![Coverage Status](https://coveralls.io/repos/rm-hull/nvd-clojure/badge.svg?branch=master)](https://coveralls.io/r/rm-hull/nvd-clojure?branch=master)
[![Dependencies Status](https://byob.yarr.is/dotemacs/actions-play/dependencies)](https://github.com/rm-hull/nvd-clojure/actions?query=workflow%3A%22dependencies%22)
[![Downloads](https://versions.deps.co/rm-hull/nvd-clojure/downloads.svg)](https://versions.deps.co/rm-hull/nvd-clojure)
[![Clojars Project](https://img.shields.io/clojars/v/nvd-clojure.svg)](https://clojars.org/nvd-clojure)
[![Maintenance](https://img.shields.io/maintenance/yes/2021.svg?maxAge=2592000)]()

[National Vulnerability Database](https://nvd.nist.gov/) dependency-checker
library (and plugin for Leiningen).
[National Vulnerability Database](https://nvd.nist.gov/) dependency checker
library.

When run in your project, all the JARs on the classpath
will be checked for known security vulnerabilities. `nvd-clojure` extracts project
Expand All @@ -22,49 +21,89 @@ dependencies and passes them to a library called [Dependency-Check](https://gith
> a given dependency. If found, it will generate a report linking to the
> associated CVE entries.

### Installation
### Installation and basic usage

> _Please see also:_ [Avoiding classpath interference](#avoiding-classpath-interference)

#### Leiningen

<details>

Please create a separate project consisting exclusively of `[nvd-clojure/nvd-clojure "2.0.0"]`. Said project can be located inside the targeted repo's Git repository.

Please do not add nvd-clojure as a dependency or plugin in the project.clj of the project to be analysed.

Then you can run, within this helper project:

```
lein with-profile -user run -m nvd.task.check "" "$(cd <YOUR_PROJECT>; lein with-profile -user,-dev classpath)"
```

An empty string is passed as the first argument, for backwards compatibility reasons. You can also pass a filename instead, denoting a .json file with extra options ([example](https://github.com/rm-hull/nvd-clojure/blob/master/.github/nvd-config.json)).

The `classpath` command should reflect a production-like classpath as closely as possible: it should not include dev/test tooling, plugins, etc.

If you are using a multi-modules solution (e.g. `lein-sub`, `lein-monolith`, `trapperkeeper`), you should ensure that each module is included in this classpath; else they will not be analysed.

</details>

#### Clojure CLI

To install in a given project, you can add `nvd-clojure/nvd-clojure {:mvn/version "1.9.0"}` to your deps.edn.
<details>

Please create a separate project consisting exclusively of `nvd-clojure/nvd-clojure {:mvn/version "2.0.0"}`. Said project can be located inside the targeted repo's Git repository.

Please do not add nvd-clojure as a dependency in the deps.edn of the project to be analysed.

> You can accomplish something similar with user-level aliases, or with the `:replace-deps` option.

Then you can run, within this helper project:

```
clojure -J-Dclojure.main.report=stderr -m nvd.task.check "" "$(cd <YOUR_PROJECT>; clojure -Spath -A:any:aliases)"
```

An empty string is passed as the first argument, for backwards compatibility reasons. You can also pass a filename instead, denoting a .json file with extra options ([example](https://github.com/rm-hull/nvd-clojure/blob/master/.github/nvd-config.json)).

The `-Spath` command should reflect a production-like classpath as closely as possible: it should not include dev/test tooling, etc.

If you are using a multi-modules solution (e.g. [Polylith](https://github.com/polyfy/polylith)), you should ensure that each module is included in this classpath; else they will not be analysed.

</details>

#### Clojure CLI Tool

<details>

If you have CLI version 1.10.3.933 or later, you can also install `nvd-clojure` as a "tool":

```bash
clojure -Ttools install nvd-clojure/nvd-clojure '{:mvn/version "RELEASE"}' :as nvd
```
and then you can run the tool like this:

Then you can run:

```bash
clojure -Tnvd nvd.task/check :classpath '"'"$(clojure -Spath -A:any:aliases)"'"'
clojure -J-Dclojure.main.report=stderr -Tnvd nvd.task/check :classpath '"'"$(clojure -Spath -A:any:aliases)"'"'
```

under `:aliases` in _~/.clojure/deps.edn_, or add it to `:aliases` in
the project local `deps.edn`, to look something like this:
You can optionally pass a `:config-filename`, denoting a .json file with extra options ([example](https://github.com/rm-hull/nvd-clojure/blob/master/.github/nvd-config.json)).

```clojure
:aliases {:nvd {:extra-deps {nvd-clojure/nvd-clojure {:mvn/version "1.9.0"}}
:main-opts ["-m" "nvd.task.check"]}}
```
The `-Spath` command should reflect a production-like classpath as closely as possible: it should not include dev/test tooling, etc.

#### Leiningen
If you are using a multi-modules solution (e.g. [Polylith](https://github.com/polyfy/polylith)), you should ensure that each module is included in this classpath; else they will not be analysed.

To install globally, add `[lein-nvd "1.9.0"]` into the `:plugins` vector of
your `:user` profile in _~/.lein/profiles.clj_, or on a per-project basis, add
to the profiles section of your _project.clj_.
</details>

## Usage
## Usage overview

Run `lein nvd check` or `clj -M:nvd` (if you've chosen the alias `:nvd`, like
above) in your project. The first time the plugin runs,it will download (and
Run the program as indicated in the previous section. The first time it runs, it will download (and
cache) various databases from https://nvd.nist.gov. Subsequent runs will
periodically check and update the local database, but the initial run could
therefore be quite slow - of the order of ten minutes or more, so give it time.

On completion, a summary table is output to the console, and a suite of reports
will be produced in the project's _./target/nvd/_ directory. If vulnerabilities
will be produced in the project's `./target/nvd/` directory. If vulnerabilities
are detected, then the check process will exit abnormally, thereby
causing any CI build environment to error. (This behaviour can be overriden by
setting a `:fail-threshold` in the project [configuration](#configuration-options)).
Expand Down Expand Up @@ -110,43 +149,6 @@ dependency relationships are:
dependencies, and suggest upgraded versions, and can optionally be configured
to update the project file.

## Other commands

Running the following command shows what sub-commands are available:

$ lein help nvd

Scans project dependencies, attempting to detect publicly disclosed
vulnerabilities contained within dependent JAR files. It does this by
determining if there is a Common Platform Enumeration (CPE) identifier
for a given dependency. On completion, a summary table is displayed on
the console (showing the status for each dependency), and detailed report
linking to the associated CVE entries.

This task should be invoked with one of three commands:

check - will optionally download the latest database update files,
and then run the analyze and report stages. Typically, if
the database has been updated recently, then the update
stage will be skipped.

purge - will remove the local database files. Subsequently running
the 'check' command will force downloading the files again,
which could take a long time.

update - will attempt to download the latest database updates, and
incorporate them into the local store. Usually not necessary,
as this is incorporated into the 'check' command.

Any text after the command are treated as arguments and are passed directly
directly to the command for further processing.

Arguments: ([command & args])

While `purge` and `update` are available, it is not normally required to use them,
and purging will cause a subsequent `check` or `update` to download the whole
database again.

## Configuration options

The default settings for `nvd-clojure` are usually sufficient for most projects, but
Expand Down Expand Up @@ -182,41 +184,11 @@ There are some specific settings below which are worthy of a few comments:

## Avoiding classpath interference

nvd-clojure has some Java dependencies, which in turn can have CVEs themselves, or in any case interfere with your project's dependency tree, that would be computed in absence of nvd-clojure.

For this reason, you might want to invoke `nvd.task.check`'s main function by passing a classpath string as an argument.

Said classpath string should try reflecting a _production's classpath_ as accurately as possible: it should not include dev/test tooling, plugins (like nvd-clojure or any other), etc.
nvd-clojure has some Java dependencies, which in turn can have CVEs themselves.

#### Lein example

```bash
lein run -m nvd.task.check "" "$(lein with-profile -user,-dev classpath)"
```

#### deps.edn example

```bash
clojure -m nvd.task.check "" "$(clojure -Spath)"
```

...in both cases, an empty string is passed as the first argument, for backwards compatibility reasons. You can also pass a filename instead, denoting a .json file with extra options ([example](https://github.com/rm-hull/nvd-clojure/blob/master/.github/nvd-config.json)).

For extra isolation, it is recommended that you invoke `nvd.task.check` from _outside_ your project - e.g. from an empty project, a git clone of this very repo, or from $HOME (assuming you have nvd-clojure as a dependency in your [user-wide Lein profile](https://github.com/technomancy/leiningen/blob/2586957f9d099ff11d50d312a6daf397c2a06fb1/doc/PROFILES.md)).

## Building locally

Build and install the core module, then do the same for the plugin:

$ lein test
$ lein install
$ cd plugin
$ lein test
$ lein install
$ cd ../example
$ lein nvd check
Likewise, a given project's dependencies can overlap and therefore affect nvd-clojure's, leading it to incorrect functioning.

A sample report is available for testing in the _example_ sub-directory.
For these reasons, it is strongly advised to follow the installation/usage instructions carefully.

## Attribution

Expand Down
3 changes: 1 addition & 2 deletions deps.edn
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,7 @@
org.slf4j/slf4j-simple {:mvn/version "2.0.0-alpha1"}
org.owasp/dependency-check-core {:mvn/version "6.5.1"}
rm-hull/table {:mvn/version "0.7.1"}
trptcolin/versioneer {:mvn/version "0.2.0"}
org.clojure/tools.deps.alpha {:mvn/version "0.12.1090"}}
trptcolin/versioneer {:mvn/version "0.2.0"}}
:mvn/repos {"central" {:url "https://repo1.maven.org/maven2/"}
"clojars" {:url "https://repo.clojars.org/"}}
:tools/usage {:ns-default nvd.task}
Expand Down
Loading