vulnerability-engine

Vulnerability Engine

Versioning

This project uses semantic versioning https://semver.org/. This process is automated by using Python Semantic Release. Commits need to use this format.

Requirements

• VMaaS (Vulnerability Metadata as a Service)

  • Provides core functionality for evaluating vulnerabilities
  • https://github.com/RedHatInsights/vmaas

• Insights Platform

  • Upload service, Inventory, Kafka message queue
  • Mocked Platform service is part of this repository (for development purposes)

Local setup

Install a local python environment with pre-commit:

poetry install --sync --no-root
poetry shell
pre-commit install

Local testing

Build images and start containers:

docker-compose up --build

Engine usage:

# Generate testing Insights archive
echo '{"package_list": ["kernel-3.10.0-862.el7.x86_64"], "repository_list": ["rhel-7-server-rpms"]}' | ./scripts/generate_insights_archive.py -o /tmp/insights-archive.tar.gz -

# Upload Insights archive to Platform mock
./scripts/3scale-mock -o 123456 curl -X POST -H "x-include-rules: false" -F "file=@/tmp/insights-archive.tar.gz" http://localhost:8100/api/v1/upload

# Check systems details
./scripts/3scale-mock -o 123456 curl -X GET http://localhost:8300/api/vulnerability/v1/systems

# Upload Insights archive to Platform mock and send message to Listener 10 times
./scripts/3scale-mock -a 123456 curl -X POST -F "file=@/tmp/insights-archive.tar.gz" -H "x-upload-multiplier: 10" http://localhost:8100/api/v1/upload

# Delete system
curl -X DELETE http://localhost:8100/api/v1/delete/be012439-26ae-456c-99a6-27b402331064

Database

Switch into database container and run database terminal:

docker exec -it vulnerability-engine-database bash -c "psql -d vulnerability"

Feature Flags

Feature flags are supported by Unleash. Connection to an Unleash server is done by a Clowder provided config.

Local development can be done by supplying bootstrapped feature flags via UNLEASH_BOOTSTRAP_FILE environment variable. UNLEASH_BOOTSTRAP_FILE should point to a JSON file conforming to Unleash's API /api/client/features.

Run tests

You can run all tests from scratch just after cloning repo using command:

docker-compose -f docker-compose.test.yml up --build --exit-code-from test

Or locally:

# install postgresql-devel postgresql-server
poetry install --sync --no-root
poetry shell
pytest -vvv tests/

Developing / Debugging

You can tune metrics using Prometheus and Grafana dev containers, see doc/metrics.md.

---

Structure

Detailed structure