Skip to content

Latest commit

 

History

History
242 lines (177 loc) · 11.8 KB

README.md

File metadata and controls

242 lines (177 loc) · 11.8 KB

PRevent

Overview

A self-hosted GitHub app that listens for pull request events, scans them for malicious code, and comments detections directly on the pull request.

Typically, security scans are run by workflow files. However, files can be modified, and when dealing with source modification attacks, should be avoided. A GitHub app approach addresses this gap, ensuring the scan is not bypassed. The app's logic can be leveraged to run any scan. All you need is to add a scanner method to the scan logic.

Malicious Code Detection

Currently, PRevent detects dynamic code execution and obfuscation, patterns found in nearly 100% of malicious code attacks reported to this day, while being rare in benign code, making the scan very effective. It uses Apiiro's malicious-code-ruleset for Semgrep, alongside additional Python-based detectors. Only rules and detectors with low false-positive rates are included.

Extra Capabilities

Optional features:

  • Select repositories and branches to include or exclude from the scan (default: all).
  • Trigger code reviews from designated reviewers.
  • Block merging until a reviewer's approval is granted or the scan passes.
  • Run only the rules and detectors with the lowest false-positive rates.

Deployment:

  • Supports containerization.
  • Non-containerized deployment is fully automated with an interactive setup script.
  • To manage GitHub key (required for any GitHub app), multiple secret managers are supported:
    • HashiCorp Vault
    • AWS Secrets Manager
    • Azure Key Vault
    • Google Cloud Secret Manager
    • Local HashiCorp Vault (for development and testing)

Supported languages

  • Bash
  • Clojure
  • C#
  • Dart
  • Go
  • Java
  • JavaScript
  • TypeScript
  • Lua
  • PHP
  • Python
  • Ruby
  • Rust
  • Scala

Setup

Deploying PRevent involves three parts, typically completed in 5 minutes to an hour, depending on your setup and familiarity:

  1. Configure an existing secret manager or create a new one.
  2. Create a GitHub app within your GitHub organization or account.
  3. Deploy the application to a server.

Non-Containerized Setup

Parts 1 and 2 are handled during the interactive setup process in step 3:

  1. Clone this repository: 
    git clone https://github.com/apiiro/prevent.git
cd prevent
  2. Install dependencies by either: 
    poetry install  

pip install -r requirements.txt
  3. Go through the setup process: 
    python3 -m setup.setup
  4. Start the server: 
    gunicorn --bind 0.0.0.0:8080 src.app:app

Containerized Setup

1. Secret Manager

The application communicates with GitHub via authenticated requests, which require three sensitive parameters:

  • Private Key (GITHUB_APP_PRIVATE_KEY)
  • App ID (GITHUB_APP_INTEGRATION_ID)
  • Webhook Secret (WEBHOOK_SECRET)

To minimize security risks, these parameters should be stored in a secret manager with minimal permissions. Optionally, you may store additional sensitive details such as:

  • Repositories and branches (BRANCHES_INCLUDE, BRANCHES_EXCLUDE)
  • Accounts and teams (SECURITY_REVIEWERS)

The application handles all parameters exclusively through the secret manager (see supported managers below). In containerized deployments, this applies also for insensitive parameters (all are optional), for centralization and simplicity. Upon initialization, these parameters are written to src/settings.py to avoid repeated fetching during runtime. These include:

  • Block PRs (BLOCK_PR)
  • False Positive Strictness (FP_STRICT)
  • JWT Expiry (JWT_EXPIRY_SECONDS)
  • Webhook Port (WEBHOOK_PORT)

Secret Manager Setup Instructions

First, set SECRET_MANAGER in your secret manager to either: vault, aws, azure, gcloud, or local.

Dedicate a section in your secret manager for this app, separated from the rest. Create an app role with minimal permissions, to access the dedicated section only. If you are not sure how, try the following instructions:

python3 setup/secret_managers/print_instructions.py SECRET_MANAGER

Permissions required to operate the role:

Permission Vault AWS Azure GCloud
read read secretsmanager:GetSecretValue KeyVaultSecret:Get secretmanager.secrets.get
write create, update secretsmanager:PutSecretValue KeyVaultSecret:Set secretmanager.secrets.add
scope path = "prevent-app/*" resource = "prevent-app/*" secret = "prevent-app/*" secret = "prevent-app/*"

2. GitHub App

  1. Go to https://github.com/settings/apps to create a new GitHub App.
  2. Set metadata:
  3. Set the webhook URL: the address where the app will listen. Endpoint: /webhook. Examples:
  4. Under the webhook URL field, set the secret field in order to process only requests originating from GitHub. You can run python -c 'import secrets; print(secrets.token_hex(32))' to generate one. Then, store it in your secrets manager as WEBHOOK_SECRET.
  5. Set required permissions:
Parent Permission Action Reason
Repository Pull requests Read and Write Read PR, write comments (if enabled: trigger reviews)
Repository Commit statuses Read and write Monitor scan-results by setting commits-statuses
Repository Contents Read-only Get full files, can't build AST from diff
  1. Set optional permissions:
Parent Permission Action Reason
Organization Members Read-only Trigger reviews
Repository Administration Read and write Manage branch protection

  1. Subscribe to the following events:

    • Pull request
    • Pull request review

  2. Click "Create GitHub App". Copy the App ID and store it in your secret manager as GITHUB_APP_INTEGRATION_ID.

  3. Generate a private key, store it in your secret manager as GITHUB_APP_PRIVATE_KEY, and make sure to delete the file.

3. Deployment

Optional Parameters

  • PR_BLOCK: To block merging until either a reviewer approves the pull request or the scan passes, set it to True in your secret manager.

  • SECURITY_REVIEWERS: To trigger code reviews upon detections, configure it in your secret manager with a Python list of reviewer accounts or teams (e.g., ['account1', 'account2', 'team:appsec']). Ensure you run json.dumps(security_reviewers) or an equivalent method beforehand.

  • INCLUDE_BRANCHES or EXCLUDE_BRANCHES: To include or exclude specific repos and branches for monitoring, set either in your secret manager with a Python dictionary. Use {'repo1': 'all'} to include or exclude all repo's branches, or specify a list of branches (e.g., {'repo1': ['main', 'branch2'], 'repo2': 'all'}). Ensure you run json.dumps(security_reviewers) or an equivalent method beforehand. By default, all repositories and branches are monitored.

  • FP_STRICT: To minimize false positives by running only ERROR severity rules and detectors (primarily a small subset of obfuscation detection), set it to True in your secret manager.

Deployment

TODO: Add a CLEAR explanation on how to securely pass these to the container. Also, clarify and expand on anything else in this section that deserves it.

Credentials required to operate your dedicated app role:

Vault AWS Azure GCloud
VAULT_ADDR AWS_ACCESS_KEY_ID AZURE_CLIENT_ID GOOGLE_APPLICATION_CREDENTIALS_JSON
VAULT_TOKEN AWS_SECRET_ACCESS_KEY AZURE_CLIENT_SECRET GOOGLE_CLOUD_PROJECT
AWS_SESSION_TOKEN (optional) AZURE_TENANT_ID (optional) GOOGLE_CLOUD_REGION (optional)
GOOGLE_API_KEY (optional)
  1. Build the app using the provided Dockerfile:
docker buildx build -t prevent .
  1. Push the image to your container registry (e.g. GCR):
PREVENT_TAG=1.0
docker buildx build \
  --platform linux/arm64/v8,linux/amd64 \
  --push --pull \
  -t us-docker.pkg.dev/user/public-images/prevent:$PREVENT_TAG \
  .
  1. Run the container:
PREVENT_TAG=1.0
docker run --rm -it us-docker.pkg.dev/user/public-images/prevent:$PREVENT_TAG
  1. Access the container:
docker run --rm -it --entrypoint /bin/sh us-docker.pkg.dev/user/public-images/prevent:$PREVENT_TAG

Configuration Parameters Summary

Parameter Name Purpose Source Required Type Default Example
secret manager SECRET_MANAGER SM to use (cli client, Python package, calls) user yes str vault aws
private key GITHUB_APP_PRIVATE_KEY Authenticates the app with GitHub GitHub yes str - -----BEGIN RSA...
app ID GITHUB_APP_INTEGRATION_ID Authenticates the app with GitHub GitHub yes str - 1234567
webhook secret WEBHOOK_SECRET Validates requests source (>32 random characters) GitHub yes str - 039e362cd52...
included branches BRANCHES_INCLUDE Repos and branches to scan (all by default) user no dict[str, list | str] {} {'r1': ['b1', 'b2']}
exclude branches BRANCHES_EXCLUDE Repos and branches to not scan user no dict[str, list | str] {} {'r': 'all'}
security reviewers SECURITY_REVIEWERS GitHub accounts and teams to review detections user no list [] ['jdoe', 'team:sec']
block merging BLOCK_PR Block merging in pull requests with detections user no bool False True
minimize FP FP_STRICT Run only ERROR severity rules, exclude WARNING user no bool False True
webhook port WEBHOOK_PORT The port on which the app listens user no int 8080 8443
JWT expiry time JWT_EXPIRY_SECONDS Limit the app's GitHub auth token TTL user no int 120 60

Contributing

Contributions are welcome through pull requests or issues.

License

This repository is licensed under the MIT License.

For more information: https://apiiro.com/blog/prevent-malicious-code