_______  _______                    _______  _______
(       )(  ___  )|\     /||\     /|(  ____ \(       )
| () () || (   ) |( \   / )| )   ( || (    \/| () () |
| || || || (___) | \ (_) / | (___) || (__    | || || |
| |(_)| ||  ___  |  \   /  |  ___  ||  __)   | |(_)| |
| |   | || (   ) |   ) (   | (   ) || (      | |   | |
| )   ( || )   ( |   | |   | )   ( || (____/\| )   ( |
|/     \||/     \|   \_/   |/     \|(_______/|/     \|

HTTP failure injection proxy for testing resilience.

What It Does

Mayhem sits between client and server, matches requests against rules, injects failures with configured probability. Basically tests if your application handles errors correctly.

WARNING

This tool breaks HTTP traffic deliberately:

• Drops connections
• Returns errors
• Adds latency
• Corrupts responses
• Throttles bandwidth

Do not:

• Run against production without approval and rollback plan
• Use on third-party APIs you don't control
• Deploy without monitoring
• Enable without understanding blast radius

Installation

Requires Go 1.22+

git clone <repository>
cd malice
make build

Binary: ./malice

Quick Start

Create config.yaml:

version: v1

rules:
  - name: api-errors
    matcher:
      paths: ["/api/*"]
    probability: 0.1
    action:
      type: error
      status_code: 503

Start proxy:

./malice start -f config.yaml -t http://localhost:3000 -l :8080

Traffic to :8080 proxies to localhost:3000 with failures injected per rules.

Configuration

Matchers

Rules match on:

matcher:
  methods: [GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS]
  paths: ["/api/*", "/users/*/profile"] # Wildcards supported
  headers:
    Content-Type: "application/json"
    X-API-Key: "test-*"
  query_params:
    format: "json"
  body:
    json_path: "$.amount"
    operator: ">"
    value: 100

All conditions are AND. First matching rule wins. Rules evaluated in order.

Actions

delay

Add latency:

action:
  type: delay
  duration: 500ms # Fixed
  # OR
  duration_min: 100ms # Random range
  duration_max: 2s

error

Return HTTP error:

action:
  type: error
  status_code: 503
  body: '{"error": "unavailable"}'
  headers:
    Content-Type: "application/json"

abort

Close connection:

action:
  type: abort
  timing: before_response # before_response | after_headers | mid_body

throttle

Limit bandwidth:

action:
  type: throttle
  bytes_per_second: 10240 # 10KB/s

corrupt

Damage response:

action:
  type: corrupt
  mode: truncate # truncate | flip_bits | invalid_json | random_bytes
  rate: 0.01 # For flip_bits/insert_garbage

timeout

Hold connection then close:

action:
  type: timeout
  duration: 30s

random

Pick random action:

action:
  type: random
  choices:
    - type: error
      status_code: 500
    - type: delay
      duration: 5s
    - type: abort

pass

Explicitly allow through (no failure):

action:
  type: pass

Probability

0.0 to 1.0:

• 1.0 = always inject
• 0.5 = 50% chance
• 0.1 = 10% chance
• 0.0 = never (same as enabled: false)

Commands

./malice validate -f config.yaml           # Validate config
./malice start -f config.yaml -t URL       # Start proxy
./malice start -f config.yaml -t URL -l :9000  # Custom listen port
./malice status                            # Show metrics (table)
./malice status -o json                    # Metrics as JSON
./malice status -o yaml                    # Metrics as YAML
./malice version                           # Show version

Metrics

Tracked:

• Requests (total, current, by method)
• Rules (loaded, enabled, matched)
• Injections (total, by rule, by action type, active, dropped)
• Delays (duration by rule)
• Errors (by status code)
• Aborts (by timing)
• Throttled bytes

View: ./malice status

Prometheus-compatible gauges and counters available via internal collector.

Logging

Structured JSON via zerolog.

export MALICE_LOG_LEVEL=debug    # trace | debug | info | warn | error
export MALICE_LOG_PRETTY=true    # Human-readable console output

All logs include:

• component - which package logged
• timestamp - RFC3339
• request_id - trace requests (when applicable)

Example:

{
  "level": "info",
  "component": "proxy",
  "request_id": "abc123",
  "method": "GET",
  "path": "/api/users",
  "matched_rule": "api-errors",
  "action": "error",
  "status_code": 503,
  "timestamp": "2025-01-01T12:00:00Z"
}

Examples

examples/simple.yaml - Minimal starter

examples/comprehensive.yaml - All action types, advanced matching

Testing

go test ./...              # All tests
go test -race ./...        # With race detector
make test                  # Via Makefile
make lint                  # Run linters

267 tests. Full coverage of config, matcher, metrics, runner, proxy.

Safety

No built-in guardrails. Does exactly what you configure.

Best practices:

1. Validate config before starting: ./malice validate -f config.yaml

2. Start with low probability (< 0.1)

3. Test in staging first

4. Monitor metrics during injection

5. Have rollback plan ready

6. Use health check pass-through:

   - name: health-pass
     matcher:
       paths: ["/health", "/ready"]
     probability: 1.0
     action:
       type: pass

Architecture

internal/
├── cli/       # Cobra command definitions
├── config/    # YAML loading, validation, schema
├── matcher/   # Request matching, probability evaluation
├── metrics/   # Prometheus-style collectors, snapshots
├── runner/    # Action executors (delay, error, abort, etc.)
├── proxy/     # HTTP reverse proxy, action coordination
├── log/       # Structured logging setup
└── version/   # Build version info

Request flow:

1. Client → Proxy (:8080)
2. Proxy evaluates rules in order
3. First match triggers probability check
4. If injected: execute action (delay, error, abort, etc.)
5. Proxy → Backend (if not aborted)
6. Backend → Client (possibly corrupted/throttled)

Advanced Patterns

Chaos for specific users

- name: test-user-chaos
  matcher:
    headers:
      X-User-ID: "test-*"
  probability: 0.5
  action:
    type: random
    choices:
      - type: error
        status_code: 500
      - type: delay
        duration: 10s

Simulate cascading failures

- name: dependency-timeout
  matcher:
    paths: ["/api/orders/*"]
  probability: 0.3
  action:
    type: delay
    duration: 30s

- name: circuit-breaker-open
  matcher:
    paths: ["/api/payments"]
  probability: 0.5
  action:
    type: error
    status_code: 503

Network partition simulation

- name: partition
  matcher:
    paths: ["/*"]
  probability: 0.2
  action:
    type: abort
    timing: before_response

Configuration Reference

See docs/config-schema-v1.md for complete schema.

Metrics Schema

See docs/metrics.md for metric definitions and export formats.