# Docker Image Security in DevSecOps Pipelines Example

This repository is part of a DevSecOps lab. Here is a Python example with a simple server. This repository explores various security issues with containers and Docker images. Each version aims to improve security by addressing identified vulnerabilities.

## Table of Contents

- [Vulnerable App](#vulnerable-app)
  - [Overview](#overview)
  - [Issues](#issues)
  - [How to Run](#how-to-run)
  - [Application Screenshots](#application-screenshots)
  - [Docker Image Analysis](#docker-image-analysis)
  - [Automated Security Scanning](#automated-security-scanning)
  - [Security Findings](#security-findings)
- [Fix #1: Update Base Image to Latest Version](#fix-1-update-base-image-to-latest-version)
  - [Description](#description)
  - [Changes Made](#changes-made)
  - [Build and Run](#build-and-run)
  - [Security Improvements](#security-improvements)
- [Fix #2: Comprehensive Security Hardening](#fix-2-comprehensive-security-hardening)
  - [Fix #2 Description](#fix-2-description)
  - [Fix #2 Changes Made](#fix-2-changes-made)
  - [Fix #2 Build and Run](#fix-2-build-and-run)
  - [Fix #2 Application Screenshots](#fix-2-application-screenshots)
  - [Security Analysis](#security-analysis)
  - [Secret Detection Limitation](#secret-detection-limitation)
- [Fix #3: Kubernetes Secret Management](#fix-3-kubernetes-secret-management)
  - [Fix #3: Description](#fix-3-description)
  - [Fix #3: Changes Made](#fix-3-changes-made)
  - [Fix #3: Kubernetes Resources](#fix-3-kubernetes-resources)
  - [Deploy to Kubernetes](#deploy-to-kubernetes)
  - [Fix #3 Application Screenshots](#fix-3-application-screenshots)
  - [Security Verification](#security-verification)
- [Runtime Security with Falco](#runtime-security-with-falco)
  - [Falco Description](#falco-description)
  - [Falco Setup](#falco-setup)
  - [Attack Simulation](#attack-simulation)
  - [Security Event Detection](#security-event-detection)

## Vulnerable App

### Overview

The `vulnerable-app` is a simple FastAPI application designed to demonstrate common security issues in Dockerized applications. It is intentionally insecure and should not be used in production environments.

### Issues

1. **Hardcoded Secret**: The application contains a hardcoded API secret, which is a security risk.

2. **Dockerfile Vulnerabilities**:
   - The application runs as `root`, increasing the attack surface.
   - The `EXPOSE` instruction unnecessarily exposes the port.

3. **No Environment Variable Usage**: Secrets and configurations are not externalized, making the application less secure.

### How to Run

To run the application, use the provided `Dockerfile`:

1. Build the Docker image:

   ```bash
   docker build -t vulnerable-app -f vulnerable-app/Dockerfile vulnerable-app
   ```

2. Run the Docker container:

   ```bash
   docker run -p 8000:8000 vulnerable-app
   ```

3. Access the application:
   - Open your browser and navigate to `http://localhost:8000` to see the welcome message.
   - Navigate to `http://localhost:8000/api/secret` to view the hardcoded secret (for demonstration purposes).

### Application Screenshots

- Application Running: ![Application Running](screenshots/vulnerable-app/image1.png)
- Application Home: ![Application Home](screenshots/vulnerable-app/image2.png)
- API Secret Endpoint: ![API Secret Endpoint](screenshots/vulnerable-app/image3.png)

### Docker Image Analysis

Analyze the Docker image for vulnerabilities using Trivy:

```bash
trivy image vulnerable-app:latest
```

This will scan the image and report known vulnerabilities in the base image and installed dependencies.

### Automated Security Scanning

The `scan.sh` script automates comprehensive security analysis of Docker images using Trivy. It generates three separate reports:

- **vulnerabilities-critical-high.txt**: Critical and high-severity vulnerabilities with available fixes
- **secrets.txt**: Hardcoded secrets and sensitive data
- **misconfigurations.txt**: Dockerfile and container configuration issues

- Script After Running: ![Script After Running](screenshots/trivy-scan/v1/image.png)

**Usage:**

```bash
./scan.sh vulnerable-app:latest
```

The script scans for vulnerabilities, secrets, and misconfigurations, saving results to text files for review.

### Security Findings

Analysis of the `vulnerable-app` image reveals multiple security issues across three categories:

**1. Vulnerabilities (16 total):**

- **Critical (1)**: SQLite integer truncation vulnerability (CVE-2025-6965)
- **High (10)**: Including glibc, libexpat, GnuTLS, and XZ decoder vulnerabilities
- **Python Dependencies (5)**: Setuptools (3) and Starlette (2) vulnerabilities

**2. Misconfigurations (2):**

- **High Severity**: Container runs as `root` user (AVD-DS-0002)
- **Low Severity**: Missing `HEALTHCHECK` instruction (AVD-DS-0026)

**3. Secrets:**

- No secrets detected in the image layers (hardcoded secrets in source code are not scanned by Trivy's secret scanner)

These findings demonstrate the importance of regular security scanning and following container security best practices.

## Fix #1: Update Base Image to Latest Version

### Description

The first security improvement updates the base Docker image from `python:3.8-slim` to `python:3.12-slim`. This update significantly reduces vulnerabilities in system-level packages while maintaining the same application functionality.

### Changes Made

- Updated base image: `python:3.8-slim` → `python:3.12-slim`
- Located in `vulnerable-app-new-base-image/` directory

### Build and Run

```bash
docker build -t vulnerable-app-new-base-image -f vulnerable-app-new-base-image/Dockerfile vulnerable-app-new-base-image
docker run -p 8000:8000 vulnerable-app-new-base-image
```

### Security Improvements

Scanning with `scan.sh` shows significant vulnerability reduction:

```bash
./scan.sh vulnerable-app-new-base-image:latest
```

- Script After Running: ![Script After Running](screenshots/trivy-scan/v2/image.png)

**Results:**

- **Vulnerabilities**: Reduced from 16 to 2 (both HIGH severity in Starlette library)
  - All system-level vulnerabilities (glibc, libexpat, GnuTLS, SQLite, XZ decoder) eliminated
  - Remaining: 2 Starlette CVEs (CVE-2023-30798, CVE-2024-47874) requiring library updates
  
- **Misconfigurations**: No change (2 issues remain)
  - HIGH: Container still runs as `root` user
  - LOW: Missing `HEALTHCHECK` instruction
  
- **Secrets**: No secrets detected in image layers

Updating the base image eliminated all system-level vulnerabilities but application-level issues persist, requiring additional fixes in subsequent iterations.

## Fix #2: Comprehensive Security Hardening

### Fix #2 Description

The second fix implements comprehensive security best practices including multi-stage builds, non-root user execution, Alpine Linux base, and updated dependencies. The image achieves zero CVEs and zero misconfigurations according to Trivy scans.

### Fix #2 Changes Made

Located in `secured-app-image/` directory:

1. **Multi-stage build**: Separates build dependencies from runtime environment
2. **Alpine Linux base**: Switched to `python:3.12-alpine` for minimal attack surface
3. **Non-root user**: Creates and uses `appuser` (UID 1001) instead of root
4. **Updated dependencies**: Latest secure versions (FastAPI 0.124.4, Uvicorn 0.38.0)
5. **Removed EXPOSE**: Eliminated unnecessary port declaration
6. **Environment variables**: Application reads secrets from environment instead of hardcoding in source

### Fix #2 Build and Run

```bash
docker build -t secured-app-image -f secured-app-image/Dockerfile secured-app-image
docker run -p 8000:8000 secured-app-image
```

### Fix #2 Application Screenshots

- Application Running: ![Application Running](screenshots/secured-app-image/image1.png)
- Application Home: ![Application Home](screenshots/secured-app-image/image2.png)
- API Secret Endpoint: ![API Secret Endpoint](screenshots/secured-app-image/image3.png)

### Security Analysis

Scanning with `scan.sh` demonstrates complete vulnerability remediation:

```bash
./scan.sh secured-app-image:latest
```

- Script After Running: ![Script After Running](screenshots/trivy-scan/v3/image.png)

**Results:**

- **Vulnerabilities**: 0 CVEs detected (eliminated all 16 previous vulnerabilities)
  - Alpine base resolved all system-level issues
  - Updated Python packages eliminated Starlette CVEs
  
- **Misconfigurations**: 0 issues (both previous issues resolved)
  - Container now runs as non-root user `appuser`
  - Security best practices fully implemented
  
- **Secrets**: 0 detected by Trivy

### Secret Detection Limitation

Despite achieving zero vulnerabilities and misconfigurations, the Dockerfile contains a hardcoded secret in an ENV instruction:

```dockerfile
ENV OPENAI_API_KEY=sk-proj-r7k9pL2mX8vQ4tWfY6nZ3bJ5cH1aD9eG8uT0sR6yI4oP2wE5qU7xZ3cV9nB6mL1kA8jF5hG
```

**Trivy limitation**: Failed to detect this hardcoded API key in the Dockerfile ENV variable.

**Docker build detection**: Modern Docker BuildKit with secret scanning features (e.g., `docker scout`) successfully identifies this secret during build time, highlighting the importance of using multiple security tools in a DevSecOps pipeline.

This demonstrates that while Trivy excels at vulnerability and misconfiguration detection, it has limitations in detecting secrets embedded in Dockerfile instructions. A comprehensive security strategy requires layered tooling including static analysis, build-time secret scanning, and runtime monitoring.

## Fix #3: Kubernetes Secret Management

### Fix #3: Description

The third improvement eliminates hardcoded secrets from the Docker image by implementing Kubernetes-native secret management. Secrets are injected at runtime through Kubernetes Secret resources, ensuring sensitive data never exists in the container image layers.

### Fix #3: Changes Made

Located in `kubernetes-secret-version/` directory:

1. **Removed ENV secret**: Dockerfile no longer contains hardcoded API keys
2. **Kubernetes Secret**: Created separate Secret resource for sensitive data
3. **Environment injection**: Secrets mounted as environment variables at pod runtime
4. **Enhanced security context**: Added pod-level security policies
5. **Service exposure**: NodePort service for external access

### Fix #3: Kubernetes Resources

The `k8s-all.yaml` manifest includes:

- **Secret**: Stores `OPENAI_API_KEY` securely in Kubernetes
- **Deployment**: 2 replicas with secret environment injection and security policies
- **Service**: NodePort service exposing the application
- **Attacker Pod**: BusyBox pod for security testing

### Deploy to Kubernetes

1. Start Minikube cluster:

   ```bash
   minikube start
   ```

   ![Minikube Status](screenshots/kubernetes-secret-version/minikube-status.png)

2. Build the Docker image:

   ```bash
   docker build -t secure-app:latest -f kubernetes-secret-version/Dockerfile kubernetes-secret-version
   ```

3. Apply Kubernetes manifests:

   ```bash
   kubectl apply -f kubernetes-secret-version/k8s-all.yaml
   ```

   ![Kubectl Apply](screenshots/kubernetes-secret-version/kubectl-apply.png)

4. Verify deployment:

   ```bash
   kubectl get pods
   kubectl get svc
   ```

5. Access the application:

   ```bash
   minikube service secure-app-svc
   ```

### Fix #3 Application Screenshots

- Application Running: ![Application Running](screenshots/kubernetes-secret-version/image1.png)
- Application Home: ![Application Home](screenshots/kubernetes-secret-version/image2.png)
- API Secret Endpoint: ![API Secret Endpoint](screenshots/kubernetes-secret-version/image3.png)

### Security Verification

The deployment demonstrates proper secret isolation:

![Nodes Running](screenshots/falco/1-nodes-running.png)
![NodePort Service](screenshots/falco/2-nodeport-service.png)
![Secret Correctly Injected](screenshots/falco/3-secret-correctly-injected.png)

**Key Security Features:**

- Secrets stored in Kubernetes etcd (encrypted at rest when configured)
- No secrets in Docker image layers or Dockerfile
- Runtime injection prevents secret exposure in version control
- Security context enforces non-root execution and capability restrictions

## Runtime Security with Falco

### Falco Description

Falco provides runtime security monitoring for Kubernetes workloads, detecting anomalous behavior and potential security threats in real-time. This layer adds behavioral analysis beyond static image scanning.

### Falco Setup

Falco monitors kernel system calls to detect suspicious activities such as:

- Unexpected shell executions in containers
- Privilege escalation attempts
- Unauthorized file access
- Network anomalies

### Attack Simulation

To demonstrate Falco's detection capabilities, a simulated attack was performed by attaching a shell to a running container:

1. Attach shell to the attacker pod:

   ```bash
   kubectl exec -it $(kubectl get pod -n sec-lab -l app=secure-app -o jsonpath="{.items[0].metadata.name}") -n sec-lab -- sh
   ```

   ![Shell Attachment](screenshots/falco/attack-simulation/1-attach-a-shell.png)

2. Falco immediately detected the suspicious activity:

   ![Falco Detection](screenshots/falco/attack-simulation/2-falco-detection.png)

### Security Event Detection

Falco successfully identified the shell execution as a potential security incident, generating alerts with:

- Process details (PID, command line)
- Container and pod identification
- Timestamp and severity level
- Contextual metadata for incident response

This demonstrates the importance of runtime security monitoring as a complementary layer to static image analysis, providing defense-in-depth for containerized applications.