basic-docker-engine

Basic docker engine implementation from scratch

Build steps

build go code

@j143 ➜ /workspaces/basic-docker-engine (main) $ go build -o basic-docker main.go 
@j143 ➜ /workspaces/basic-docker-engine (main) $ ./basic-docker 
Environment detected: inContainer=true, hasNamespacePrivileges=true, hasCgroupAccess=false
Usage:
  basic-docker run <command> [args...]  - Run a command in a container
  basic-docker ps                       - List running containers
  basic-docker images                   - List available images
  basic-docker info                     - Show system information

create necessary folders

sudo mkdir -p /tmp/basic-docker/containers
sudo mkdir -p /tmp/basic-docker/images
sudo mkdir -p /sys/fs/cgroup/memory/basic-docker

Set proper permissions

sudo chmod -R 755 /tmp/basic-docker
sudo chmod -R 755 /sys/fs/cgroup/memory/basic-docker

Run a simple command in a container

Note: This needs to be run as root due to namespace operations

sudo ./basic-docker run /bin/sh -c "echo Hello from container"

$ sudo ./basic-docker run /bin/sh -c "echo Hello from container" Starting container container-1743306338 Error: failed to set memory limit: open /sys/fs/cgroup/memory/basic-docker/container-1743306338/memory.limit_in_bytes: permission denied

Architecture

flowchart TB
    subgraph "Basic Docker Engine"
        CLI[CLI Interface]
        ENV[Environment Detection]
        CMD[Command Handler]
        
        subgraph "Container Runtime"
            EXEC[Container Execution]
            FS[Filesystem Manager]
            NET[Network Manager]
            RES[Resource Controller]
        end
        
        subgraph "Image Management"
            IMG[Image Store]
            PULL[Image Puller]
        end
        
        subgraph "Isolation Components"
            FULL["Full Isolation\n(Linux Namespaces)"]
            LIMITED["Limited Isolation\n(chroot/unshare)"]
            FALLBACK["Minimal Isolation\n(runWithoutNamespaces)"]
        end
    end
    
    subgraph "Environment Detection"
        DETECT{Environment?}
        PRIV{Privileges?}
        CGROUP{CGroup Access?}
    end
    
    subgraph "Host System"
        LINUX[Linux Kernel]
        CONTAINERS[Container Environment]
    end
    
    %% Main flow
    CLI --> CMD
    CMD --> ENV
    ENV --> DETECT
    
    %% Environment detection flow
    DETECT -->|"Pure Linux"| PRIV
    DETECT -->|"Container"| LIMITED
    
    PRIV -->|"Root"| FULL
    PRIV -->|"Non-root"| LIMITED
    
    ENV --> CGROUP
    CGROUP -->|"Yes"| RES
    CGROUP -->|"No"| EXEC
    
    %% Execution paths
    FULL --> EXEC
    LIMITED --> EXEC
    FALLBACK --> EXEC
    
    %% Container runtime components
    EXEC --> FS
    EXEC -.-> NET
    EXEC -.-> RES
    
    %% Image management
    CMD -.-> IMG
    IMG -.-> PULL
    PULL -.-> FS
    
    %% Host system interactions
    FULL --> LINUX
    LIMITED --> CONTAINERS
    FALLBACK --> CONTAINERS
    
    %% Styling
    classDef primary fill:#d0e0ff,stroke:#3080ff,stroke-width:2px
    classDef secondary fill:#ffe0d0,stroke:#ff8030,stroke-width:2px
    classDef tertiary fill:#d0ffe0,stroke:#30ff80,stroke-width:2px
    classDef highlight fill:#ffff80,stroke:#808000,stroke-width:2px
    
    class CLI,CMD,ENV primary
    class EXEC,FS,NET,RES secondary
    class FULL,LIMITED,FALLBACK tertiary
    class DETECT,PRIV,CGROUP highlight

Loading

Image Build Logic

graph TD
    A[Start] --> B[Check if Base Layer Exists]
    B -->|Yes| C[Load Base Layer Metadata]
    B -->|No| D[Create Base Layer]
    D --> E[Initialize Base Layer with Minimal Rootfs]
    E --> F[Save Base Layer Metadata]
    C --> G[Create App Layer]
    G --> H[Add Container-Specific Files to App Layer]
    H --> I[Save App Layer Metadata]
    F --> G
    I --> J[Mount Layers to Create Rootfs]
    J --> K[Run Container with Rootfs]
    K --> L[End]
    
    %% Styling
    classDef startEnd fill:#d0e0ff,stroke:#3080ff,stroke-width:2px
    classDef process fill:#ffe0d0,stroke:#ff8030,stroke-width:2px
    classDef decision fill:#d0ffe0,stroke:#30ff80,stroke-width:2px
    
    class A,L startEnd
    class B decision
    class C,D,E,F,G,H,I,J,K process

Loading

The logic implemented for building images and running containers in the basic Docker engine. It starts by checking if the base layer exists, initializes it if necessary, and then creates an app layer for container-specific files. Finally, the layers are mounted to create the root filesystem, and the container is executed.

Code Structure

Diagram of Classes and Functions

graph TD
    subgraph Image Management
        Image["Image"]
        Registry["Registry Interface"]
        Manifest["Manifest"]
        Pull["Pull Function"]
        ListImages["ListImages Function"]
        ExtractLayer["extractLayer Function"]
    end

    subgraph Main Application
        Main["main Function"]
        ListContainers["listContainers Function"]
        Run["run Function"]
        PrintSystemInfo["printSystemInfo Function"]
        ExecCommand["execCommand Function"]
    end

    Main -->|Delegates| Pull
    Main -->|Delegates| ListImages
    Pull -->|Uses| Registry
    Pull -->|Uses| Manifest
    Pull -->|Calls| ExtractLayer
    ListImages -->|Reads| Image
    Run -->|Creates| Image
    ExecCommand -->|Interacts| Containers

Loading

This diagram provides an overview of the key classes and functions in the project, showing their relationships and responsibilities.

Basic docker prompts

Docker Monitoring System

The basic-docker engine now includes comprehensive monitoring capabilities that address the "Docker monitoring problem" by providing visibility across process, container, and host isolation levels.

Monitor Commands

# Monitor host-level metrics
./basic-docker monitor host

# Monitor specific process
./basic-docker monitor process <PID>

# Monitor specific container
./basic-docker monitor container <container-id>

# Monitor all levels (process, container, host)
./basic-docker monitor all

# Analyze monitoring gaps between isolation levels  
./basic-docker monitor gap

# Show correlation between monitoring levels
./basic-docker monitor correlation <container-id>

Example Output

./basic-docker monitor correlation container-1234

Shows the correlation table as described in the Docker monitoring problem:

Aspect	Process	Container	Host
Spec	Source	Dockerfile	Kickstart
On disk	.TEXT	/var/lib/docker	/
In memory	PID	Container ID	Hostname
In network	Socket	veth*	eth*
Runtime context	server core	host	data center
Isolation	moderate	private OS view	full

See MONITORING.md for detailed documentation.

basic-docker info

/workspaces/basic-docker-engine (main) $ ./basic-docker info
Environment detected: inContainer=true, hasNamespacePrivileges=true, hasCgroupAccess=false
Lean Docker Engine - System Information
=======================================
Go version: go1.24.1
OS/Arch: linux/amd64
Running in container: true
Namespace privileges: true
Cgroup access: false
Available features:
  - Process isolation: true
  - Network isolation: true
  - Resource limits: false
  - Filesystem isolation: true

basic-docker run

/workspaces/basic-docker-engine (main) $ ./basic-docker run /bin/echo "Hello from container"
Environment detected: inContainer=true, hasNamespacePrivileges=true, hasCgroupAccess=false
Starting container container-1743307284
unshare: unshare failed: Operation not permitted
Error: exit status 1
Container container-1743307284 exited

@j143 ➜ /workspaces/basic-docker-engine (main) $ sudo ./basic-docker run /bin/echo "Hello from container"
Environment detected: inContainer=true, hasNamespacePrivileges=true, hasCgroupAccess=true
Starting container container-1743307290
Hello from container
Container container-1743307290 exited

basic-docker ps

@j143 ➜ /workspaces/basic-docker-engine (main) $ sudo ./basic-docker ps
Environment detected: inContainer=true, hasNamespacePrivileges=true, hasCgroupAccess=true
CONTAINER ID    STATUS  COMMAND
container-1743307284    N/A     N/A
container-1743307290    N/A     N/A

basic-docker run /bin/sh

@j143 ➜ /workspaces/basic-docker-engine (main) $ sudo ./basic-docker run /bin/echo "Hello from container"
Environment detected: inContainer=true, hasNamespacePrivileges=true, hasCgroupAccess=true
Starting container container-1743307567
Hello from container
Container container-1743307567 exited

basic-docker run /bin/busybox

@j143 ➜ /workspaces/basic-docker-engine (main) $ sudo ./basic-docker run /bin/busybox echo "Hello from BusyBox"
Environment detected: inContainer=true, hasNamespacePrivileges=true, hasCgroupAccess=false
Starting container container-1744512443
Hello from BusyBox
Container container-1744512443 exited

Verify the prompt functionality

@j143 ➜ /workspaces/basic-docker-engine (main) $ chmod +x verify.sh 
@j143 ➜ /workspaces/basic-docker-engine (main) $ ./verify.sh 
==== System Information ====
Environment detected: inContainer=true, hasNamespacePrivileges=true, hasCgroupAccess=false
Lean Docker Engine - System Information
=======================================
Go version: go1.24.1
OS/Arch: linux/amd64
Running in container: true
Namespace privileges: true
Cgroup access: false
Available features:
  - Process isolation: true
  - Network isolation: true
  - Resource limits: false
  - Filesystem isolation: true
===========================


==== Running Simple Command ====
Environment detected: inContainer=true, hasNamespacePrivileges=true, hasCgroupAccess=true
Starting container container-1743307590
Hello from container
Container container-1743307590 exited


==== Listing Containers ====
Environment detected: inContainer=true, hasNamespacePrivileges=true, hasCgroupAccess=true
CONTAINER ID    STATUS  COMMAND
container-1743307284    N/A     N/A
container-1743307290    N/A     N/A
container-1743307567    N/A     N/A
container-1743307590    N/A     N/A


==== Testing with busybox ====
Environment detected: inContainer=true, hasNamespacePrivileges=true, hasCgroupAccess=true
Starting container container-1743307590
Error: failed to create symlink for sh: symlink busybox /tmp/basic-docker/containers/container-1743307590/rootfs/bin/sh: file exists


==== Skipping isolation tests (needs root) ====


==== All tests completed ====

Test Strategy

Current Testing Status

graph TD
    A[Unit Tests] --> B[Integration Tests]
    B --> C[End-to-End Tests]
    C --> D[Verification Script]

    %% Styling
    classDef process fill:#ffe0d0,stroke:#ff8030,stroke-width:2px

    class A,B,C,D process

Loading

• Unit Tests: Present in main_test.go and image_test.go.
  • Includes tests for AddResourceCapsule, CapsuleManager, and Docker integration.
  • Benchmarks for capsule and volume access.
• Integration Tests: Covered partially in verify.sh.
  • Verifies container creation, image listing, and network functionality.
• End-to-End Tests: Basic functionality tested via verify.sh.
  • Simulates real-world usage scenarios.
• Verification Script: verify.sh runs additional checks and validations.
  • Includes system information, container commands, and BusyBox tests.