DERP - Durable Execution, Really Poorly

A Rust-based Durable Execution service that compiles TypeScript to WebAssembly Components.

This is designed to be an educational projects for those looking to build similar systems. I'm slower at writing blog posts than I am at code, though, so this is getting released first.

In the meantime, please reach out if you have questions and let me know if you do something cool with this!

Features

• TypeScript Functions: Write serverless functions in TypeScript
• WebAssembly Components: Compiled to WASM Components via ComponentizeJS
• Custom Host Functions: Built-in HTTP client, logging, sql, and execution context
• Type-Safe: WIT-based interfaces for host/guest communication
• Persistent Storage: SQLite database for metadata, filesystem for artifacts
• RESTful API: HTTP API for function management and execution
• CLI Interface: Complete command-line tool for all operations

Architecture

TypeScript Source
      ↓ (esbuild)
JavaScript Bundle
      ↓ (componentize-js)
WASM Component
      ↓ (wasmtime)
Execution

Components

• derp-cli: Command-line interface
• derp-server: HTTP API server (Axum)
• derp-runtime: WASM runtime with custom host functions (wasmtime)
• derp-build: Build pipeline (esbuild + componentize-js)
• derp-storage: Storage layer (SQLite + filesystem)
• derp-types: Shared types

Prerequisites

• Rust 1.89.0+
• Node.js (for esbuild and componentize-js)
• SQLite

Install External Tools

# Install esbuild
npm install -g esbuild

# Install componentize-js
npm install -g @bytecodealliance/componentize-js

Quick Start

1. Build the Project

cargo build --release

2. Start the Server

./target/release/derp serve

The server will listen on http://127.0.0.1:8888 by default.

3. Upload a Function

./target/release/derp function upload hello-world examples/hello-world.ts

4. Check Build Status

./target/release/derp function build status hello-world 1

5. Run the Function

./target/release/derp function run hello-world --input '{"name": "World"}'

CLI Commands

Server

# Start HTTP server
derp serve [--addr ADDRESS]

Functions

# Upload a function
derp function upload <NAME> <FILE>

# List functions
derp function list

# Run a function
derp function run <NAME> --input <JSON>

Builds

# Get build status for a function
derp function build status <NAME> <ID>

# List builds for a function
derp function build list <NAME>

# Get build status for a workflow
derp workflow build status <NAME> <ID>

# List builds for a workflow
derp workflow build list <NAME>

Environment Variables

# Set an environment variable for a function
derp function env set <NAME> <KEY> <VALUE>

# Get an environment variable
derp function env get <NAME> <KEY>

# List environment variables
derp function env list <NAME>

# Delete an environment variable
derp function env delete <NAME> <KEY>

# Same commands work for workflows
derp workflow env set <NAME> <KEY> <VALUE>
derp workflow env list <NAME>

HTTP API

Create Function

POST /functions
Content-Type: application/json

{
  "name": "my-function",
  "source": "export function handler(req) { ... }"
}

Execute Function

POST /functions/:name/execute
Content-Type: application/json

{
  "body": { "key": "value" },
  "metadata": {}
}

Get Function Info

GET /functions/:name

Get Build Status

GET /functions/:name/builds/:id
GET /workflows/:name/builds/:id

Manage Environment Variables

GET /functions/:name/env
PUT /functions/:name/env/:var_name
DELETE /functions/:name/env/:var_name

GET /workflows/:name/env
PUT /workflows/:name/env/:var_name
DELETE /workflows/:name/env/:var_name

Configuration

Environment variables:

• DATABASE_URL: Database path (default: sqlite:./data/db.sqlite)
• ARTIFACT_DIR: Artifact storage directory (default: ./data/artifacts)
• WIT_DIR: WIT interface definitions (default: ./wit)

Or use command-line flags:

derp --database-url sqlite:./my-db.sqlite \
     --artifact-dir ./my-artifacts \
     --wit-dir ./my-wit \
     serve

Writing Functions

See examples/README.md for detailed examples.

Function Structure

Every function must export a handler object with two optional methods:

export const handler = {
    // Optional: One-time initialization (runs before first handle call)
    setup() {
        // Initialize database schema, set up state, etc.
    },

    // Required: Handle incoming requests
    handle(req: Request<InputData>): Response {
        // Process the request and return a response
    }
};

Method	Required	Called	Purpose
setup()	No	Once, before first handle()	Database schema creation, one-time initialization
handle(req)	Yes	Every invocation	Process requests and return responses

Function Lifecycle

┌─────────────────────────────────────────────────────────┐
│                    Function Upload                      │
└─────────────────────────────────────────────────────────┘
                           │
                           ▼
              ┌───────────────────────────────┐
              │  TypeScript → WASM Component  │
              │  (esbuild + componentize-js)  |
              └───────────────────────────────┘
                           │
                           ▼
              ┌────────────────────────┐
              │  Function Ready        │
              └────────────────────────┘
                           │
        ┌──────────────────┴──────────────────┐
        │         First Invocation            │
        ▼                                     │
┌───────────────┐                             │
│   setup()     │ ◄─── Runs once              │
│  (if exists)  │                             │
└───────────────┘                             │
        │                                     │
        ▼                                     │
┌───────────────┐                             │
│   handle()    │ ◄─── Runs on every request ─┘
└───────────────┘
        │
        ▼
┌───────────────┐
│   Response    │
└───────────────┘

Basic Template

import type { Request, Response } from 'derp:runtime/types';
import { success, error } from 'derp:runtime/context';
import { exec } from 'derp:runtime/sql';
import * as log from 'derp:runtime/logger';

interface InputData {
    name: string;
}

export const handler = {
    // Optional: runs once before first handle() call
    setup() {
        exec(`
            CREATE TABLE IF NOT EXISTS greetings (
                id INTEGER PRIMARY KEY,
                name TEXT NOT NULL,
                created_at INTEGER DEFAULT (unixepoch())
            )
        `);
    },

    handle(req: Request<InputData>): Response {
        log.info('Function called');

        // req.body is already parsed - no JSON.parse needed!
        const { name } = req.body;

        // Store the greeting
        exec('INSERT INTO greetings (name) VALUES (?)', name);

        // success() accepts objects directly - no JSON.stringify needed!
        return success({ greeting: `Hello, ${name}!` });
    }
};

Available Host Functions

HTTP Client:

import { fetch, getJson } from 'derp:runtime/http';

// Low-level fetch
const response = fetch({
    method: 'GET',
    url: 'https://api.example.com',
    headers: [['User-Agent', 'DERP']],
});

// Convenience method for JSON APIs (auto-parses response)
const { status, body } = getJson<User>('https://api.example.com/users/1');

Logger:

import * as log from 'derp:runtime/logger';

log.debug('Debug message');
log.info('Info message');
log.warn('Warning message');
log.error('Error message');

Context:

import { getInfo, success, error } from 'derp:runtime/context';

const info = getInfo();
console.log(info.functionName, info.executionId);

SQL (Per-Function SQLite Database):

import { exec, transaction } from 'derp:runtime/sql';

// Execute queries with parameter binding
const cursor = exec('SELECT * FROM users WHERE id = ?', 123);

// Cursor methods
cursor.toArray();   // All rows as array of objects
cursor.one();       // Exactly one row (throws if 0 or >1)
cursor.first();     // First row or undefined

// Transactions (auto-rollback on error)
const id = transaction(() => {
    exec('INSERT INTO users (name) VALUES (?)', 'Alice');
    return exec('SELECT last_insert_rowid() as id').one().id;
});

Environment Variables:

import * as env from 'derp:runtime/env';

const apiKey = env.get('API_KEY');           // string | undefined
const debug = env.getOrDefault('DEBUG', 'false');

Database Schema

Functions Table

• name (PRIMARY KEY)
• status (storing | compiling | ready)
• content_hash
• version
• created_at
• updated_at

Function Builds Table

• id (PRIMARY KEY)
• function_name (FOREIGN KEY)
• status (running | success | failure)
• input_file
• output_file
• error
• created_at
• updated_at

Function Environment Variables Table

• id (PRIMARY KEY)
• function_name (FOREIGN KEY)
• name
• value
• created_at
• updated_at

Workflow Builds Table

• id (PRIMARY KEY)
• workflow_name (FOREIGN KEY)
• status (running | success | failure)
• input_file
• output_file
• error
• created_at
• updated_at

Workflow Environment Variables Table

• id (PRIMARY KEY)
• workflow_name (FOREIGN KEY)
• name
• value
• created_at
• updated_at

Project Structure

part1/
├── Cargo.toml              # Workspace configuration
├── crates/
│   ├── derp-cli/          # CLI binary
│   ├── derp-server/       # HTTP API
│   ├── derp-runtime/      # WASM runtime
│   ├── derp-build/        # Build pipeline
│   ├── derp-storage/      # Storage layer
│   └── derp-types/        # Shared types
├── wit/                    # WIT interface definitions
├── runtime/                # TypeScript type definitions
├── examples/               # Example functions
├── migrations/             # Database migrations
└── data/                   # Runtime data (git-ignored)
    ├── db.sqlite          # SQLite database
    └── artifacts/         # Compiled functions

Development

Run Tests

cargo test

Format Code

cargo fmt

Build for Release

cargo build --release

License

See LICENSE file.