Rhodium Pipeline Template

A methodology template for formally-verified data processing pipelines using the Rhodium Standard Repositories (RSR) methodology.

Contents

• 1. Overview
• 2. Philosophy
  • 2.1. Core Principles
• 3. Quick Start
• 4. Template Variables
• 5. Generated Structure
• 6. License
• 7. Reference Implementation
• 8. Contributing
• 9. Documentation

1. Overview

Rhodium Pipeline Template generates production-ready data processing pipelines with:

Component	Purpose
Rust CLI	Comprehensive command-line interface with full flag taxonomy
Isabelle/HOL	Formal proofs of structural invariants (partitions, bijections, checksums)
Nickel	Type-safe build-time configuration with contracts
Guile Scheme	Runtime configuration validation and scripting
just	Task automation with layered recipe organisation
Julia (optional)	Numerical/ML integration via FFI

2. Philosophy

Prove the scaffolding, test the logic.

— Rhodium Methodology

2.1. Core Principles

1. Formal proofs for structure — Isabelle verifies data doesn’t disappear, splits are disjoint, mappings are bijective

2. Empirical tests for domain — Property-based and unit tests for business logic

3. Configuration as code — Nickel (build-time) + Guile (runtime) + env vars (deploy-time)

4. Automation over documentation — Procedures are recipes; docs explain why

5. No Python — Rust + Julia + Guile + shell

3. Quick Start

# Install cargo-generate if needed
cargo install cargo-generate

# Generate a new project
cargo generate --git https://gitlab.com/hyperpolymath/rhodium-pipeline-template

# Enter project
cd my-pipeline

# Verify toolchain
just check-deps

# Build and validate
just full

4. Template Variables

Variable	Type	Description	Example
project-name	string	Kebab-case project name	vae-normalizer
project_name	string	Snake_case for Rust modules	vae_normalizer
ProjectName	string	PascalCase for Isabelle theories	VAEDataset
description	string	One-line project description	Dataset normalizer for VAE artifacts
author	string	Author name	Jonathan D.A. Jewell
email	string	Contact email	enquiries@joshuajewell.dev
license	choice	Base license (MIT or AGPL-3.0)	AGPL-3.0
include_julia	bool	Include Julia FFI integration	true
checksum_algo	choice	blake3, shake256, sha3-256, xxhash	blake3
isabelle_version	string	Isabelle release version	Isabelle2024
min_rust_version	string	Minimum supported Rust	1.75.0
default_threads	string	Default parallelism	4
enable_telemetry	bool	Optional telemetry hooks	false
target_platforms	string	Comma-separated targets	linux-x86_64,macos-arm64

5. Generated Structure

{{project-name}}/
├── Cargo.toml                    # Rust manifest
├── src/
│   ├── main.rs                   # CLI entry (500+ flag combinations)
│   ├── cli.rs                    # Argument definitions
│   ├── config.rs                 # Configuration loading
│   ├── pipeline.rs               # Core pipeline logic
│   ├── validation.rs             # Input/output validation
│   ├── checksum.rs               # Integrity verification
│   └── {{module}}.rs             # Domain-specific logic
├── proofs/
│   ├── {{ProjectName}}_Invariants.thy    # Core invariants
│   ├── {{ProjectName}}_Partition.thy     # Partition proofs
│   ├── {{ProjectName}}_Bijection.thy     # Bijection proofs
│   ├── {{ProjectName}}_Checksum.thy      # Checksum proofs
│   └── ROOT                              # Isabelle session
├── config/
│   ├── default.ncl               # Nickel configuration
│   ├── presets/
│   │   ├── dev.ncl
│   │   ├── prod.ncl
│   │   ├── ci.ncl
│   │   └── paranoid.ncl
│   └── contracts/
│       ├── paths.ncl
│       ├── ratios.ncl
│       └── checksums.ncl
├── schemes/
│   ├── config.scm                # Guile runtime validation
│   ├── validators.scm            # Custom validators
│   └── transforms.scm            # Data transforms
├── justfile                      # 200+ recipes
├── scripts/
│   └── (integrated into just)
├── tests/
│   ├── integration/
│   └── property/
├── fixtures/
│   └── sample_data/
├── docs/
│   ├── README.adoc
│   ├── QUICKSTART.adoc
│   ├── CLI.adoc
│   ├── ARCHITECTURE.adoc
│   ├── PROOFS.adoc
│   └── man/
│       └── {{project-name}}.1
└── LICENSE

6. License

This template is released under the Palimpsest License v0.8.

Generated projects inherit Palimpsest v0.8 layered on your choice of:

• MIT License

• AGPL-3.0 License

See LICENSE and LICENSING.adoc for details.

7. Reference Implementation

zerostep — VAE dataset normalizer built using this methodology.

8. Contributing

See CONTRIBUTING.adoc for guidelines.

9. Documentation

• HANDOVER.adoc — Complete methodology documentation

• QUICKSTART.adoc — 5-minute getting started

• ARCHITECTURE.adoc — Design decisions and rationale

• CLI.adoc — Complete CLI reference

• PROOFS.adoc — Isabelle proof patterns