cui-registry

Vendored CUI (Controlled Unclassified Information) registries, converted to structured YAML and split into one crate per registry. Designed to be consumed as [build-dependencies] by Rust projects that codegen Rust types/validators from the registry data (e.g. Marque).

For now the workspace ships one registry: the DoD CUI Registry (Categories and Abbreviations, by Index Group), as crates/dod (cui-registry-dod). This mirrors the structure of ism-data (vendored ODNI XML schemas), scaled down for a much smaller, YAML-native dataset. Additional registries (e.g. the government-wide NARA CUI Registry) can be added as sibling crates under crates/ later.

Layout

cui-registry/                      (workspace root)
  Cargo.toml                       ← virtual workspace
  README.md
  SECURITY.md                      ← threat model + consumer hardening
  provenance.toml                  ← vendor snapshot metadata
  vendor/
    raw/                           ← raw HTML snapshots (audit trail only,
                                      NOT shipped in the crate)
    sources.json                   ← per-page source URL + wayback
                                      timestamp + SHA-256
  crates/
    dod/                           ← cui-registry-dod
      Cargo.toml
      build.rs                     ← verifies data/ at compile time
      src/lib.rs                   ← path + integrity API
      data/
        DOD/
          categories.yaml
          index-groups.yaml
          change-log.yaml
          meta.yaml
        _provenance/
          manifest.txt             ← SHA-256 of every file under data/
          manifest.sha256          ← SHA-256 of manifest.txt itself
  tools/                           ← vendor + regen + release scripts
  .github/workflows/               ← integrity + monthly-vendor + release CI

Consume from your project

[build-dependencies]
cui-registry-dod = "20260716.0.0"

// build.rs
fn main() {
    let categories_yaml = cui_registry_dod::package_root().join("categories.yaml");
    println!("cargo:rerun-if-changed={}", categories_yaml.display());

    // Parse with your YAML library of choice and codegen from here.
    my_codegen::generate_rust(&categories_yaml);
}

Crate API

Every cui-registry-<registry> crate exposes:

pub const PACKAGE_NAME: &str;          // e.g. "DOD"
pub const MANIFEST_DIGEST: &str;       // SHA-256 of data/_provenance/manifest.txt
pub const FILE_COUNT: usize;           // count of files under data/

pub fn data_root() -> PathBuf;         // -> .../cui-registry-dod/data
pub fn package_root() -> PathBuf;      // -> .../cui-registry-dod/data/DOD

pub fn verify_integrity() -> Result<usize, VerifyError>;
pub fn verify_file(rel: &str) -> Result<(), VerifyError>;

env!("CARGO_MANIFEST_DIR") resolves at the consumer's compile time to the location where Cargo placed the crate's source, so data_root() / package_root() paths just work.

Integrity

build.rs re-hashes every file under data/ against data/_provenance/manifest.txt at the consumer's compile time and refuses to compile on mismatch. See SECURITY.md for the full threat model.

Versioning

[workspace.package].version is YYYYMMDD.MAJOR.PATCH, where YYYYMMDD is the date this workspace's data was vendored (captured from the live registry), not a date the registry itself publishes — DoD updates categories on a rolling basis rather than shipping dated releases. See CLAUDE.md for the full scheme.

CI / automation

• integrity.yml — every PR + weekly cron. Re-derives the manifest from crates/dod/data/, diffs against the committed manifest, and runs the full cargo test suite including the heavy --ignored full-tree integrity check.
• monthly-vendor.yml — 1st of every month + manual dispatch. Re-fetches the entire registry, diffs the converted output against what's committed (not raw HTML, which is too volatile byte-for-byte to diff usefully), and — if anything changed — regenerates the YAML, bumps the version, and opens a PR carrying all of it. Never commits to main or triggers a release directly.
• vendor-pr-merged.yml — fires when a human merges an automated-vendor-labeled PR into main; tags the merge commit and dispatches release.yml. Merging the PR is the only action that ships a release. See monthly-vendor.yml's header comment for how to switch back to a fully-automated direct-commit flow instead.
• release.yml — tag push (from vendor-pr-merged.yml, or manual) — publishes cui-registry-dod to crates.io via trusted publishing.

Fetch strategy

www.dodcui.mil sits behind an Akamai bot/geo mitigation that returns HTTP 403 to many non-browser and non-US-network clients — observed even from a real headless browser in some environments. tools/vendor_lib.py tries a direct fetch first and falls back to the nearest Internet Archive Wayback Machine capture at or before a target date. This is why provenance.toml and vendor/sources.json record, per page, whether the fetch was direct or via Wayback, and the resolved Wayback timestamp when applicable.

What's intentionally not here

• No runtime YAML parsing. This workspace is data + path helpers only, exactly like ism-data. Pair it with serde_yaml or whatever library suits your codegen.
• Raw HTML is not shipped in the crate. It's kept under vendor/raw/ for audit and diffing only; the crate ships the converted YAML.

License

Registry content is a U.S. Government public-domain work. Crate scaffolding (Rust source, build scripts, CI, config) is dual-licensed under MIT-0 or Unlicense. See LICENSES/ and the LICENSE-* files.