Skip to content

Incremental compilation and parallel TypeTransformer #21

Description

@danfma

Two perf improvements to the compiler pipeline that become relevant as project size grows. Grouped in one issue because they share the dependency-graph infrastructure.

Incremental compilation

  • Compute a content hash per `.cs` file (file bytes + Roslyn compilation reference metadata that affects it)
  • Persist the hash → generated-files map to disk between runs (`.metano/cache.json` or similar)
  • On subsequent runs, skip types whose inputs (source file, referenced types, mapping metadata) haven't changed
  • Invalidate transitively: if a referenced type regenerates, every type that imports it regenerates too
  • Integration with `--clean` (opt-out of cache)
  • Tests covering hit, miss, and invalidation paths

Parallel TypeTransformer

  • Transpilation is embarrassingly parallel per top-level type — `TypeTransformer` can process each type on a separate worker
  • Use `Parallel.ForEach` or `Dataflow` depending on fan-out characteristics
  • Shared state (symbol maps, `_transpilableTypeMap`, `_crossAssemblyTypeMap`) needs to be read-only during the parallel phase — audit for mutation
  • `TypeScriptTransformContext` already exists as immutable shared state (ADR-0002) — verify it's safe to share across threads
  • Diagnostics collection needs thread-safe aggregation (`ConcurrentBag` or similar)

Combined benefit

Incremental + parallel together make a large monorepo build `O(changed)` instead of `O(total)`, and the baseline of all-types-changed gets faster by a factor close to the number of cores.

Dependencies

Shares machinery with #18 (watch mode) — both need to know "which files would change if this C# file is touched". Keep the dependency graph layer in the core so both can reuse it.

Part of #14.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions