Skip to content

phase2-attest-baseline: drift-aware --baseline diff for azureclaw attest (Phase 2 S11.1) - #61

Merged
Pal Lakatos-Toth (pallakatos) merged 1 commit into
devfrom
phase2-attest-baseline
Apr 28, 2026
Merged

Pal Lakatos-Toth (pallakatos) merged 1 commit into
devfrom
phase2-attest-baseline

Conversation

@pallakatos

Copy link
Copy Markdown
Collaborator

Outcome-shaped follow-up to S11 (#59). Turns azureclaw attest from "print JSON" into a CI-gate / change-control primitive.

Real-world workflow this unlocks

# Day 0 — capture approved posture
$ azureclaw attest demo --format json > approved.json
$ git add approved.json && git commit -m "approved: demo posture"

# Every PR / nightly job — fail the build on drift
$ azureclaw attest demo --baseline approved.json || exit $?
✗ ToolPolicy 'tp-prod' versionHash drifted (sha256:abc1234… → sha256:def5678…)
✗ new SSA manager touched the object: 'kubectl-edit'
DRIFT: 2 delta(s) — exit code 2

Surface

  • cli/src/commands/attest.ts adds diffAttestations (pure function), loadBaseline, describeDelta, --baseline flag, exit-code handling.
  • cli/src/commands/attest.test.ts adds 11 new cases (every delta variant, set-vs-count invariant, missing/invalid baseline, exhaustive describeDelta).
  • CHANGELOG.md — S11.1 entry above S11.
  • docs/security-audits/2026-04-28-phase2-attest-baseline.md — 11-section audit (reuse, STRIDE, out-of-scope, set-vs-count rationale).

Deltas surfaced (one variant per human-meaningful change)

Variant Trigger
specHash ClawSandbox.spec itself changed
phase Running ↔ Overlay ↔ Degraded
policyVersionHash Referenced policy CR was recompiled
policyAdded / policyRemoved Spec references a different policy set
fieldOwnerAdded / fieldOwnerRemoved New/removed SSA manager

Set-comparison, not count-comparison, on field owners. SSA bumps the per-field count on every controller reconcile (noisy); set comparison is the right granularity to flag "did a new actor touch this object?" without paging on every reconcile. Asserted in tests.

Exit codes (CI-friendly)

  • 0 — match
  • 2 — drift (deltas reported in human + JSON output)
  • 3 — baseline file missing (stderr before any kubectl)

Pure-function design

diffAttestations(baseline, current) has no IO, no time, no kubectl. A future Phase 3 azureclaw verify <bundle> companion can reuse it unchanged.

Reuse, no duplication (§0.2 #11)

  • Diff inputs are the S11 AttestationReport shape; baseline file is the S11 attestation JSON envelope. No second schema.
  • No new dependency (node:fs/promises already in CLI).
  • No CRD change, no controller change, no new K8s object.

Out of scope (Phase 3)

  • Signed-baseline verification (lands when controller emits cosign signatures).
  • azureclaw verify <bundle> companion (will reuse diffAttestations).
  • --at <ts> time-travel mode (needs controller-side persistent receipt log).
  • --all / --baseline-dir fleet mode (separate slice).

Verification

  • cd cli && npx tsc --noEmit ✅
  • cd cli && npm test — 315 passed | 2 skipped (was 304+2; +11 from this slice) ✅
  • cd cli && npm run lint ✅ (preexisting warnings only)
  • BASE_REF=origin/dev bash ci/no-stubs.sh ✅
  • BASE_REF=origin/dev bash ci/no-custom-crypto.sh ✅
  • BASE_REF=origin/dev bash ci/check-loc.sh ✅

Phase 2 progress on dev after merge: ✅ S1 #51, S2 #52, S3 #53, S4 #54, S5 #55, S6 #56, S8 #57, S11 #59, S11.1 (this PR) — 9 of ~14 slices.

…test`

Outcome-shaped follow-up to S11. Turns `azureclaw attest` from "print
JSON" into a CI-gate / change-control primitive: pass --baseline
<file> and the command compares the live sandbox against a previously-
saved attestation, surfaces typed deltas, and exits 2 on drift / 3 on
missing baseline so a pipeline step can `set -e` against it.

Real workflow this unlocks:

    # Day 0 — capture approved posture
    azureclaw attest demo --format json > approved.json
    git add approved.json && git commit -m "approved: demo posture"

    # Every PR / nightly job — fail the build on drift
    azureclaw attest demo --baseline approved.json || exit $?

What deltas are surfaced (one human-meaningful change per delta):

  - specHash             — ClawSandbox.spec changed
  - phase                — Running ↔ Overlay ↔ Degraded
  - policyVersionHash    — referenced policy CR recompiled
  - policyAdded/Removed  — spec now references a different policy set
  - fieldOwnerAdded/Removed — new (or removed) SSA manager touched
                              the object since baseline

Set-comparison, not count-comparison, on field owners: SSA bumps the
per-field count on every controller reconcile (noisy), but the set of
managers is what a CI gate actually wants to flag — "did a human or a
tool that wasn't here before edit this object?". Asserted in tests.

Pure-function design: diffAttestations(baseline, current) is the only
new logic; no IO, no time, no kubectl. Means a future Phase 3
`azureclaw verify <bundle>` companion can reuse it unchanged.

Exit codes:
  0 — match
  2 — drift (deltas reported)
  3 — baseline file missing (printed to stderr before any kubectl)

JSON output grows a `baselineDiff: { baseline, current, deltas, drift }`
field. Base envelope unchanged so existing consumers continue parsing.

Tests: CLI workspace 304 → 315 (+11). 11 new cases covering every
delta variant, set-comparison vs count-fluctuation invariant, missing
baseline, invalid baseline, exhaustive describeDelta. tsc --noEmit +
vitest + oxlint green; ci/no-stubs.sh + ci/no-custom-crypto.sh +
ci/check-loc.sh green with BASE_REF=origin/dev.

Audit: docs/security-audits/2026-04-28-phase2-attest-baseline.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@pallakatos
Pal Lakatos-Toth (pallakatos) merged commit 19ba11a into dev Apr 28, 2026
14 of 15 checks passed
@pallakatos
Pal Lakatos-Toth (pallakatos) deleted the phase2-attest-baseline branch April 28, 2026 08:00
Pal Lakatos-Toth (pallakatos) added a commit that referenced this pull request May 12, 2026
…test` (#61)

Outcome-shaped follow-up to S11. Turns `azureclaw attest` from "print
JSON" into a CI-gate / change-control primitive: pass --baseline
<file> and the command compares the live sandbox against a previously-
saved attestation, surfaces typed deltas, and exits 2 on drift / 3 on
missing baseline so a pipeline step can `set -e` against it.

Real workflow this unlocks:

    # Day 0 — capture approved posture
    azureclaw attest demo --format json > approved.json
    git add approved.json && git commit -m "approved: demo posture"

    # Every PR / nightly job — fail the build on drift
    azureclaw attest demo --baseline approved.json || exit $?

What deltas are surfaced (one human-meaningful change per delta):

  - specHash             — ClawSandbox.spec changed
  - phase                — Running ↔ Overlay ↔ Degraded
  - policyVersionHash    — referenced policy CR recompiled
  - policyAdded/Removed  — spec now references a different policy set
  - fieldOwnerAdded/Removed — new (or removed) SSA manager touched
                              the object since baseline

Set-comparison, not count-comparison, on field owners: SSA bumps the
per-field count on every controller reconcile (noisy), but the set of
managers is what a CI gate actually wants to flag — "did a human or a
tool that wasn't here before edit this object?". Asserted in tests.

Pure-function design: diffAttestations(baseline, current) is the only
new logic; no IO, no time, no kubectl. Means a future Phase 3
`azureclaw verify <bundle>` companion can reuse it unchanged.

Exit codes:
  0 — match
  2 — drift (deltas reported)
  3 — baseline file missing (printed to stderr before any kubectl)

JSON output grows a `baselineDiff: { baseline, current, deltas, drift }`
field. Base envelope unchanged so existing consumers continue parsing.

Tests: CLI workspace 304 → 315 (+11). 11 new cases covering every
delta variant, set-comparison vs count-fluctuation invariant, missing
baseline, invalid baseline, exhaustive describeDelta. tsc --noEmit +
vitest + oxlint green; ci/no-stubs.sh + ci/no-custom-crypto.sh +
ci/check-loc.sh green with BASE_REF=origin/dev.

Audit: docs/security-audits/2026-04-28-phase2-attest-baseline.md.

Co-authored-by: Pal Lakatos-Toth <pallakatos@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant