Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions bin/mega
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@
# bin/mega -- STABLE consumer entrypoint for the mega subsystem (SPEC-184,
# ADR-0034 decision 7: every subsystem gets its bin entry).
#
# Consumers reference THIS stable path, never the deep `$DWARVES_KIT/lib/mega.sh`
# Consumers reference THIS stable path, never the deep `$DWARVES_KIT/lib/mega/mega.sh`
# path, so an internal lib reorg (e.g. a future promotion to lib/mega/mega.sh
# when a second verb lands) never breaks them again.
#
# Usage: mega status <slug> [args...]
set -euo pipefail
SELF_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec bash "$SELF_DIR/../lib/mega.sh" "$@"
exec bash "$SELF_DIR/../lib/mega/mega.sh" "$@"
2 changes: 1 addition & 1 deletion docs/consumer-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ The adopt-injected `CLAUDE.md` block already does this for you:
| `bin/gate` | `lib/gate/gate.sh` |
| `bin/goal` | `lib/goal/goal.sh` |
| `bin/learn` | `lib/learn/learn.sh` (`learn debt <list\|collect\|mark-paid>`; `propose`/`drain` refuse until SPEC-195/196) |
| `bin/mega` | `lib/mega.sh` |
| `bin/mega` | `lib/mega/mega.sh` |
| `bin/queue` | `lib/queue/queue.sh` |
| `bin/session` | `lib/session/session.sh` (`session <intel\|observe\|recall\|report\|semantic>`, ex the five `bin/session-*`) |
| `bin/spec` | `lib/spec/spec.sh` |
Expand Down
114 changes: 114 additions & 0 deletions docs/verification/lib-mega-folder.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# Proof of done: fold the mega engine into lib/mega/ (ID-287, sanctioned slice)

**Change class:** behavioral (moved `.sh`/`.py` + repointed callers in `bin/`, `lib/`, `tests/`).

**Claim.** The three loose mega-goal scripts move from `lib/` root into a cohesive
`lib/mega/` module as a unit, with every live reference repointed and every
root-assuming internal anchor re-anchored one level up:

- `lib/mega.sh` -> `lib/mega/mega.sh`
- `lib/mega-report.py` -> `lib/mega/mega-report.py`
- `lib/mega-review.py` -> `lib/mega/mega-review.py`

## Why this slice (and not the other loose files)

`lib/config/module-registry.md` already registers a **`mega` module** (row :49
"mega | Build (Execute) | spine machinery") and ADR-0034 explicitly names
`lib/mega/mega.sh` + `bin/mega` as the promotion target, so this move is sanctioned
and the three files must move together (they call each other by same-dir path). The
other four loose files (`adopt.sh`, `explain.sh`, `pitch.sh`, `precedent.sh`) are
labeled **"deliberate orphans, no cluster"** by both `lib/README.md` and
`docs/decisions/0034` (bin-less, command-invoked, not module CLIs). Foldering them
into single-file dirs conflicts with that ADR, a source-of-truth question a human
must settle, so they are deferred to a follow-up with that ruling. `onboard-detect.sh`
has no obvious module home either (lib/session vs a new lib/onboard). See the batch
report.

## Internal anchor edits (root-assuming -> one-level-deeper)

- `mega.sh`: `source "$self_dir/telemetry/..."` -> `"$self_dir/../telemetry/..."` (x2).
The `$self_dir/mega-review.py` / `mega-report.py` calls stay (same-dir, co-moved).
- `mega-review.py`: `os.path.join(_SELF_DIR, "gate"...)` and `"learn"...` gain a `".."`;
the `mega.sh` join stays (same-dir).
- `mega-report.py`: no internal path anchor (arg-driven), no edit.

## Repointed live references

`bin/mega:12` (the stable forwarder), `lib/board/board.sh:183` (computed
`$BOARD_DIR/../mega/mega.sh`), `lib/queue/orchestrate.sh:1745` + WARN string,
`tests/test-mega.sh`, `tests/test-mega-review.sh`, `tests/test-mega-report.sh`,
`lib/config/module-registry.md:113,284`, and a `lib/mega/` row added to
`lib/README.md`. Comment references in the moved files + `board.sh` + `test-tier4-close.sh`
updated for accuracy. Historical records left untouched.

## Review disposition (review-team + fable advisor)

| Finding | Lens | Applied? |
|---|---|---|
| `cmd_report`'s re-anchored telemetry `source` had zero red-on-wrong coverage (the mega-report tests call python directly; the one `bin/mega report` call passed no slug) | test-coverage (HIGH) | Applied, added an end-to-end `bin/mega report demo` launcher assertion in `test-mega-report.sh`; verified load-bearing (broken anchor -> 15/17, restored -> 17/17) |
| `docs/consumer-contract.md:64` `bin/mega -> lib/mega.sh` stale post-move (a live onboarding ref, not historical) | architecture (LOW) | Applied -> `lib/mega/mega.sh` |
| `mega.sh` header still framed the file as a deliberate orphan awaiting promotion; `mega-review.py:53` `_SELF_DIR` comment said "lib/ ... orphan file" | architecture (MEDIUM/LOW) + fable + security | Applied, both rewritten to describe the `lib/mega/` module |
| Re-anchoring complete/correct; taxonomy right (mega is a distinct `mega` module, not `goal`); orphan deferral correct (not over-reading ADR-0034) | architecture 8/10, fable, security 10/10 | Confirmed, no change |
| `onboard-detect.sh` absent from `lib/README.md` root-orphans row | fable/architecture | Pre-existing gap, reported for the orphan follow-up (not this branch's regression) |

## Acceptance criteria

| # | Criterion | Result |
|---|---|---|
| 1 | The three files live in `lib/mega/`; old `lib/mega.sh` path gone | PASS |
| 2 | `bin/mega` forwarder reaches the engine at the new path | PASS (30/30 forwarders) |
| 3 | Internal anchors resolve telemetry/gate/learn siblings from the new depth | PASS (mega suites green) |
| 4 | No live (non-doc) reference to an old `lib/mega*.{sh,py}` path remains | PASS |
| 5 | All affected suites green | PASS |

## Confirmation run

```
Command: [ ! -f lib/mega.sh ] && ls lib/mega/
Exit: 0 # mega.sh mega-report.py mega-review.py
Command: rg 'lib/mega\.sh|lib/mega-report\.py|lib/mega-review\.py' bin/ lib/ tests/ --glob '!*.md' | grep -v 'lib/mega/'
Exit: 1 # no un-repointed live ref remains
Command: bash tests/test-mega.sh
Exit: 0
Command: bash tests/test-mega-review.sh
Exit: 0 # 26 passed
Command: bash tests/test-mega-report.sh
Exit: 0
Command: bash tests/test-bin-forwarders.sh
Exit: 0 # all 30 passed
Command: bash tests/test-tier4-close.sh
Exit: 0
Command: bash tests/test-meta.sh
Exit: 0 # Passed: 717 / 717
Command: bash tests/test-hooks.sh
Exit: 0 # Passed: 487 / 487
Verdict: PASS
```

## NEGATIVE CONTROL

The `bin/mega` repoint is load-bearing. Revert only it to the old path and the
forwarder can no longer reach the engine:

```
# revert bin/mega:12 to ../lib/mega.sh (old path):
bash tests/test-bin-forwarders.sh
-> FAIL: mega forwarder reaches mega.sh (29 passed, 1 FAILED) [RED]
# restore ../lib/mega/mega.sh:
bash tests/test-bin-forwarders.sh
-> all 30 passed, 0 skipped [GREEN]
```

**Verdict: PASS.**

## Reproduce

```
bash tests/test-mega.sh tests/test-mega-review.sh tests/test-mega-report.sh
bash tests/test-bin-forwarders.sh # 30/30 (the bin/mega forwarder)
bash tests/test-meta.sh # 717/717
bash tests/test-hooks.sh # 487/487
```

**Rollback:** `git revert` this commit (`git mv` back + restore the anchors/refs); no
state or data step.
1 change: 1 addition & 0 deletions lib/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ orphans with no cluster stay as bare scripts at the root.
| `classify/` | `lane-classify.sh` `role-classify.sh` `significance-classify.sh` `task-type-classify.sh` `route-suggest.sh` |
| `spec/` | `spec-index.sh` `spec-next.sh` |
| `goal/` | `goal-drafts.sh` `goal-registry.sh` `mega-merge.sh` `stack-merge.sh` `handoff-gen` + `handoff/` (`handoff_gen.py`, `cc_compact.py`) |
| `mega/` | `mega.sh` `mega-report.py` `mega-review.py` (the mega-goal engine + its two python halves; stable entry via `bin/mega`) |
| `telemetry/` | `lane-telemetry.sh` `kit-log-dir.sh` |
| `session/` | `parse_transcript.py` `parse-transcript.sh` (the shared transcript parser + its tests) |
| *(root)* | `adopt.sh` `explain.sh` `pitch.sh` `precedent.sh`, orphans, no cluster |
Expand Down
4 changes: 2 additions & 2 deletions lib/board/board.sh
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@
# every ACTIVE mega under
# <repo-root>/_meta/megagoals/*/
# (>=1 unchecked sub-goal box), the
# `lib/mega.sh status <slug>
# `lib/mega/mega.sh status <slug>
# --rollup-only` line (roadmap-vs-git
# reconciliation, drift-flagged --
# never a re-render of the roadmap's
Expand Down Expand Up @@ -180,7 +180,7 @@ PARSE_BOARD_SH="$BOARD_DIR/parse-board.sh"
BOARD_MIRROR_SH="$BOARD_DIR/board-mirror.sh"
BOARD_WRITEBACK_SH="$BOARD_DIR/board-writeback.sh"
COCKPIT_PY="$(cd "$BOARD_DIR/.." && pwd)/sync/cockpit.py" # lib/sync/, the P2 sync-engine port (ID-290)
MEGA_SH="$(cd "$BOARD_DIR/.." && pwd)/mega.sh" # lib/mega.sh, one level up from lib/board/
MEGA_SH="$(cd "$BOARD_DIR/.." && pwd)/mega/mega.sh" # lib/mega/mega.sh, one level up from lib/board/

[ -f "$BACKLOG_SH" ] || { echo "board: lib/board/backlog.sh not found at $BACKLOG_SH" >&2; exit 1; }
[ -f "$PARSE_BOARD_SH" ] || { echo "board: lib/board/parse-board.sh not found at $PARSE_BOARD_SH" >&2; exit 1; }
Expand Down
4 changes: 2 additions & 2 deletions lib/config/module-registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ real reader consumes it today (all rows below are, except where noted).
| - | mega.merge_autonomy | `"gated-final"` | [design] | mega | `"gated-final"` or `"full-auto"`; no env override found. |
| - | mega.default_model | `"sonnet"` | [design] | mega | GLOBAL model fallback; real control is the per-sub-goal goal-file `Model:` field. Precedence: goal-file `Model:` > project cfg > this default. |
| - | mega.over_test | `false` | [design] | mega | GLOBAL scaffold-rigor default; real control is per-sub-goal `Done-mode: over-test` (SPEC-112). |
| MEGAGOALS_ROOT | env-only | (none) | [impl] | mega | Root dir where mega-goal folders live; unset falls through to further path resolution in `lib/mega.sh`. |
| MEGAGOALS_ROOT | env-only | (none) | [impl] | mega | Root dir where mega-goal folders live; unset falls through to further path resolution in `lib/mega/mega.sh`. |
| MEGA_MERGE_PR_INFO_CMD | env-only | (none) | [impl] | mega | Override the command used to fetch PR info at merge time; called directly when set. |
| MEGA_MERGE_GATE_LEDGER | env-only | `$LIB_ROOT/gate/gate-ledger.sh` | [impl] | mega | Which `gate-ledger.sh` `mega-merge.sh` shells out to. |
| MEGA_MERGE_GH | env-only | `gh` | [impl] | mega | Override the `gh` binary/wrapper used for PR ops at merge. |
Expand Down Expand Up @@ -281,7 +281,7 @@ against any of these bare tokens as covered without a registry row.
| KIT_DIR | `lib/plugin-check/tests/smoke.sh`: test-fixture scratch dir. |
| KIT_KNOWN_MODULES | `install.sh`: a hardcoded bash array literal, never read from the environment. |
| KIT_LIB | Script-local computed dir in most readers (e.g. `lib/telemetry/lane-telemetry.sh`); the real env-overridable cousin is `DWARVES_KIT_LIB` (Python, `lib/stats/src/stats/config.py`), which the bash-oriented seed regex cannot see (no `$` sigil in Python source) , documented here rather than silently dropped: see `lib/stats/README.md`'s own env table for `DWARVES_KIT_LIB`'s default (this repo's own `lib/`, kit-internal). |
| MEGA_SH | `lib/board/board.sh`: computed `$BOARD_DIR/../mega.sh`. |
| MEGA_SH | `lib/board/board.sh`: computed `$BOARD_DIR/../mega/mega.sh`. |
| PANE_VIEWER_ALLOWED | `lib/queue/orchestrate.sh`: a hardcoded allowlist string, not itself env-read; it validates `PANE_VIEWER`. |
| STATS_DB_REMOVED | Dead/vestigial test-fixture token, see its row above , no product reader exists. Kept OUT of the drift-fail set (registered above instead of silently dropped, per the scope fence) but also allowlisted so the lint does not double-count it as a live undocumented knob. |

Expand Down
File renamed without changes.
16 changes: 8 additions & 8 deletions lib/mega-review.py → lib/mega/mega-review.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
to this surface -- a projection, always safe to re-run, never cached):

1. ROADMAP.md truth -- this mega's sub-goal lines + each sub-goal's own goal file
(**Branch:** line). Reuses `lib/mega.sh status`'s OWN git-truth
(**Branch:** line). Reuses `lib/mega/mega.sh status`'s OWN git-truth
reconciliation (OK/CLAIM-UNVERIFIED/MERGED-UNCHECKED/STALLED/
WIP/PENDING/INFO) by shelling out to it and parsing its stable,
documented per-line format -- ONE classifier, not a second one
Expand Down Expand Up @@ -34,7 +34,7 @@
source renders "-" (honest-dash), NEVER a fabricated zero -- SPEC-197 Design.

Usage: mega-review.py <slug> --megagoals-root DIR --code-root DIR [--base BRANCH] [--out PATH]
Env: KIT_LOG_DIR the resolved durable ledger root (set by the lib/mega.sh `review` launcher,
Env: KIT_LOG_DIR the resolved durable ledger root (set by the lib/mega/mega.sh `review` launcher,
mirroring lib/gate/proof-table-gen.sh's own KIT_ROOT/KIT_LOG_DIR export convention -- this
script never re-resolves it itself, so there is one resolver, not two).
"""
Expand All @@ -50,14 +50,14 @@
import subprocess
import sys

_SELF_DIR = os.path.dirname(os.path.abspath(__file__)) # lib/ (mega.sh's own dir; orphan file)
_SELF_DIR = os.path.dirname(os.path.abspath(__file__)) # lib/mega/ (this file's own dir; siblings resolve one level up)


def _load_proof_table_gen():
"""Import lib/gate/proof-table-gen.py's `parse_ledger` + `_normalize_rid` directly (same
cross-subsystem sourcing convention gate-ledger.sh itself uses for lib/telemetry + lib/ledger
siblings) instead of re-implementing GATE/OUTCOME line parsing a second time."""
path = os.path.join(_SELF_DIR, "gate", "proof-table-gen.py")
path = os.path.join(_SELF_DIR, "..", "gate", "proof-table-gen.py")
spec = importlib.util.spec_from_file_location("proof_table_gen", path)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod) # type: ignore[union-attr]
Expand All @@ -80,7 +80,7 @@ def _run(argv, cwd=None):
return None


# ---- 1. ROADMAP truth + git-truth classification (delegates to lib/mega.sh status) -----------
# ---- 1. ROADMAP truth + git-truth classification (delegates to lib/mega/mega.sh status) -----------

def parse_roadmap(roadmap_path):
"""[(box, sub_slug, prose)] in file order. Honest-empty: no matching lines -> []."""
Expand All @@ -96,7 +96,7 @@ def parse_roadmap(roadmap_path):


def git_truth_status(mega_slug, megagoals_root, code_root, base):
"""Shells out to the ALREADY-SHIPPED `lib/mega.sh status` (one classifier, not a second one)
"""Shells out to the ALREADY-SHIPPED `lib/mega/mega.sh status` (one classifier, not a second one)
and parses its stable per-line format into {sub_slug: {label, pr, prstate, branch, commits,
openpr}}. Never fatal: a `mega.sh status` failure (e.g. no ROADMAP.md) degrades to an empty
map -- every sub-goal then renders with git-truth fields honest-dash, not a crash."""
Expand Down Expand Up @@ -300,7 +300,7 @@ def _learned_queued():


def _unpaid_debt_count(code_root):
wb = os.path.join(_SELF_DIR, "learn", "weekend-batch.sh")
wb = os.path.join(_SELF_DIR, "..", "learn", "weekend-batch.sh")
if not os.path.isfile(wb):
return None
res = _run(["bash", wb, "list", "--all-repos"], cwd=code_root)
Expand Down Expand Up @@ -523,7 +523,7 @@ def main(argv=None):
# lib/queue/orchestrate.sh's own locally-derived basename and a trusted local CLI
# invocation, never a webhook/network trigger -- but cheap and worth closing): a slug
# carrying a path separator could otherwise write REVIEW.html outside megagoals-root via
# os.path.join's own "absolute/.. wins" semantics. Same charset lib/mega.sh's ROADMAP
# os.path.join's own "absolute/.. wins" semantics. Same charset lib/mega/mega.sh's ROADMAP
# sub-goal regex already requires (`[0-9]+-[A-Za-z0-9_-]+`), loosened only to allow the
# mega SLUG shape (letters/digits/dot/underscore/dash, no leading number requirement).
if not re.match(r"^[A-Za-z0-9._-]+$", args.slug):
Expand Down
Loading
Loading