Milestone 2: port verified direct-PostgreSQL analysis into the protected product architecture

[seonghobae]

Buyer problem

PR #77 contains valuable direct-PostgreSQL analysis and browser/product evidence, but it is a parallel repository replacement that deletes the protected package, migrations, ADRs, ontology profile, license, and reviewed v0.77 lineage. The evidence must be preserved without merging that destructive history.

Source evidence to preserve

Use codex/milestone-2-direct-postgres-product as a read-only source branch. Preserve only claims that can be reproduced from aggregate, privacy-safe evidence and exact source hashes. The historical run records report bounded direct-database analysis, event/semantic graph persistence, report evaluation/calibration, Valkey outbox behavior, browser UX traversal, and evidence-scoped APIs. Do not copy raw records, source identifiers, base64/image bytes, credentials, organization-specific names, or source-table names into the public repository.

Required additive delivery slices

1. Runtime schema bridge

   • Map the verified analysis_* objects into new migrations after the current v0.77 schema.
   • Keep every database object two-or-more-word snake_case and third-normalized.
   • Provide forward migration, rollback evidence, source-to-target reconciliation, and idempotency.

2. TEPP measurement port

   • Call TEPP through its versioned import/REST boundary; do not fork or duplicate TEPP arithmetic.
   • Preserve multi-clock evidence, event/document/entity provenance, temporal leakage controls, cross-classified multiple membership, and uncertainty.

3. contextual-orchestrator port

   • Keep a versioned fail-closed REST adapter until reviewed upstream main exposes the required portable task envelope.
   • Route simple extraction to bounded single-model work and complex ontology/report/multimodal work to deep thinker/worker/verifier/synthesizer workflows with explicit recursion, access lists, reasoning effort, prompt/model hashes, and ablation evidence.

4. Product UI

   • Port the validated React/PostgreSQL customer, workspace, document popup, source drawer, event DAG, Keyman, issue/todo/calendar, report, and KG surfaces into the existing frontend rather than adding a second web application.
   • Maintain the approved Figma information architecture and automated browser traversal.

5. Identity and authorization

   • Production compose must bind to real Keyverse. No Keyverse-shaped compose identity service, default account, default client secret, or synthetic production claims.
   • Test doubles belong under test-only fixtures and may never be routable from product compose.
   • Enforce actor-scoped ABAC/RBAC in SQL/API responses and prove hidden-evidence non-disclosure.

6. Actual-data acceptance

   • Run read-only source profiling first, then write only normalized analysis outputs in a separately governed schema.
   • Reconcile source row/document/thread counts, graph node/edge counts, report/item/calibration rows, outbox delivery, and orphan checks.
   • Store an aggregate-only signed manifest outside public source control; public documentation may contain only non-sensitive aggregate acceptance ranges and reproducibility metadata.

Merge gates

• Start from the post-feat: ship PROV-O actor lineage, accessible labels, and analysis-run evidence (v0.83.0) #74 protected head and preserve monotonic versioning.
• Use the committed universal lock and supported Python/Rust/Node toolchains.
• 100% production statement and branch coverage for changed production code; public API docstrings complete.
• Exact-head Python, PostgreSQL, React, browser, SAST, security, migration, and public-content boundary checks.
• No unaccounted test skips; GPU/LLM live lanes use NVIDIA_NIM_API_KEY, never COPILOT_GITHUB_TOKEN.
• Independent current-head review and protected auto-merge.

Supersession

Closing PR #77 after creating this issue does not discard its source branch. It removes an unsafe merge candidate while retaining a reproducible source for bounded additive ports.

[seonghobae]

Canonical Milestone 2 source and ownership handoff

Fresh queue cleanup established the following immutable source-evidence boundaries:

• closed PR feat: publish direct-PostgreSQL LineageWeave product #77 branch codex/milestone-2-direct-postgres-product remains read-only at 96116aa4c44d0fe5dd2515b2e348a75f8f025a48;
• closed PR chore: retain milestone-2 audit follow-up as read-only source evidence #81 branch codex/surface-audit-continue-mainbase is a one-commit descendant at dc8b536e42756011301c78762a20cf430906c772; its run-audit and abstention-safe parser deltas must be isolated rather than reopening the 235-file repository replacement;
• closed PR feat: retain semantic-span prototype as TEPP implementation evidence #82 semantic-span prototype is retained as test/algorithm evidence only; TEPP PR feat: retain analysis-run registry prototype as superseded evidence #83 owns the canonical immutable source-span, model-profile token-budget, hierarchy, and embedding-space contract.

First additive implementation lane after #74

The next canonical PR must start from the exact protected result of #74 and implement only the runtime schema bridge:

1. inventory the analysis_* objects from the read-only feat: publish direct-PostgreSQL LineageWeave product #77 branch;
2. classify each object as reusable, renamed, split, rejected, or already represented by the post-feat: ship PROV-O actor lineage, accessible labels, and analysis-run evidence (v0.83.0) #74 schema;
3. allocate new sequential migrations without replacing existing migrations or package history;
4. retain source evidence, event/document/entity provenance, model/prompt hashes, temporal clocks, uncertainty, run status, and outbox state in third-normalized tables;
5. provide forward migration, downgrade/rollback evidence, idempotent replay, orphan checks, and source-to-target aggregate reconciliation;
6. add no second React application, no Keyverse-shaped compose identity service, no public raw-data fixture, and no direct cross-service table access.

Adapter sequence

After the schema bridge is protected-main truth:

• TEPP is consumed through its versioned import/REST boundary; LineageWeave does not duplicate semantic-unit or psychometric arithmetic.
• contextual-orchestrator remains a fail-closed versioned REST adapter until the required task envelope is on reviewed upstream main.
• the existing React frontend receives the document/customer/workspace/DAG/report/admin vertical slices incrementally.
• actual-data acceptance writes only aggregate signed manifests outside public source control and never commits organization-specific names, source-table identifiers, raw records, base64 payloads, credentials, or source identifiers.

This comment is the continuation pointer for the next bounded TDD branch; it does not promote any closed-branch behavior to shipped product truth.

[seonghobae]

Canonical Milestone 2 delivery update — corrected 2026-08-16

The runtime-schema lane now has one canonical merge candidate: PR #89 (feat/analysis-run-registry-v079-clean). It is deliberately Draft and stacked directly on #74.

The earlier pointer to #83 is superseded. #83, #84, #85, #86, and the broader #88 prototype remain closed branch evidence only; none may be reopened as a competing migration 0018 or ADR 0013 authority.

Current bounded contract on #89 includes:

• immutable source captures and aggregate count evidence without public raw records;
• snapshot-owned maximum_available_time and run-owned knowledge_cutoff, enforcing maximum_available_time <= knowledge_cutoff;
• real user_account requester identity and requester-scoped idempotency;
• immutable run requests and append-only legal lifecycle evidence;
• one scope shape per run and a derived current-status view;
• forward and fail-closed rollback migrations;
• real-PostgreSQL, naming, idempotency, temporal-leakage, lifecycle, and rollback tests;
• APA 7 doctoring and explicit TEPP/contextual-orchestrator ownership boundaries.

Dependency order:

1. complete feat: ship PROV-O actor lineage, accessible labels, and analysis-run evidence (v0.83.0) #74's source-aware person-projection repair and protected integration;
2. complete feat: show related-person business-side labels (v0.78.0) #80's accessible business-side label repair, retarget to protected main, and integrate;
3. rebase or rebuild feat: add normalized analysis-run registry (Milestone 2 slice 1) #89 on the exact protected result, then run all PostgreSQL/product/security/review gates;
4. port the source-redacting repository/API and service-run references as a separate slice using the preserved closed branches as read-only evidence;
5. add the DB-grounded Analysis Run Console to the existing React product after its API contract is protected;
6. consume TEPP and contextual-orchestrator only through reviewed versioned boundaries;
7. execute private aggregate-only actual-data acceptance outside public source control.

This correction removes competing ownership without discarding the useful closed-branch evidence.

[seonghobae]

Continuation (2026-08-16 hourly loop)

Protected main is still v0.71.0. #92 (0.72.0 person chips + stacked #93 0.73.0 org chips) is auto-merge + review-blocked; ruleset merge of #74/#91 returned policy prohibition — no admin retry.

Next buyer-felt #77 port is PR #95 (feat/analysis-run-authorized-read, v0.79.0), stacked on #89:

• authorized GET /api/analysis-runs + home-page Analysis runs panel
• SQL scope; hidden tenant runs 404
• labels + aggregate counts only (3 synthetic documents after make seed)
• no AGENTS.md rewrite, no real-org data, TEPP unwired except existing tepp_client

Do not merge #95 or #89 ahead of #74. #77 stays closed.