Skip to content

Milestone 2.1: add normalized runtime-analysis schema bridge after v0.77 #87

Description

@seonghobae

Buyer problem

The retained Milestone 2 source branch contains reproduced direct-PostgreSQL analysis outputs, but it cannot merge because it replaces the protected product architecture. The current product therefore lacks a normalized, migration-safe place to store real analysis runs, source-bound evidence, event/semantic graph outputs, evaluation items/calibration evidence and outbox delivery state without reintroducing a second application.

Ownership boundary

  • LineageWeave owns: product-visible analysis-run metadata and normalized projections used by Records, Lineage, Entity Catalog, Calendar/Reports and cited chat.
  • TEPP owns: semantic evidence measurement, multi-clock temporal/event contracts, multilevel/multiple-membership analysis and scientific artifacts, exposed through a versioned import/REST boundary.
  • contextual-orchestrator owns: provider/model routing and deep workflow execution, exposed through a fail-closed versioned task envelope.
  • Keyverse owns: identity; this slice adds no local IdP.
  • No service may read another service's application tables.

Required TDD slice

Start from exact protected post-PR-#74 main, not from codex/milestone-2-direct-postgres-product.

  1. Write failing PostgreSQL migration-contract tests that prove the new objects are absent.
  2. Extract only the useful analysis_* concepts from the retained branch; do not copy its product schema, compose, frontend, credentials, raw records or identifiers.
  3. Add new sequential migrations after the v0.77 migration chain.
  4. Normalize at least these responsibilities into separate two-or-more-word snake_case objects:
    • analysis run identity, status and immutable configuration digest;
    • source-document/evidence reference with exact source hash and location, not public source content;
    • model/prompt/provider/reasoning policy references;
    • event and semantic-graph artifact references with provenance;
    • evaluation criterion/item response and abstention-safe metric result;
    • report/calibration artifact reference;
    • transactional outbox delivery and idempotency evidence.
  5. Use foreign keys, unique constraints and explicit lookup tables instead of arrays or repeated text enums; maintain 3NF.
  6. Bind every analysis record to tenant/visibility authority already present in the product. Hidden evidence must not appear through broader run/report access.
  7. Provide forward migration, rollback rehearsal, second-run idempotency, orphan checks and source-to-target aggregate reconciliation.
  8. Keep real source table names, organization names, row identifiers, raw text, images, credentials and exact private counts out of public Git content.
  9. Add beginner-readable schema documentation, Mermaid ERD, ADR, traceability and CHANGELOG entry.

API boundary

This issue does not expose public CRUD endpoints yet. A subsequent slice may expose authorized read projections only after SQL authorization and hidden-evidence non-disclosure tests pass. There is no generic table or arbitrary-SQL API.

Acceptance evidence

  • Real ephemeral PostgreSQL migration tests from fresh install and sequential v0.77 upgrade.
  • Rollback/reapply and concurrent idempotency tests.
  • 100% changed owned production statement/branch coverage and public docstrings for migration/application helpers.
  • git diff --check, Python compile/lint, complete product suite, SAST and Security Scan.
  • Public-content denylist for private source/product names and raw identifiers.
  • Exact-head review, zero valid unresolved findings and protected merge.

Source evidence

Use codex/milestone-2-direct-postgres-product and closed PRs #77/#81 as read-only evidence only. Issue #79 remains the Milestone 2 parent. The DB-grounded Figma defines eventual product surfaces but does not authorize fields that the schema does not model.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions