feat: ship PROV-O actor lineage, accessible labels, and analysis-run evidence (v0.83.0) - #74
feat: ship PROV-O actor lineage, accessible labels, and analysis-run evidence (v0.83.0)#74seonghobae wants to merge 130 commits into
Conversation
Confirmed against real Milestone 2 SAP CRM VOC data: post_summary.py's
R&R extraction forced every named actor into a person slot, but real
business correspondence routinely names an organization acting in its
own name ("당사," "SEWA," "Siemens," "GECO"), not an individual.
- RoleResponsibility.actor_name (renamed from person_name) gains
actor_type_code (prov_person/prov_organization, W3C PROV-O grounded:
Lebo, Sahoo, & McGuinness, 2013) and an LLM-inferred
affiliated_organization_name for person actors -- a bare name
without an employer is hard to place.
- Ontology: :RoleActorPerson rdfs:subClassOf prov:Person,
:RoleActorOrganization rdfs:subClassOf prov:Organization -- genuine
subclasses of the real external PROV-O classes, distinct from the
ontology's existing :Person (a cataloged Keyman with a stable
person_id; an R&R actor is a free-text name with no cataloged
identity).
- migrations/0012_role_responsibility_agent_type.sql renames the
column via RENAME COLUMN (preserves existing rows), not a
drop/recreate.
- Popup R&R list shows a Person/Organization badge and the inferred
affiliation; only a person actor still links to the Keyman panel.
- Also fixes a real deployment gap found via browser E2E testing:
migrations 0005-0011 had accumulated on main without ever being
applied to the long-running demo Postgres volume, surfacing as
CORS-looking failures (missing-table 500s lose their CORS header)
on Evaluate, Reports, Summary, and Chat.
ADR 0006.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 Walkthrough<hidden_range_assignment> 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Drop real-organization names from docs, prompts, and comments. Seed a synthetic organization actor so the Person/Organization badge is visible without a live LLM, and reject unknown actor_type_code values.
PersonMention now carries an optional job_title extracted by the LLM from role phrasing (e.g. "our legal counsel, Sam Okonkwo"), not just named affiliations. cataloged_person.last_known_job_title persists it, and _upsert_person treats a conflicting stated title as evidence that a same-name match is a different real person rather than a re-mention, so two "Kim Cheolsu"s with different titles get distinct person rows. Keyman panel renders the title next to the person and per-affiliation role_title, which existed in the schema but was never surfaced before. Migration 0013 adds the column additively; 0001_initial_schema.sql bakes it in for fresh installs, matching this repo's existing pattern.
After make seed, Ada West / Priya Nair / Jordan Hale carry last_known_job_title so the new title chip is visible without a live extraction.
seonghobae
changed the title
Strix flagged the local-dev password literal in seed_demo_data.py after this branch started editing that file. make seed still injects the compose default; a direct script run requires KEYCLOAK_ADMIN_PASSWORD.
…70.0)
Real post text named a company sub-unit ("설계팀"/design team) that
neither ADR 0006's prov_person nor prov_organization fits -- it's part
of a company, not a person and not the company itself. actor_type_code
gains prov_team, grounded in the W3C Organization Ontology's
org:OrganizationalUnit (Reynolds, 2014), a different W3C vocabulary
from PROV-O that exists specifically for this meso-level case.
A team actor requires affiliated_organization_name in the same way a
person actor does -- unlike an organization actor, a team's own name
never answers "which company." Fixed a real bug the new type surfaced:
the R&R badge's label text was a binary Person/Organization ternary
that would have mislabeled a team as "Organization" (the CSS class
name was already generic; the display text was not).
Migration 0014 is purely additive (one lookup row insert), no schema
change -- actor_type_code already stores an arbitrary FK'd code.
Real post text names organizations by abbreviation ("한수원" for
"한국수력원자력") that corporate_hierarchy_resolution's character-
similarity matching cannot bridge -- an initialism shares almost no
substring with its expansion, so no similarity threshold recovers it.
New lineageweave/organization_name_resolution.py: an LLM proposes the
full name from context (or declines with UNKNOWN), then the *existing*
relation_verification Searxng client cross-verifies the specific raw/
resolved pairing -- no second web-search integration built, reusing
what this repo already has for a structurally identical problem. Only
a search-corroborated resolution is ever substituted in for
resolve_corporate_entity; an unresolved or unverified name still flows
through unchanged, same never-trust-an-unverified-guess discipline as
every other channel here.
Cached in a new organization_name_resolution table
(migrations/0015), keyed by the raw name so the same abbreviation
across many posts is resolved once, not re-queried every mention.
Grounded in SKOS skos:altLabel/skos:prefLabel (Miles & Bechhofer, 2009).
Wired into backend/app/keyman_ingestion.py's affiliation loop and the
private real-data batch script's paced re-implementation of it -- which
was also found missing role_title persistence entirely (a stale copy
predating that feature), fixed alongside this.
Known, documented gap (ADR 0008): the same request's entity-
relationship classification step still uses the raw, unresolved
organization names -- not fixed here, tracked honestly instead of
silently shipped as if both sides already agreed.
…atch (v0.72.0) _parse_description required a single regex to match TEXT/CAPTION/TAGS in that exact order in one pass. Reproduced live against real embedded images from the Milestone 2 batch: real vision responses with the content right but the formatting only mostly right (bolded labels, reordered labels, a missing TAGS line) were rejected wholesale, producing the same "[image: content unavailable]" placeholder as a genuinely unconfigured vision channel -- discarding real, already-paid-for content, not a "genuinely could not get it" case. Each label is now parsed independently by scanning lines for a TEXT:/CAPTION:/TAGS: prefix (tolerant of markdown emphasis and any order); only a response with neither TEXT nor CAPTION content raises ImageDescriptionParseError. Multi-line TEXT (real multi-line OCR output) is still preserved with real newlines, not flattened.
…ity-agent-ontology # Conflicts: # CHANGELOG.md # frontend/package.json # lineageweave/__init__.py # pyproject.toml
…v0.74.0)
Extraction runs per-post; a team or organization's identity did not
survive across posts the way a Keyman's already did via
cataloged_person -- "설계팀" named in ten posts was ten unrelated
strings, not one entity the KG could link through. Extraction results
must themselves become cross-post lineage clues, not just per-post
artifacts.
New cataloged_team catalog (migrations/0016), identity key (team_name,
affiliated_organization_name) since a bare team name is not by itself
identifying ("설계팀" exists at many real companies) -- reuses the same
resolve_corporate_entity matching Keyman affiliations already use for
the team's parent org, not a second algorithm. An organization actor
resolves against the existing corporate_entity catalog directly, no
new table needed.
knowledge_graph_edges_for_post gains three new edge kinds
(edge_mention_team, edge_team_affiliation, edge_mention_organization)
as distinct object properties, not widened domain/range on the
existing :mentions (which would let RDFS entail every :mentions
subject is both a person and a team). persist_post_summary now
resolves each R&R actor's identity and calls the same
persist_edges_for_post Keyman ingestion already uses -- one function
computes a post's whole edge set regardless of trigger.
A person R&R actor is opportunistically joined to an existing
cataloged_person row by name, never originated by R&R itself --
documented as a real, deliberate gap in ADR 0009 (cataloged_person
needs person_side_code, which R&R's prompt does not currently ask
for), not silently half-done.
Pull request was converted to draft
…y (v0.75.0) corporate_hierarchy_resolution's similarity matching only ever finds an ALREADY-cataloged corporate_entity -- it has no path to create one. Real Milestone 2 data confirmed the actual consequence: 0 of 4,154 person_affiliation rows and 0 of 9,852 R&R organization-actor mentions ever resolved, because corporate_entity for the real dataset only holds the employer's own 2-row hierarchy. The standing "통합 고객사 계열 tree AI" requirement (Samsung -> Samsung Electronics Korea -> ...) was never actually populated for real extraction. New lineageweave/corporate_hierarchy_inference.py: an LLM proposes a Group/Company/Plant placement (level + parent name) from the post's own text, or declines with UNKNOWN. New backend/app/corporate_entity_ingestion.py's get_or_create_corporate_entity tries similarity matching first (unchanged), then only creates a real new row once the proposal is corroborated by the *existing* relation_verification Searxng client -- no new search integration, reusing the same reused-verification-client pattern ADR 0008 already established. Recurses up a bounded (4-level) parent chain so the whole hierarchy gets real parent_entity_id links, not an orphaned row. Auto-created corporate_entity_code values are AUTO-<hash>-prefixed -- that column doubles as the real login corp-code Keycloak claim, so an auto-created counterparty must never collide with that namespace. Wired into both existing organization-resolution call sites (keyman_ingestion.py's affiliation loop, post_summary_ingestion.py's R&R organization-actor loop) rather than a third path, so both routes to corporate_entity share one creation policy. Found and fixed a pre-existing gap in backend/tests/test_api.py's seeded_db fixture along the way: it never seeded the 'plant' corporate_entity_level lookup row.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
#74 already contains ADR 0021 / v0.88.0. Keep that start path and follow with a purge-aware 0023 so a granted empty still works after edges exist. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review of 1cf0bd7 only (parent 69c035b)
Authorized start (ADR 0021 / v0.88.0) now lives on this PR. Open a Pending Demo Corp lineage row, then Start reconstruction. The designed A-100 fork (revised quote and delivery question under the pricing follow-up) is the acceptance tree. TEPP and period-report stay 422. This path does not invent a theta.
This commit closes the five start defects from #142 / #167
FOR UPDATEof the run row happens before Running.UniqueViolationis 409, not 500. The HTTP handler wraps the write in one transaction, so a crash after Running rolls back to Pending.- Create freezes
analysis_source_snapshot_member. Start prefers that bag over a later cutoff re-query. - Detail renders the Result digest prefix next to Code and Config; hover reads the parent-choice hash.
fetch_reconstructed_edgesapplies the same public-or-affiliated title rule as cutoff posts.- CI A-100 goes through
records_from_source_posts, not only librarysample_records(). The live API test (self-skip without Keycloak) recovers the revised-quote / delivery-question fork.
Prefer this head over #142, #152, and #167 for in-process start. Do not open another start-reconstruction PR unless 1cf0bd7 regresses.
Residuals (not merge blockers for this slice)
- The live start test inserts fixture posts before create, so a post-create backfill with
created_at <= cutoffis not proven excluded. Implementation still prefers member rows when the table has them. analysis_run_lineage_edge.reconstructed_atrepeats the parent reconstruction clock (3NF residual).- Failed lineage still has no Start button; retry remains a new Request (new snapshot), which matches ADR 0021's Pending-only start.
- Home-list AccName, person-catalog
LIMIT 1, live write-clock, and default-home fixture stay on their existing landings (#188, #179, #176, #186). Those successors must not reuse 0.88.0 / ADR 0021 now that this head occupies both. #186 is already 0.88.1.
Do not
- Treat this COMMENT as independent approval of the 120-commit PR.
- Merge #74 from this review.
- Self-approve.
- Invent a theta.
- Merge #142 / #152 / #167 in parallel.
Sent by Cursor Automation: Fix Issues
#193) Analysis-run detail marks in-cutoff titles rewritten after the run. Demo public post is the edited counter-example; Demo private post still matches the January cutoff. Bodies stay live.
…194) Opening a marked analysis-run title still shows the live post. The popup now says to compare it with this run instead of inventing a cutoff snapshot.
….0) (#195) * feat: open reconstructed analysis-run edges as live posts (v0.91.0) After start, the titled A-100 parent and child are buttons. A marked child still shows the live-body warning. The popup does not invent a cutoff snapshot. * feat: start pending TEPP measurement through tepp_client (v0.92.0) POST /api/analysis-runs/{id}/start submits AnalysisRunRequest via tepp_client. A missing or refused transport stays Failed. An accepted envelope is not persistable yet. Period-report remains 422. No theta is invented.
…4.0) (#196) * feat: seed the designed A-100 fork on the Demo Corp lineage run (v0.93.0) make seed already stamped Succeeded. The home detail now persists ThreadWeave's parent choices so open-after-seed shows the revised quote and delivery question. No TEPP theta is invented. * feat: persist start work on a durable outbox (v0.94.0) Start commits Running plus one outbox row, wakes Valkey, then delivers ThreadWeave or tepp_client. A crash no longer loses the work item. No TEPP theta is invented.
| import hashlib | ||
| import json | ||
| from datetime import datetime, timezone | ||
| from typing import Any |
…0.96.0) (#197) * feat: show labeled outbox delivery on analysis-run detail (v0.95.0) Open the seeded Demo Corp lineage run to see Claimed then Delivered. Stream entry ids stay off the payload. No TEPP theta is invented. * feat: seed the Demo Corp period report on the analysis-run registry (v0.96.0) After make seed, open Period report · Succeeded · Demo Corp to confirm the cutoff posts. Mean θ stays on the period-report panel. Start stays 422. No TEPP theta is invented.
#198) * feat: open the scored week from a period-report analysis run (v0.97.0) After make seed, open Period report · Succeeded · Demo Corp and click Open period report 2026-W02. Mean θ stays on the report panel. No TEPP theta is invented. * fix(ui): focus the opened week and keep failed report rows closed Open period report 2026-W02 now focuses the report period field. Failed rows stay without the button. Seed still writes scope_key 2026-W02 and never a theta.
* feat: open the corp grouping with the scored week (v0.98.0) Open period report 2026-W02 from a corporate-entity analysis run also switches Report grouping to Corporate entity. Mean θ stays on the report panel. No TEPP theta is invented. * fix(ui): name the opened Demo Corp grouping Open period report 2026-W02 now marks Demo Corp current and shows that label instead of a UUID. The persisted scope grouping key is the corporate entity, never the week or a theta.
….98.1) (#200) List and detail queries no longer format a WHERE fragment. The $1 / $2 / $3 binds are unchanged. Semgrep no longer treats the predicate as user-concatenated SQL.
…#201) When the operator is already on 2026-W02, Open period report focuses the Demo Corp comparison chip instead of the unchanged period field.
3 participants
Buyer impact
Ships the reviewed LineageWeave product through v0.83.0 with three connected buyer-visible capabilities:
Our sideandCounterparty; andmake seed, Demo Analyst seesLineage reconstruction · Succeeded · Demo Corp. Another tenant's run 404s and never appears in the list.The v0.78.0 accessibility slice was reviewed and merged as stacked PR #80. Current
main(c93d449, merged PR #91 adaptive orchestration defaults) is on this branch. Duplicate PR #96 was closed unmerged. This PR does not create a second application or database authority.Product and data contracts
knowledge_graph_edgeas a buyer navigation projection while normalizedprovenance_*tables retain semantic truth and derivation evidence.0016upgrade copies R&R names and leaves Keymanmention_contextin place.migrations/0018_analysis_run_registry.sql. Reads are SQL-scoped; payloads are labels and aggregates only.automode (ADR 0015); explicit checkedverifypaths remain unchanged.snake_casedatabase objects.Accessibility and next action
Related-node person chips use the localized business-side label supplied by the authorized API. The same caption is exposed in the button accessible name, so a screen reader announces
Related nodes for Priya Nair (Counterparty)rather than the name alone. Home analysis-run rows are buttons that open the authorized detail.Documentation and release
Exact-head verification
Exact head:
88a1a0f8b925206083884f5350f7bbab41433fa7Base:
main@c93d44978c5ab724d591753127e8ff0b8b7a823231955059893); prior head59cde23was green (31952379160)31955059986); prior head59cde23was green (31952379253)31955059866); prior head59cde23was green (31952379129)31955059900); prior head59cde23was green (31952379237)31955058003); prior head59cde23was green (31952378323, org backend-unavailable skip after a 0-finding fallback scan; see PR comments)The branch is intentionally waiting on the independent approval gate and green exact-head required checks. No earlier review, stacked-base result, local verifier, or owner-authored comment substitutes for that approval. This update does not merge the PR.
Standards references
Lebo, T., Sahoo, S., McGuinness, D., Belhajjame, K., Cheney, J., Corsar, D., Garijo, D., Soiland-Reyes, S., Zednik, S., & Zhao, J. (2013). PROV-O: The PROV ontology. World Wide Web Consortium.
Reynolds, D. (Ed.). (2014). The organization ontology. World Wide Web Consortium.
Miles, A., & Bechhofer, S. (Eds.). (2009). SKOS simple knowledge organization system reference. World Wide Web Consortium.
Summary by CodeRabbit
새로운 기능
보안 및 안정성
문서화