Skip to content

The graph exports as RDF, with its intervals and its evidence - #320

Merged
WaylandYang merged 2 commits into
devfrom
feat/an-auditor-reads-it-without-us
Sep 4, 2026
Merged

WaylandYang merged 2 commits into
devfrom
feat/an-auditor-reads-it-without-us

Conversation

@WaylandYang

Copy link
Copy Markdown
Contributor

Closes #308. Design in 0020.

owl_import.rs read OWL in and five packs shipped inside the binary; nothing went the other way. GET /api/v1/kbs/{id}/export?format=turtle|jsonld now streams a whole base as RDF — the vocabulary, the entities, every fact with both its clocks, the evidence behind each one, and the derivations with their premises.

What a statement looks like

Real output, from a base whose only content arrived through remember over MCP:

<urn:utopia:kb:01a0…7441:fact:01a0…4973> a rdf:Statement ;
	rdf:subject <urn:utopia:kb:01a0…7441:entity:01a0…f68a> ;
	utopia:proposedPredicate "previously_managed" ;
	rdf:object <urn:utopia:kb:01a0…7441:entity:01a0…0da6> ;
	schema:validThrough "2026-04-15"^^xsd:date ;
	prov:generatedAtTime "2026-09-04T19:40:56Z"^^xsd:dateTime ;
	utopia:confidence 0.90 ;
	prov:wasDerivedFrom <urn:utopia:kb:01a0…7441:document:01a0…d427> ;
	utopia:quote "[2026-04-15 12:00] Lin Zhao took over Project Aurora from Wei Chen…" .

That one block carries the four things a REST response could not: the interval, the moment we came to believe it, the document it came from, and the sentence itself. Note what is missing — rdf:predicate. The ontology never accepted previously_managed, so no predicate is minted for it (0010); the model's wording rides as a literal where a reader can see that the vocabulary has a hole.

The choices the record argues

  • Imported classes keep their IRI. A schema.org base exports schema:Organization. Home-grown ones are minted from their key, so the file stays readable.
  • Reification, not RDF-star or named graphs. The natural shapes are the ones most consumers cannot open, and a file nobody can load is not an export.
  • Standard terms wherever they exist. prov:invalidatedAtTime is exactly our recording axis; schema:validFrom / validThrough are the world axis. Three terms have no standard spelling and get a minted one under urn:utopia:ns: — confidence, the derived flag, and the proposed predicate.
  • Only facts held now and valid now get a plain triple, so a consumer that ignores reification reads a correct present tense instead of "Zhang San still runs the project he handed over in 2024". Derived facts get no plain triple at all: an inference should not arrive looking like something a document said.
  • A year stays a year. valid_from_precision maps onto xsd:gYear / xsd:gYearMonth / xsd:date, and "ended, date unknown" exports as utopia:endedUnknown rather than being flattened into "still true".

Identity defaults to a URN and ?base=https://…/ mints HTTP IRIs for a deployment that publishes at a known address. Stability over dereferenceability: two exports a year apart have to line up, and a self-hosted instance does not know its own public URL.

Streaming

A base with a hundred thousand facts is exactly the base someone wants exported, and exactly the one that dies if the file is assembled in memory first. Every source is paged by id and serialised into a buffer that is drained into the response after each page. The cost is honest and stated in the module header: an error mid-file truncates it, because the headers went out long ago.

Verified

Eleven unit tests in rdf.rs that parse the output back rather than grepping it, since the only thing an export must guarantee is that it loads: imported IRIs survive; a live fact is both a statement and a triple; a closed interval and a retracted row are statements only; a year stays xsd:gYear; "ended, date unknown" says so; no predicate is invented; an attribute carries its datatype; a derivation names its rule and premises; evidence points back at the document and the sentence; JSON-LD and Turtle yield the same triple count; a bad base is refused.

an_export_carries_the_whole_ledger.rs, database-backed, asserts the queries do not inherit the interface's filters: the retracted fact is there, the deleted document is there, the ontology-less wording is there, the premise is there — and the merged-away entity is the one thing that is gone.

End to end against a running deployment: confirmed two pending facts, exported, and got text/turtle with Content-Disposition: attachment; filename="general.ttl", prefixes, classes with their descriptions, statements with intervals and lineage, documents as prov:Entity; the JSON-LD form parses as twelve node objects; ?base= mints HTTP IRIs; ?format=n3 is refused with 422.

Full utopia-store suite (40 binaries) green against a freshly migrated database with UTOPIA_TEST_REQUIRE_DB=1, 142 server unit tests green, cargo fmt --check and cargo clippy --all-targets clean.

Not included

A download button, a docs page, and a SPARQL endpoint — the record says why each waits. oxrdfio is one new dependency; oxrdf and oxttl were already in the tree for the import side.

🤖 Generated with Claude Code

WaylandYang and others added 2 commits September 5, 2026 03:44
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: WaylandYang <wayland0916@gmail.com>
@WaylandYang
WaylandYang merged commit 54e66a6 into dev Sep 4, 2026
@WaylandYang
WaylandYang deleted the feat/an-auditor-reads-it-without-us branch September 4, 2026 23:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The graph imports OWL and exports no RDF

1 participant