feat(content): add lossless authored core

.gitattributes

@@ -8,5 +8,8 @@ tools/package-release.sh export-ignore
 /maps/** text eol=lf
 /prototypes/authored-syntax-v1/** text eol=lf
 /schemas/authored-content-v1/** text eol=lf
+/schemas/content-core-v1/** text eol=lf
+/tools/atrinik-content text eol=lf
+/tools/content_core/** text eol=lf
 /tools/content_schema/** text eol=lf
 /tools/syntax_evaluation/** text eol=lf

.github/workflows/check.yml

@@ -53,3 +53,6 @@ jobs:
 
       - name: Verify authoritative schema projections
         run: python -m unittest tools.tests.test_content_schema
+
+      - name: Exercise lossless content core and safe transactions
+        run: python -m unittest tools.tests.test_content_core

AGENTS.md

@@ -27,14 +27,22 @@
   prototype implementation digest, committed measurement evidence, decision
   text, and Linux/Windows tests synchronized. Do not promote the neutral
   physical-record comparison model into the final typed schema.
+- `tools/content_core` is the production lossless parser, typed source view,
+  project index, and targeted writer for legacy maps and archetypes. New content
+  analyzers and editors must consume it instead of adding another ADS parser.
+  Keep `schemas/content-core-v1`, CLI output, transaction preconditions, safety
+  rules, Linux/Windows tests, and `docs/CONTENT_CORE.md` synchronized. Writes are
+  dry-run-first and limited to authored source roots and `arch/*.arc` or `maps/`
+  targets; never weaken digest/fingerprint checks or output/runtime refusals.
 - Trace every changed map path, archetype, animation, image, artifact, treasure,
   faction, interface, and script reference. Do not mask missing references with
   absolute paths, generated placeholders, or duplicated parsers.
 - Generated runtime collection belongs under `build/` or another isolated
   output directory, never in source. Do not overwrite mutable server state.
 - `tools/world_content_audit.py` is a read-only exploratory report. It may reveal
   review targets but never replaces `tools/validate.py` or the catalog, and its
-  output is not generated source.
+  output is not generated source. Its map/archetype traversal must use the
+  common lossless core.
 - Run `python3 -m tools.content_contracts validate --root .`,
   `python3 -m tools.syntax_evaluation --root .`,
   `python3 -m tools.content_schema validate --root .`,

README.md

@@ -42,6 +42,12 @@ and the lossless parity corpus live under `contracts/content-v1/`. See
 authoritative server sources, compatibility rules, fixture coverage, and
 read-only inspection command.
 
+`tools/content_core` is the production byte-lossless legacy ADS core. Its
+versioned `tools/atrinik-content` CLI provides bounded inspection, validation,
+semantic comparison, catalog search, and dry-run-first primitive transactions.
+See [`docs/CONTENT_CORE.md`](docs/CONTENT_CORE.md) for the JSON contracts,
+preconditions, safety boundary, exit codes, and examples.
+
 The accepted future authored surface is a strict, bounded JSONC dialect. See
 [`docs/AUTHORED_SYNTAX_DECISION.md`](docs/AUTHORED_SYNTAX_DECISION.md) for the
 decision, parser limits, cross-language implementation constraints, raw

docs/ARCHITECTURE.md

@@ -11,9 +11,11 @@ The source-to-runtime flow is:
    cross-references;
 3. `schemas/authored-content-v1/source.json` types the closed field surface and
    generates shared loader/compiler, editor, and logical-document metadata;
-4. `tools/validate.py` runs catalog, schema, grammar-contract, corpus, collection,
+4. `tools/content_core` provides the byte-lossless production parser, typed
+   source-located views, project index, and transaction-safe targeted writer;
+5. `tools/validate.py` runs catalog, schema, grammar-contract, corpus, collection,
    and licensing checks without modifying authored sources; and
-5. `tools/build_runtime.py` creates an isolated runtime tree and digest manifest
+6. `tools/build_runtime.py` creates an isolated runtime tree and digest manifest
    below `build/` or another explicit output path.
 
 Stable identity ownership and rename/removal policy are documented in
@@ -22,8 +24,9 @@ and load-mode inventory, producer/consumer survey, versioned interchange schemas
 and byte-preserving parity corpus are documented in
 [`CONTENT_GRAMMAR_CONTRACTS.md`](CONTENT_GRAMMAR_CONTRACTS.md). Those contracts
 characterize the external server and checker boundaries that a future shared
-lossless implementation must satisfy; they do not transfer implementation
-ownership into this repository.
+lossless implementation must satisfy. The production implementation now lives
+in `tools/content_core`; the characterization inspector remains a parity oracle
+and is not imported by that core.
 
 The authoritative typed field and logical-document contract is documented in
 [`CONTENT_SCHEMA.md`](CONTENT_SCHEMA.md). Its declarative source is checked
@@ -40,11 +43,13 @@ comment/span requirements, representative measurements, and cross-language
 adapter constraints are authoritative inputs to schema and lossless-core work.
 The syntax prototypes and physical-record model remain evaluation/migration
 oracles; they are not another production parser or content IR. The generated
-logical schema defines the eventual typed interchange shape, while the future
-lossless core owns production parsing and rewriting.
+logical schema defines the typed interchange shape, while the lossless core owns
+production parsing and rewriting. Its CLI and transaction contracts are
+documented in [`CONTENT_CORE.md`](CONTENT_CORE.md).
 
 Generated runtime files are outputs, never authored sources. Exploratory reports
 from `tools/world_content_audit.py` are review artifacts, not another identity or
-grammar authority. A new loader, writer, checker, collector, or analyzer must use
-the existing catalog and versioned contract boundaries instead of introducing a
+grammar authority. Its map and archetype traversal uses the common core. A new
+loader, writer, checker, collector, or analyzer must use the existing catalog,
+lossless core, and versioned contract boundaries instead of introducing a
 duplicate parser or inventory.

docs/CONTENT_CORE.md

@@ -0,0 +1,92 @@
+# Lossless content core
+
+`tools/content_core` is the production parser and targeted writer for legacy
+Atrinik maps and archetypes. It keeps the original bytes as the serialization
+authority while exposing typed, source-located semantic views derived from
+`schemas/authored-content-v1/field-metadata.json`. It does not import the legacy
+contract inspector or either syntax prototype; those remain independent parity
+oracles.
+
+## Guarantees and limits
+
+An unchanged document serializes to the exact input bytes, including comments,
+custom records, message bodies, nesting, multipart separators, record order,
+line endings, trailing whitespace, and a missing final newline. Every node and
+property has a half-open byte span. Standard property values are typed from the
+authoritative field metadata; extension records retain a stable custom ID and
+their raw value.
+
+Input is strict UTF-8 without NUL. The shared fail-closed limits bound source
+bytes, lines, line length, comments, message bodies, properties, nodes, and
+nesting. The parser reports legacy compatibility diagnostics and additional
+typed-value or schema-limit failures. A document is valid only when it has no
+error-severity diagnostics.
+
+`ProjectIndex` caches parsed documents by modification time and size, supports
+explicit invalidation, and exposes deterministic lookup through the existing
+stable content catalog. It does not create a second asset or identity index.
+
+## Headless CLI
+
+The CLI contract version is reported by `--version`. Machine output is
+deterministic JSON when `--json` is present.
+
+```sh
+tools/atrinik-content --version
+tools/atrinik-content --root . inspect maps/hall_of_dms --json
+tools/atrinik-content --root . validate arch/indoor/quill_pen.arc --json
+tools/atrinik-content --root . diff arch/indoor/quill_pen.arc arch/indoor/rubbish.arc --semantic --json
+tools/atrinik-content --root . catalog search --kind archetype --text goblin --limit 20 --json
+tools/atrinik-content --root . apply --patch build/change.json --json
+tools/atrinik-content --root . apply --patch build/change.json --apply --json
+```
+
+`inspect` returns typed nodes, stable per-parse handles, node fingerprints,
+source spans, comments, and common diagnostics. `validate` uses the same result
+and exits nonzero for invalid content. `diff` compares ordered typed trees while
+ignoring only comments, line-ending style, and trailing whitespace. `catalog
+search` returns bounded stable catalog entries.
+
+Exit status `0` is success, `1` is a semantic difference, `2` is command-line
+usage, `3` is invalid syntax or encoding, `4` is a stale precondition or
+concurrent change, `5` is a safety refusal, `6` is an I/O publication failure,
+and `7` is an invalid JSON contract or other schema error.
+
+## Transaction boundary
+
+The schemas under `schemas/content-core-v1/` version inspection, catalog-search,
+transaction, and transaction-result JSON. A transaction lists 1 to 64 sorted,
+unique files and at most 10,000 primitive operations in total. Supported
+operations are `set-property`, `unset-property`, `add-object`, and
+`remove-object`.
+
+Every file supplies the SHA-256 of its exact starting bytes. Every existing
+node target also supplies the fingerprint returned by `inspect`. The core first
+checks every path, digest, fingerprint, operation, source document, and complete
+result document in memory. A stale or invalid member therefore prevents all
+writes. Unknown standard fields, reserved fields without a legacy spelling,
+wrong contexts, ambiguous duplicate properties, overlapping edits, and invalid
+typed values fail closed.
+
+`apply` is a dry run unless `--apply` is explicit. Publication creates a staged
+file and backup in each target's directory, flushes content, preserves file
+mode, rechecks every digest, and then uses same-directory atomic replacement.
+If publication fails, already replaced files are restored and all temporary
+files are removed. No portable filesystem primitive can make several files one
+crash-atomic unit; callers must still recover from machine loss between
+individual replacements, but handled process and I/O failures roll back the
+whole transaction.
+
+Writes require the Atrinik authored-source markers and are allowlisted to
+existing regular, non-symlink `arch/*.arc` and `maps/` files. Paths containing
+reserved build, generated, collected, packaged, runtime, distribution, or
+server-state components are refused. Collected runtime trees and mutable server
+state lack the source markers and are therefore not writable through the core.
+
+## Consumer migration
+
+`tools/world_content_audit.py` is the first migrated consumer. Its map and
+archetype traversal adapts the common `Document` and `Node` model back to the
+audit's established report shape. The dedicated regression test locks that
+shape while the full corpus parity test independently compares all 14 grammar
+fixtures with the legacy characterization expectations.

schemas/content-core-v1/catalog-search.schema.json

@@ -0,0 +1,52 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://atrinik.org/schema/content/v1/core-catalog-search.schema.json",
+  "title": "Atrinik content catalog search v1",
+  "type": "object",
+  "additionalProperties": false,
+  "required": ["schema_version", "kind", "query", "results", "truncated"],
+  "properties": {
+    "schema_version": {"const": 1},
+    "kind": {"const": "catalog-search"},
+    "query": {"$ref": "#/$defs/query"},
+    "results": {
+      "type": "array",
+      "maxItems": 100,
+      "items": {"$ref": "#/$defs/result"}
+    },
+    "truncated": {"type": "boolean"}
+  },
+  "$defs": {
+    "query": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["kind", "text", "limit"],
+      "properties": {
+        "kind": {"oneOf": [{"type": "string", "minLength": 1, "pattern": "^[a-z][a-z0-9-]*$"}, {"type": "null"}]},
+        "text": {"type": "string"},
+        "limit": {"type": "integer", "minimum": 1, "maximum": 100}
+      }
+    },
+    "location": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["path", "line", "column"],
+      "properties": {
+        "path": {"type": "string", "minLength": 1},
+        "line": {"type": "integer", "minimum": 1},
+        "column": {"type": "integer", "minimum": 1}
+      }
+    },
+    "result": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["domain", "key", "location", "metadata"],
+      "properties": {
+        "domain": {"type": "string", "minLength": 1},
+        "key": {"type": "string", "minLength": 1},
+        "location": {"$ref": "#/$defs/location"},
+        "metadata": {"type": "object", "additionalProperties": {}}
+      }
+    }
+  }
+}

schemas/content-core-v1/examples/catalog-search.json

@@ -0,0 +1,14 @@
+{
+  "schema_version": 1,
+  "kind": "catalog-search",
+  "query": {"kind": "archetype", "text": "oak", "limit": 10},
+  "results": [
+    {
+      "domain": "archetype",
+      "key": "oak_tree",
+      "location": {"path": "arch/example.arc", "line": 1, "column": 8},
+      "metadata": {"name": "oak tree"}
+    }
+  ],
+  "truncated": false
+}

schemas/content-core-v1/examples/inspection.json

@@ -0,0 +1,19 @@
+{
+  "schema_version": 1,
+  "kind": "document-inspection",
+  "document": {
+    "path": "maps/example",
+    "format": "map",
+    "logical_id": "/example",
+    "byte_sha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
+    "size": 13,
+    "line_endings": "lf",
+    "terminal_newline": true,
+    "valid": true
+  },
+  "nodes": [],
+  "top_level": [],
+  "multipart_continuations": 0,
+  "comments": [],
+  "diagnostics": []
+}

schemas/content-core-v1/examples/manifest.json

@@ -0,0 +1,9 @@
+{
+  "schema_version": 1,
+  "examples": [
+    {"kind": "catalog-search", "path": "examples/catalog-search.json"},
+    {"kind": "inspection", "path": "examples/inspection.json"},
+    {"kind": "transaction", "path": "examples/transaction.json"},
+    {"kind": "transaction-result", "path": "examples/transaction-result.json"}
+  ]
+}

schemas/content-core-v1/examples/transaction-result.json

@@ -0,0 +1,16 @@
+{
+  "schema_version": 1,
+  "kind": "transaction-result",
+  "dry_run": true,
+  "applied": false,
+  "files": [
+    {
+      "path": "maps/example",
+      "before_sha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
+      "after_sha256": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
+      "operation_count": 1,
+      "diff": "--- a/maps/example\n+++ b/maps/example\n"
+    }
+  ],
+  "diagnostics": []
+}

schemas/content-core-v1/examples/transaction.json

@@ -0,0 +1,20 @@
+{
+  "schema_version": 1,
+  "kind": "content-transaction",
+  "files": [
+    {
+      "path": "maps/example",
+      "format": "map",
+      "base_sha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
+      "operations": [
+        {
+          "kind": "set-property",
+          "node_handle": "node-000001",
+          "node_fingerprint": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
+          "field_id": "map-header.width",
+          "value": 24
+        }
+      ]
+    }
+  ]
+}