refactor: unify data access — make data.latest() the canonical accessor

.claude/skills/swamp-model/references/data-chaining.md

@@ -3,7 +3,11 @@
 The `command/shell` model enables data chaining by running shell commands and
 capturing output for use in other models. For JSON output, use `jq` in the
 command to extract specific fields, then access the result via
-`resource.result.result.attributes.stdout`.
+`data.latest("model-name", "result").attributes.stdout`.
+
+> **Note:** `data.latest()` is the preferred accessor for all cross-model data.
+> The `model.*.resource` pattern is deprecated and will be removed in a future
+> release. Existing examples below show both patterns for reference.
 
 ## command/shell Data Attributes
 

.claude/skills/swamp-model/references/examples.md

@@ -11,21 +11,21 @@
 
 ## CEL Expression Quick Reference
 
-| Expression Pattern                                           | Description                                | Example Value                 |
-| ------------------------------------------------------------ | ------------------------------------------ | ----------------------------- |
-| `model.<name>.resource.<spec>.<instance>.attributes.<field>` | Cross-model resource reference (PREFERRED) | VPC ID, subnet CIDR, etc.     |
-| `model.<name>.resource.result.result.attributes.stdout`      | command/shell stdout                       | AMI ID from aws cli command   |
-| `model.<name>.file.<spec>.<instance>.path`                   | File path from another model               | `/path/to/file.txt`           |
-| `self.name`                                                  | Current model's name                       | `my-vpc`                      |
-| `self.version`                                               | Current model's version                    | `1`                           |
-| `self.globalArguments.<field>`                               | This model's own global argument           | CIDR block, region, etc.      |
-| `inputs.<name>`                                              | Runtime input value                        | `production`, `true`, etc.    |
-| `env.<VAR_NAME>`                                             | Environment variable                       | AWS region, credentials       |
-| `vault.get("<vault>", "<key>")`                              | Vault secret                               | API key, password             |
-| `data.version("<model>", "<name>", <version>)`               | Specific version of data                   | Rollback to version 1         |
-| `data.latest("<model>", "<name>")`                           | Latest version snapshot                    | Workflow-start snapshot       |
-| `data.findBySpec("<model>", "<spec>")`                       | Find all instances from a spec             | All subnets from scanner      |
-| `data.findByTag("<key>", "<value>")`                         | Find data by tag                           | All resources tagged env=prod |
+| Expression Pattern                                           | Description                               | Example Value                 |
+| ------------------------------------------------------------ | ----------------------------------------- | ----------------------------- |
+| `data.latest("<model>", "<name>").attributes.<field>`        | Latest data (PREFERRED, sync disk read)   | VPC ID, subnet CIDR, etc.     |
+| `data.version("<model>", "<name>", N).attributes.<field>`    | Specific version of data                  | Rollback to version 1         |
+| `data.findBySpec("<model>", "<spec>")`                       | Find all instances from a spec            | All subnets from scanner      |
+| `data.findByTag("<key>", "<value>")`                         | Find data by tag                          | All resources tagged env=prod |
+| `model.<name>.resource.<spec>.<instance>.attributes.<field>` | Cross-model resource (DEPRECATED)         | VPC ID, subnet CIDR, etc.     |
+| `model.<name>.resource.result.result.attributes.stdout`      | command/shell stdout (DEPRECATED)         | AMI ID from aws cli command   |
+| `model.<name>.file.<spec>.<instance>.path`                   | File path from another model (DEPRECATED) | `/path/to/file.txt`           |
+| `self.name`                                                  | Current model's name                      | `my-vpc`                      |
+| `self.version`                                               | Current model's version                   | `1`                           |
+| `self.globalArguments.<field>`                               | This model's own global argument          | CIDR block, region, etc.      |
+| `inputs.<name>`                                              | Runtime input value                       | `production`, `true`, etc.    |
+| `env.<VAR_NAME>`                                             | Environment variable                      | AWS region, credentials       |
+| `vault.get("<vault>", "<key>")`                              | Vault secret                              | API key, password             |
 
 ### CEL Path Patterns by Model Type
 
@@ -205,42 +205,44 @@ dryRun: true
 
 ## Cross-Model Data References
 
-### Preferred: model.* Expressions
+### Preferred: data.latest() Expressions
 
-Always use `model.*` expressions for referencing other models' data:
+Always use `data.latest()` for referencing other models' data. It reads directly
+from disk on every call, so it always reflects the latest state:
 
 ```yaml
-# CORRECT: model.* expression
+# CORRECT: data.latest() — always reads fresh data from disk
 globalArguments:
-  vpcId: ${{ model.my-vpc.resource.vpc.main.attributes.VpcId }}
+  vpcId: ${{ data.latest("my-vpc", "main").attributes.VpcId }}
 
-# AVOID: data.latest() for cross-model references
+# DEPRECATED: model.*.resource — will be removed in a future release
 globalArguments:
-  vpcId: ${{ data.latest("my-vpc", "main").attributes.VpcId }}
+  vpcId: ${{ model.my-vpc.resource.vpc.main.attributes.VpcId }}
 ```
 
-### Why model.* is Preferred
+### Why data.latest() is Preferred
 
-| Feature                   | `model.*`        | `data.latest()` |
-| ------------------------- | ---------------- | --------------- |
-| In-workflow updates       | Yes (live)       | No (snapshot)   |
-| Clear dependency tracking | Yes              | Yes             |
-| Type validation           | Yes (via schema) | No              |
-| Expression readability    | More explicit    | Less explicit   |
+| Feature                   | `data.latest()` | `model.*.resource` |
+| ------------------------- | --------------- | ------------------ |
+| Always fresh (no cache)   | Yes (sync disk) | Yes (eager load)   |
+| Supports vary dimensions  | Yes             | No                 |
+| Clear dependency tracking | Yes             | Yes                |
+| Future-proof              | Yes (canonical) | No (deprecated)    |
 
-### When to Use data.latest()
-
-Only use `data.latest()` when you specifically need:
-
-1. **Snapshot semantics** — value frozen at workflow start
-2. **Dynamic model names** — building model name from variables
+### Other data.* Functions
 
 ```yaml
-# Rollback scenario: get the previous version
+# Specific version (rollback scenario)
 previousConfig: ${{ data.version("app-config", "config", 1).attributes.setting }}
 
-# Dynamic model name (rare)
+# Dynamic model name
 dynamicValue: ${{ data.latest(inputs.modelName, "state").attributes.value }}
+
+# Find all instances of a spec
+allSubnets: ${{ data.findBySpec("scanner", "subnet") }}
+
+# Find by tag
+prodResources: ${{ data.findByTag("env", "prod") }}
 ```
 
 ### Self-References

.claude/skills/swamp-workflow/references/data-chaining.md

@@ -68,31 +68,33 @@ globalArguments:
 
 Use `dependsOn` to ensure step B runs after step A when B references A's output.
 
-## Choosing `model.*` vs `data.latest()` Expressions
+## Choosing `data.latest()` vs `model.*` Expressions
 
 Model instance definitions use CEL expressions to reference other models' data.
-Both expression forms work for cross-workflow data access, but they have
-different characteristics:
+**`data.latest()` is the preferred (canonical) accessor.** The
+`model.*.resource` and `model.*.file` patterns are deprecated and will be
+removed in a future release.
 
-| Expression                        | Sees current-run data?                              | Sees prior-run data?                              |
-| --------------------------------- | --------------------------------------------------- | ------------------------------------------------- |
-| `model.<name>.resource.<spec>`    | **Yes** — in-memory context updated after each step | **Yes** — reads persisted data with `type` intact |
-| `data.latest("<name>", "<spec>")` | **No** — snapshot taken at workflow start           | **Yes** — reads persisted data regardless of tags |
+| Expression                            | Sees current-run data?                 | Sees prior-run data? | Status         |
+| ------------------------------------- | -------------------------------------- | -------------------- | -------------- |
+| `data.latest("<name>", "<spec>")`     | **Yes** — sync disk read on every call | **Yes**              | **Preferred**  |
+| `data.version("<name>", "<spec>", N)` | **Yes** — sync disk read               | **Yes**              | **Preferred**  |
+| `model.<name>.resource.<spec>`        | **Yes** — eagerly populated            | **Yes**              | **Deprecated** |
 
 ### When to use each
 
-**Use `model.*`** for most cases:
+**Use `data.latest()` / `data.version()`** for all cases:
 
 - Intra-workflow chaining (step B reads step A's output in the same run)
 - Cross-workflow chaining (workflow B reads data from prior workflow A run)
+- Data always reflects the latest on-disk state (no stale cache)
+- Supports vary dimensions for environment isolation
 
-**Use `data.latest()`** when:
+**Avoid `model.*.resource` / `model.*.file`** — these patterns are deprecated
+and will emit a warning. They still work for backward compatibility but should
+be migrated to `data.latest()`.
 
-- You need to query data by model name dynamically
-- You want a snapshot from workflow start rather than live in-memory context
-
-Both forms are equivalent for dependency purposes — use explicit `dependsOn` to
-control step ordering.
+Use explicit `dependsOn` to control step ordering.
 
 ## Example: Multi-Step Infrastructure Workflow
 

design/expressions.md

@@ -37,15 +37,16 @@ attributes:
   message: ${{ model.foo.definition.attributes.message }}
 ```
 
-Or the data output of the same model:
+Or the data output of the same model (using the preferred `data.latest()`
+accessor):
 
 ```yaml
 id: 0bc79a8f-d9d2-4ec5-a37f-8d88bbb3ee27
 name: baz
 version: 1
 tags: {}
 attributes:
-  message: ${{ model.foo.data.attributes.message }}
+  message: ${{ data.latest('foo', 'result').attributes.message }}
 ```
 
 You can refer to your own model with `self`, for things like name, version,
@@ -90,14 +91,12 @@ dynamic configuration without modifying definition files.
 
 ## Data Versioning
 
-When accessing model data, the "latest" version is implied by default:
+`data.latest()` is the **canonical accessor** for model data. It reads directly
+from disk on every call, so it always reflects the latest on-disk state with no
+cache staleness. The `model.*.resource` and `model.*.file` patterns are
+**deprecated** and will be removed in a future release.
 
-```yaml
-message: ${{ model.foo.data.attributes.message }} # accesses latest version
-```
-
-Data is immutable and versioned. To access specific versions or list available
-versions, use the following CEL functions:
+Data is immutable and versioned. Use the following CEL functions to access data:
 
 ### data.latest(modelName, dataName)
 

design/vaults.md

@@ -293,7 +293,7 @@ version: 1
 attributes:
   vaultName: aws
   secretKey: new-auth-token
-  secretValue: ${{ model.api-generator.data.attributes.token }}
+  secretValue: ${{ data.latest('api-generator', 'result').attributes.token }}
   operation: put
 ```
 

integration/cel_data_access_test.ts

@@ -1078,3 +1078,62 @@ Deno.test("CEL Data Access: resource resolves after model delete and recreate wi
     assertEquals(result.definition.globalArguments.bucket_count, 42);
   });
 });
+
+// ============================================================================
+// Sync Disk Read: data.latest() sees fresh data written after buildContext()
+// ============================================================================
+
+Deno.test("CEL Data Access: data.latest() sees data written after buildContext()", async () => {
+  await withTempDir(async (repoDir) => {
+    await setupRepoDir(repoDir);
+    const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
+    const definitionRepo = new YamlDefinitionRepository(repoDir);
+    const type = ModelType.create("test/model");
+    const owner = createOwner("test/model:fresh-data");
+
+    const model = Definition.create({
+      name: "fresh_data_model",
+      globalArguments: {},
+    });
+    await definitionRepo.save(type, model);
+
+    // Write initial data
+    const data = Data.create({
+      name: "live_state",
+      contentType: "application/json",
+      lifetime: "infinite",
+      garbageCollection: 10,
+      tags: { type: "resource" },
+      ownerDefinition: owner,
+    });
+    await dataRepo.save(
+      type,
+      model.id,
+      data,
+      new TextEncoder().encode(JSON.stringify({ step: 1 })),
+    );
+
+    // Build context — captures a snapshot of coordinates
+    const modelResolver = new ModelResolver(definitionRepo, {
+      repoDir,
+      dataRepo,
+    });
+    const context = await modelResolver.buildContext();
+
+    // Write NEW data AFTER context was built
+    await dataRepo.save(
+      type,
+      model.id,
+      data,
+      new TextEncoder().encode(JSON.stringify({ step: 2, fresh: true })),
+    );
+
+    // data.latest() should see the fresh version (sync disk read, no cache)
+    assertExists(context.data);
+    const latest = context.data.latest("fresh_data_model", "live_state");
+    assertExists(latest);
+    assertEquals(latest.version, 2);
+    assertEquals(latest.attributes.step, 2);
+    assertEquals(latest.attributes.fresh, true);
+  });
+});