-
Notifications
You must be signed in to change notification settings - Fork 13
Description
Problem
When the same model runs across multiple environments, all data versions are stored under a single path with one latest symlink:
.swamp/data/command/shell/{model-id}/result/
├── 1/ (dev run)
├── 2/ (prod run)
├── 3/ (dev run)
└── latest -> 3 # always points to most recent, regardless of environment
The latest symlink points to whichever version was written most recently — in this case, version 3 from a dev run. There is no concept of "latest for prod" or "latest for staging" at the filesystem level.
Why This Matters
The latest symlink serves two purposes:
- Quick access — users browsing the data directory can find the most recent output
- CEL resolution — expressions like
model.deploy-app.resource.result.latest.attributes.*resolve through this symlink
In a single-environment setup, latest always points to the most recent run, which is correct. But in a multi-environment setup, latest is misleading:
- A user looking at
latestafter a dev run might think it's the current prod state - A CEL expression referencing
latestin a prod workflow might resolve to dev data if dev ran more recently - There is no way to browse to "latest prod result" without searching through version metadata
This creates a correctness hazard: the most convenient and obvious access path (latest) returns environment-incorrect data in multi-environment scenarios.
Current Workaround
Users can use swamp data search --tag environment=prod to find the right version, but this requires knowing to avoid the latest symlink and using tag-based queries instead. CEL expressions have no equivalent workaround — data.latest() and the implicit latest path always resolve to the globally most recent version.
Use Case
A deployment workflow runs multiple times per day across dev, staging, and prod. A downstream monitoring model references the deployment model's output via CEL to check the last deployed version. Without environment-aware latest resolution, the monitoring model reports whichever environment deployed most recently, not the specific environment being monitored.
This issue is related to the need for tag-filtered data access in CEL (see related issue), which would solve the CEL side. This issue specifically tracks the filesystem-level latest symlink behavior for users browsing data directly.