Skip to content

feat: support S3-backed context stores #14

Description

@beinan

Background

Context URIs are currently assumed to be local filesystem paths. Cloud-hosted agents need to persist shared context into remote object storage.

Proposal

  • Teach to detect URIs and initialize a Lance dataset backed by S3 via object store configuration.
  • Surface optional AWS credentials/configuration through environment variables and/or explicit parameters.
  • Update the Python path so S3 URIs are passed straight through and validated early.

Design Notes

  • Leverage Lance's existing integration (, ) rather than reimplementing S3 clients.
  • Allow per-run temporary credentials by honoring standard AWS env vars and the Lance object store configuration file.
  • Keep the URI format aligned with Lance's convention () so downstream tooling interoperates cleanly.

Acceptance Criteria

  • A context created with an S3 URI can append, checkout, and search records without regressions.
  • Python integration tests cover smoke usage against moto or a stubbed S3 endpoint.
  • Documentation explains required AWS permissions and credential discovery order.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions