llm-digest

A reference implementation showing how @tigrisdata/storage, @tigrisdata/agent-shell, and Karpathy's LLM Wiki pattern compose into a working overnight researcher.

A scheduled GitHub Action ingests new sources every night, updates a markdown wiki stored in Tigris, and produces a daily digest — a single shareable page summarizing what changed and what to read first. The digest gets a presigned URL you can read in your browser, share with a colleague, or email yourself.

What this is, honestly

This project is aimed at developers evaluating the Tigris stack. If you're reading the code to see what @tigrisdata/storage and @tigrisdata/agent-shell look like in production, that's the audience.

The minimum-viable version of an overnight researcher is much simpler than this. It's a cron, an LLM call, and a Slack webhook — about thirty lines, no Tigris, no agent-shell, no wiki schema. That version is useful and you can build it in an afternoon. llm-digest deliberately isn't that version.

What llm-digest does that the simple version can't:

• Maintains a denser-over-time wiki (Karpathy's pattern) rather than a flat list of summaries. By month three the page on a topic answers "what's the current state" in three seconds, with provenance back to every source that contributed to it.
• Runs safely unattended thanks to agent-shell: every overnight run either lands a complete, schema-clean batch of writes or leaves the bucket byte-for-byte unchanged. No partially-edited wiki, ever.
• Delivers the digest as a presigned URL that anyone can click without authentication, valid for 30 days.
• Supports multi-day rollback via Tigris snapshots if the agent quietly drifted from the schema days ago and you only just noticed.

If you don't need those properties, you don't need Tigris or agent-shell, and the simple version is the right answer. If you do, this is what they look like composed.

Project rule

Don't pitch tools — including Tigris or anything else — unless they solve a real problem. If a feature is just convenient, document it as convenient. If it's load-bearing, document it as load-bearing.

TIGRIS_FEATURES.md is the audit. Every primitive used — Tigris bucket, snapshots, fork, presigned URLs, agent-shell mount, the CLAUDE.md schema — gets a section explaining what it is, what fails without it, and a verdict.

Stack

• Tigris — object storage for the wiki, sources, and digests. We use Tigris's S3-compatible API for plain put/get/list, plus three Tigris-specific features: snapshots (daily history), copy-on-write fork (manual recovery + future schema experiments), and presigned URLs (digest delivery).
• @tigrisdata/storage — the Tigris S3 SDK. Module-level functions configured via env or per-call.
• @tigrisdata/agent-shell — mounts the Tigris bucket as a JS-virtual filesystem. Tool implementations write through the shell handle; flush() atomically promotes the buffered writes to the bucket on success; throwing skips the flush. This is the rollback mechanism, written down in one place instead of in user-code staging-and-promote logic.
• @anthropic-ai/sdk — the agent runner. We use the SDK directly with a custom agent loop because the loop has to run in-process for tool implementations to route through agent-shell. (Headless @anthropic-ai/claude-code would run as a child process whose OS-filesystem writes the JS-virtual shell can't see.)
• GitHub Actions — scheduler. Free for public repos, low-cost for private at this volume.
• Obsidian (optional, on the read side) — points at a folder synced from the bucket via rclone or aws s3 sync. You read; the agent writes.

The shape

Every night at 07:00 UTC, GitHub Actions runs scripts/ingest.ts. The script:

1. Reads state.json and the RSS feeds, dedupes, builds a list of URLs new since yesterday.
2. Takes a Tigris snapshot (pre-ingest-<runId>) for daily history.
3. Mounts the Tigris bucket via agent-shell.
4. Runs the SDK agent loop. The model calls tools (fetch_url, write_source, write_wiki_page, save_digest) whose implementations write through the mounted shell. The shell buffers everything in process memory.
5. On clean return, flush() promotes the buffered writes to the bucket in one atomic operation. On any throw, discard() drops the buffer and the bucket is unchanged.
6. After flush, updates state.json and posts a presigned digest URL to Slack.

Total moving parts: GitHub Actions, the Anthropic SDK, @tigrisdata/storage, @tigrisdata/agent-shell, a Slack webhook, and the CLAUDE.md schema. Roughly 1700 lines of TypeScript and 80 lines of YAML.

Where to start

If you're Claude Code, building this for the first time, read PLAN.md end to end. It's a sequenced 12-step implementation plan with acceptance criteria.

If you're a human evaluating the project, the reading order is:

1. ARCHITECTURE.md — the picture, the runtime sandwich, the data flow.
2. TIGRIS_FEATURES.md — what each piece is doing and why it earns its place.
3. docs/08-anatomy-of-the-loop.md (after Claude Code finishes Step 11) — a code walkthrough of the agent loop and the shell flush.

If you're a human about to deploy this, docs/01-quickstart.md after Claude Code finishes building.

Status

Planning skeleton. The implementation files (scripts, tool definitions, workflows) are stubbed and should be filled in by Claude Code following PLAN.md. The schema (CLAUDE.md) and the architectural intent (ARCHITECTURE.md, TIGRIS_FEATURES.md) are complete.