Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:
# README promises. This covers every package — @tessera/core,
# @tessera/koios, @tessera/verifier, and @tessera/backend carry the
# correctness-critical, hash-committed logic and previously never ran on
# a PR (only frontend/cip179 + frontend/app did).
# a PR (only cip-179 + frontend/app did).
- run: pnpm install --frozen-lockfile
- run: pnpm -r type-check
- run: pnpm -r test
Expand Down
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,15 @@ served document matches the on-chain hash.

## Repository layout

| Path | What it is |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `frontend/app` | The browser app — [SolidJS][solid] + [Vite][vite] + TypeScript. |
| `frontend/cip179` | A pure, dependency-free TypeScript library to encode / decode / validate the label-17 format. |
| `packages/core` | Shared pure domain (`@tessera/core`): types, wire codec, audit/tally rules, weighted tally, canonical artifact + hashes. |
| `packages/koios` | The Koios read path (`KoiosDataSource`, tally inputs, tx-proof decoding), shared by direct mode, backend, and verifier. |
| `packages/verifier` | Standalone CLI that re-derives a survey's result artifact from chain data and checks its content hash. |
| `backend/server` | Tier-1 serving backend: cached chain reads, response validation, weight snapshots, artifact finalization. Node or CF+D1. |
| `backend/deps` | Indexer submodules (Adder / Yaci Store / Oura) for a future Tier-2; design notes in `backend/*.md`. |
| Path | What it is |
| ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `frontend/app` | The browser app — [SolidJS][solid] + [Vite][vite] + TypeScript. |
| `packages/cip179` | The reusable `cip-179` package: label-17 codec plus the cross-implementation domain / tally / txproof / tlock surface. |
| `packages/core` | Tessera app core (`@tessera/core`): the `DataSource` seam, Explore list/health payloads, keyset paging, the survey-list aggregation adapter, and config. The reusable domain/tally surface lives in `cip-179`. |
| `packages/koios` | The Koios read path (`KoiosDataSource`, tally inputs), shared by direct mode, backend, and verifier. |
| `packages/verifier` | Standalone CLI that re-derives a survey's result artifact from chain data and checks its content hash. |
| `backend/server` | Tier-1 serving backend: cached chain reads, response validation, weight snapshots, artifact finalization. Node or CF+D1. |
| `backend/deps` | Indexer submodules (Adder / Yaci Store / Oura) for a future Tier-2; design notes in `backend/*.md`. |

## Quick start

Expand Down Expand Up @@ -85,7 +85,7 @@ entered in Settings, stored only in the browser.

## Development

The repo is a pnpm workspace (`frontend/app`, `frontend/cip179`,
The repo is a pnpm workspace (`frontend/app`, `packages/cip179`,
`packages/core`, `packages/koios`, `packages/verifier`, `backend/server`).
Packages are consumed from TypeScript source (Vite aliases / `exports` pointing
at `src`), so cross-package edits are live with no build step.
Expand Down
49 changes: 27 additions & 22 deletions backend/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ proofs, `/proposal_list` for governance links, `/tip`, and polled `/tx_status`.
shared cache — cost grows for every user as surveys accumulate, and the
`MAX_PAGES` cap (`incomplete` flag) is a real ceiling.

The `DataSource` seam (`src/data/source.ts`) was built for exactly this swap:
The `DataSource` seam (`@tessera/core`'s `source.ts`) was built for exactly this swap:
_"a future semantic indexer backend can implement the same interface and drop in
with no change to the domain or UI layers."_

Expand Down Expand Up @@ -106,12 +106,12 @@ The two constraints — "anyone can re-run on their own Cloudflare account" **an
"self-hostable without much effort" — are reconciled by **layering**: a portable
core, a thin swappable runtime/storage adapter, and a portable HTTP contract.

| Layer | Portable? | Notes |
| ------------------------------------------------------------------------------------------------ | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Core** (TS): chain-follow/decode + pure `@tessera/core` (audit, tally) + tally-input gathering | yes | No Cloudflare APIs. Runs in Worker, Node/Bun, or a CLI. |
| **Storage**: repository interface over **SQL (SQLite dialect)** | yes | D1 _is_ SQLite. Self-host → libsql/better-sqlite3 (or Postgres). KV/Cache used **only** as an optional edge cache, never as the source of truth. |
| **Runtime adapter** | thin | CF: `wrangler.toml` + fetch handler + `[triggers] crons` + D1 binding. Self-host: tiny HTTP server + `node-cron`/loop + SQLite file. Both call **Core**. |
| **HTTP `/api` contract** | yes | What `IndexerDataSource` speaks. Identical whether served by a Worker or a process. |
| Layer | Portable? | Notes |
| ------------------------------------------------------------------------------------------------------- | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Core** (TS): chain-follow/decode + pure `cip-179` domain/tally (audit, tally) + tally-input gathering | yes | No Cloudflare APIs. Runs in Worker, Node/Bun, or a CLI. |
| **Storage**: repository interface over **SQL (SQLite dialect)** | yes | D1 _is_ SQLite. Self-host → libsql/better-sqlite3 (or Postgres). KV/Cache used **only** as an optional edge cache, never as the source of truth. |
| **Runtime adapter** | thin | CF: `wrangler.toml` + fetch handler + `[triggers] crons` + D1 binding. Self-host: tiny HTTP server + `node-cron`/loop + SQLite file. Both call **Core**. |
| **HTTP `/api` contract** | yes | What `IndexerDataSource` speaks. Identical whether served by a Worker or a process. |

**Consequences**

Expand All @@ -137,10 +137,13 @@ Running the _same_ validation + tally code in the browser, the serving tier, and
a standalone verifier requires factoring the shared code out of the app. This is
load-bearing for the verifiability story, not just hygiene.

- **`cip-179`** — already a directory with its own `package.json`
(`frontend/cip179`), today consumed via a Vite/tsconfig path alias
(`../cip179/src/index.ts`). Promote it to a real pnpm-workspace package imported
by name. Low risk.
- **`cip-179`** — a pnpm-workspace package at `packages/cip179`, imported by
name. It has since grown beyond the codec into the reusable, cross-implementation
surface (subpath exports `cip-179/domain`, `cip-179/tally`, `cip-179/txproof`,
`cip-179/tlock`, and the `cip-179/evolution` serialization adapter). The
txproof/tlock stacks inject their Cardano-serialization primitives through
`TxProofCodec` / `MetadatumCodec` ports, so evolution-sdk is confined to the
adapter; see `packages/cip179/README.md`.
- **`@tessera/core`** — extract the **pure** domain from `frontend/app/src`:
- **Move:** the data-model **types** from `data/source.ts` (`ChainPos`,
`ChainTip`, `SurveyRecord`, `ResponseRecord`, `CancellationRecord`,
Expand Down Expand Up @@ -370,8 +373,9 @@ emits an artifact whose hashed body is a single **cancellation record** (cancell
- **Batch caps:** bulk POSTs have per-request element limits (cf. the existing
`TX_METADATA_BATCH = 50` in `koios.ts`); chunk accordingly. At PoC scale this
is a handful of batches in one invocation.
- **Encodings** — resolved: thin wrappers over evolution-sdk in
`packages/koios/src/bech32.ts` (no hand-rolled bech32): `stakeAddress`
- **Encodings** — resolved: thin wrappers over evolution-sdk in the
`cip-179/evolution` adapter (`packages/cip179/src/evolution/index.ts`, no
hand-rolled bech32), exposed through the `TxProofCodec` port: `stakeAddress`
(CIP-19 headers, key **and** script credentials, both networks), `drepId`
(CIP-129 `drep1…`), `govActionId` (CIP-129 `gov_action1…`) — each verified
against ids Koios itself emits.
Expand Down Expand Up @@ -465,7 +469,7 @@ and converges over successive crons. Snapshot membership still comes from each
run's fresh label-index scan, so rolled-back txs age out — their cache entries
just stop being requested.

### 6.6 Weighted tally computation (`@tessera/core`)
### 6.6 Weighted tally computation (`cip-179/tally`)

Weighting is the mechanical generalization of the existing tally: **replace
"count 1 per responder" with "add the responder's weight."**
Expand All @@ -491,7 +495,7 @@ Weighting is the mechanical generalization of the existing tally: **replace
The unit of result publication and the Koios→node seam.

- **Canonical JSON**, content-addressed by hash. Pinned (implemented in
`@tessera/core`'s `canonical.ts`, shared by emitter and verifier):
`cip-179/tally`'s `canonical.ts`, shared by emitter and verifier):
`canonicalJson()` is a strict JCS-lite — keys sorted by UTF-16 code units, no
whitespace, **safe integers only** (throws on floats/bigints/non-plain
objects) — and the hash is **blake2b-256** of its UTF-8 bytes (the hash
Expand Down Expand Up @@ -595,8 +599,8 @@ Contents (sketch):
- **Future:** the `tally` hash is the natural handle for an **on-chain anchor**,
closing the loop with CIP-179 itself.
- **Verifiability.** The `tally` embeds the counted responders, their answers (or
refs), weights, and totals, so any third party re-runs the pure `@tessera/core`
tally and reproduces both the results and the hash; every weight is re-fetchable
refs), weights, and totals, so any third party re-runs the pure `cip-179/tally`
computation and reproduces both the results and the hash; every weight is re-fetchable
from Koios at `end_epoch`. Trust reduces to Koios's stake numbers for epoch E,
which the node tier later removes — without changing this format.

Expand Down Expand Up @@ -664,7 +668,7 @@ Contents (sketch):
2. **Phase 2 — Koios tally inputs + artifacts.**
- ~~`TallyInputSource` (Koios impl): per-epoch shared snapshot (§6.5).~~
**Done** — `packages/koios/src/tallyInputs.ts` + `finalize.ts`.
- ~~Weighted per-role tally in `@tessera/core` (§6.6).~~ **Done** —
- ~~Weighted per-role tally in `cip-179/tally` (§6.6).~~ **Done** —
`weightedTally.ts` (+ §6.3 rules 1–3 in `proof.ts`/`audit.ts`/`dedupe.ts`,
persisted incrementally in `validated_response`).
- Content-addressed artifacts (§7 — in D1, not R2; **done**); optional IPFS
Expand All @@ -688,11 +692,12 @@ Contents (sketch):
- ~~**Exact Koios shapes**~~ — resolved empirically; see the §6.4 table (incl.
the deprecated-variant and `_epoch_no` pitfalls and `/epoch_info` flakiness).
- ~~**Credential-proof verification** (§6.3 rule 2)~~ — **done**: mechanism A/B
evaluated in `@tessera/core`'s `proof.ts` over `TxProof` evidence decoded by
`packages/koios/src/txProof.ts` (voting_procedures shape pinned by real
evaluated in `cip-179/domain`'s `proof.ts` over `TxProof` evidence decoded by
`packages/cip179/src/txproof/txProof.ts` (voting_procedures shape pinned by real
preview vote-tx fixtures); verdicts persisted per response (§6.5).
- ~~**Credential encodings**~~ — **done**: `packages/koios/src/bech32.ts` (§6.4).
- ~~**Canonicalization profile**~~ — **done**: `@tessera/core` `canonical.ts`
- ~~**Credential encodings**~~ — **done**: `packages/cip179/src/evolution/index.ts`
(`stakeAddress` / `drepId` / `govActionId`, behind the `TxProofCodec` port) (§6.4).
- ~~**Canonicalization profile**~~ — **done**: `cip-179/tally` `canonical.ts`
(JCS-lite + decimal-string bigints + blake2b-256), used by emitter and
verifier (§7).
- ~~**Finalization safety margin**~~ — chosen: 600 s past the `end_epoch`
Expand Down
3 changes: 2 additions & 1 deletion backend/server/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,9 @@
"dependencies": {
"@tessera/core": "workspace:*",
"@tessera/koios": "workspace:*",
"@tessera/tlock": "workspace:*",
"cip-179": "workspace:*",
"@evolution-sdk/evolution": "^0.5.9",
"@mattpiz/tlock-js": "0.10.0",
"@hono/node-server": "^1.13.0",
"hono": "^4.6.0"
},
Expand Down
10 changes: 6 additions & 4 deletions backend/server/src/finalize.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,23 @@ import {
type SurveyDefinition,
} from "cip-179";
import {
artifactHash,
credentialKey,
hexToBytes,
responderAnswers,
type CancellationRecord,
type ChainTip,
type Cip179Records,
type ResponseRecord,
type SurveyRecord,
type TxProof,
} from "cip-179/domain";
import {
artifactHash,
responderAnswers,
type TallyArtifact,
type TallyBody,
type TallyInputSource,
type TxProof,
type WeightInfo,
} from "@tessera/core";
} from "cip-179/tally";

import { loadConfig } from "./config";
import { finalizeClosedSurveys } from "./finalize";
Expand Down
20 changes: 11 additions & 9 deletions backend/server/src/finalize.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,30 +25,32 @@
import { Role, type SurveyResponse } from "cip-179";

import {
artifactHash,
auditRevealedResponses,
bytesToHex,
cancellationVerified,
laterInChain,
parseCredentialKey,
refKey,
rulesetHash,
toArtifactQuestions,
toArtifactResponders,
voteDeadlineUnix,
weightedTallySurvey,
type ArtifactRoleTally,
type ChainTip,
type Cip179Records,
type ResponseRecord,
type SurveyRecord,
type TxProof,
} from "cip-179/domain";
import {
artifactHash,
rulesetHash,
toArtifactQuestions,
toArtifactResponders,
weightedTallySurvey,
type ArtifactRoleTally,
type TallyArtifact,
type TallyBody,
type TallyInputSource,
type TxProof,
type WeightedResponder,
} from "@tessera/core";
import { isQuicknet, roundIsAvailable } from "@tessera/tlock";
} from "cip-179/tally";
import { isQuicknet, roundIsAvailable } from "cip-179/tlock";

import type { ServerConfig } from "./config";
import { tlockSealedReveal, type SealedRevealFn } from "./sealedReveal";
Expand Down
5 changes: 3 additions & 2 deletions backend/server/src/http.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,15 @@
import { describe, expect, it } from "vitest";

import { Role, type Credential, type SurveyDefinition } from "cip-179";
import { fromJsonSafe, hexToBytes, toJsonSafe } from "@tessera/core";
import { hexToBytes } from "cip-179/domain";
import { fromJsonSafe, toJsonSafe } from "cip-179/tally";
import type {
CancellationRecord,
ChainTip,
ResponseRecord,
SurveyBundle,
SurveyRecord,
} from "@tessera/core";
} from "cip-179/domain";

import { loadConfig } from "./config";
import { createApp } from "./http";
Expand Down
17 changes: 8 additions & 9 deletions backend/server/src/http.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,19 +44,18 @@ import { cors } from "hono/cors";

import {
credentialKey,
refKey,
type ChainTip,
type Cip179Records,
type GovLink,
} from "cip-179/domain";
import { fromJsonSafe, toJsonSafe } from "cip-179/tally";
import {
encodeSurveyCursor,
fromJsonSafe,
isSurveyListFilter,
parseSurveyCursor,
refKey,
searchTermsOf,
toJsonSafe,
} from "@tessera/core";
import type {
BackendHealth,
ChainTip,
Cip179Records,
GovLink,
type BackendHealth,
} from "@tessera/core";
import { KoiosDataSource } from "@tessera/koios";

Expand Down
17 changes: 7 additions & 10 deletions backend/server/src/listIndex.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,16 @@
*/

import {
aggregateSurveyList,
credentialKey,
refKey,
responseCounts,
surveyHaystack,
toJsonSafe,
} from "@tessera/core";
import type {
CancellationRecord,
ChainTip,
Cip179Records,
GovLink,
} from "@tessera/core";
type CancellationRecord,
type ChainTip,
type Cip179Records,
type GovLink,
} from "cip-179/domain";
import { toJsonSafe } from "cip-179/tally";
import { aggregateSurveyList, surveyHaystack } from "@tessera/core";

import type { SurveyIndexRow } from "./store";

Expand Down
2 changes: 1 addition & 1 deletion backend/server/src/refresh.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
* serves a half-built or blank snapshot because one fetch hiccuped.
*/

import { toJsonSafe } from "@tessera/core";
import { toJsonSafe } from "cip-179/tally";
import { KoiosDataSource, KoiosTallyInputs } from "@tessera/koios";

import type { ServerConfig } from "./config";
Expand Down
10 changes: 6 additions & 4 deletions backend/server/src/sealedReveal.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@
* offline `revealWithBeacon` loop — while tests inject a stub that returns
* decrypted answers with no network or crypto.
*
* `@tessera/tlock` is **lazy-imported** inside the default so a finalize pass with
* `cip-179/tlock` is **lazy-imported** inside the default so a finalize pass with
* no revealable sealed survey never pulls in the tlock/evolution bundle, and the
* tlock-js crypto chunk loads only when a reveal actually runs.
*/

import type { ResponseRecord } from "@tessera/core";
import type { ResponseRecord } from "cip-179/domain";
import type { SurveyResponse } from "cip-179";

/** The drand beacon a reveal used — committed to the artifact's provenance. */
Expand Down Expand Up @@ -45,13 +45,15 @@ export type SealedRevealFn = (

/**
* Default reveal: fetch (and BLS-verify) the round's beacon once, then decrypt
* every record offline. Lazy-imports `@tessera/tlock` so the crypto stack is
* every record offline. Lazy-imports `cip-179/tlock` so the crypto stack is
* pulled only when a sealed survey is actually revealed.
*/
export const tlockSealedReveal: SealedRevealFn = async (records, { round }) => {
const { fetchBeacon, revealWithBeacon } = await import("@tessera/tlock");
const [{ fetchBeacon, revealWithBeacon }, { evolutionCodec }] =
await Promise.all([import("cip-179/tlock"), import("cip-179/evolution")]);
const beacon = await fetchBeacon(round);
const revealed = await revealWithBeacon(
evolutionCodec,
records.map((r) => r.response),
beacon,
);
Expand Down
Loading