feat: Create Notes DB schema, indexes, and RLS migration - #169
Merged
Conversation
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
FrkAk
force-pushed
the
feat/pyz-248-create-notes-db-schema-indexes-and-rls
branch
from
June 30, 2026 20:03
d586230 to
1ee018d
Compare
ulascanzorer
approved these changes
Jul 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Task Reference: [PYZ-248]
Adds the Notes relational foundation: four tables in
lib/db/schema.ts(notes,note_task_links,note_links,note_revisions), their TypeScript enums inlib/types.ts, and the RLS surface indocker/rls-functions.sql,docker/rls-policies.sql, anddocker/grants.sql.notescarries the §4.1 column set (visibility, agent_writable/locked, feed_mode/feed_categories/feed_tags/feed_task_ids, version, embedding_status, share_requested_by, soft-deletedeleted_at) plus a weighted FTS generated columnsearch_tsv(setweight(left(title, 2000), 'A') || setweight(left(body, 131072), 'B'), coalesced, STORED). The length caps are exported schema constants inlined viasql.raw(NOTE_TITLE_MAX_BYTES=2000,NOTE_BODY_MAX_CHARS=200000,NOTE_SEARCH_INDEXED_CHARS=131072) so the notes CHECKs, the tsvector bounds, and thenote_revisionsCHECKs cannot desync. The search bound is 131072, not 200000, because a char-count CHECK cannot bound tsvector bytes: worst-case content (space-separated distinct hyphenated multibyte compounds, three lexemes per token) costs ~7.5 tsvector bytes per char and crosses Postgres's 1,048,575-byte tsvector ceiling at ~163k chars (measured on postgres:18); 131072 body chars + a max title lands at ~998 KB worst case. Bodies keep the full 200k contract — search ignores chars past the bound. A pending share request isshare_requested_by IS NOT NULL; there is no separate boolean to desync, and the FK'sON DELETE SET NULLauto-cancels a request whose requester was deleted. CHECK constraints pinvisibility,type,feed_mode, andembedding_statusto their value sets and boundtitle/slugto ≤ 2000 bytes (btree-safe) andbodyto ≤ 200000 chars;note_task_links.kindandnote_links(no self-link) are likewise constrained.notes_search_idx/notes_tags_idx; partialnotes_feed_idx(feed_mode <> 'none'); partial slug-unique and title indexes (deleted_at IS NULL); embedding-queue partial index; ETag index (project_id, updated_at). Link and revision tables index the FK directions a cascade or backlink query needs, without indexes a unique constraint already covers.notes_member_access(2-hop viaprojects, plus per-notevisibility='team' OR created_by = (SELECT current_app_user_id())— the caller lookup is(SELECT ...)-wrapped so it evaluates once per statement as an InitPlan, matching the file'scurrent_user_org_ids()discipline). The 3-hop link/revision policies check their endpoints with correlatedEXISTSpkey probes instead ofIN (SELECT id FROM public.notes): every policy clause carrying an IN-sublist plans its own hashed SubPlan over the caller's entire visible-notes set, so a single-row link write evaluated that set up to six times (EXPLAIN-measured 327 vs 33 buffer hits at 5k notes); EXISTS keeps identical RLS-filtered semantics at one index probe per check. Both endpoints checked fornote_task_linksandnote_links; RESTRICTIVE write floors on both link tables; anotes_insert_author_onlyfloor pinningcreated_by(strict) andupdated_by/share_requested_by(self-or-NULL) on INSERT; and anote_revisions_insert_author_onlyfloor re-pinning snapshot authorship. Newcurrent_app_user_id()helper (plpgsql STABLE SECURITY INVOKER). Defense-in-depth triggers mirroring the tasks/task_edges suite:project_idandcreated_byimmutability onnotes(thecreated_bytrigger rejects reassignment and non-author null-erasure while allowing theON DELETE SET NULLcascade), anupdated_by/share_requested_byattribution pin, and SECURITY INVOKER cross-project rejection on both link tables (an invisible note reads back like a nonexistent one, so no SQLSTATE existence oracle). The threenotestriggers are AFTER-row, not BEFORE: any BEFORE UPDATE row trigger onnotesforces the STOREDsearch_tsvrecompute on every metadata-only UPDATE, while AFTER keeps the executor's skip (verified on postgres:18: 316ms vs 7ms).note_revisionsis append-only forapp_user(UPDATE revoked ingrants.sql, table-existence-guarded so container-init stays safe).rls-functions.sqlis guarded onto_regclass, sodb:rls:owner(owner-managed, applied out-of-band — the CI migrator role has no piyaz_auth access) can run before migration 0003 and installscurrent_app_user_id()ahead of the deploy'sdb:rls:ci.apply-public-rls.tspreflights every functionrls-policies.sqlreferences and fails with the owner-apply runbook instead of a bare 42883 mid-transaction rollback.verify-rls.tsnow also asserts every trigger declared inrls-functions.sqlexists on the deploy DB and thatnote_revisionsUPDATE stays revoked, so a guarded trigger skip or grant drift blocks the deploy with an actionable message.The Drizzle migration
drizzle/0003_amusing_micromacro.sqlis committed and applies cleanly across the 0000 to 0003 chain, verified on a throwaway DB in the real deploy order (0000–0002 → owner apply → 0003 → owner apply → grants/policies/storage), with all five notes triggers installed and the UPDATE revoke in place. The reviewable source islib/db/schema.tsanddocker/*.sql.Type of change
Testing
bun run devbun run lint)bun run typecheck)tests/data/notes-rls.test.tscovers private/team visibility, cross-org isolation, GIN and partial index presence, thesearch_tsvSTORED generated column, FK cascade on note and task delete (both link directions), the cross-project rejection triggers (asserted by SQLSTATE 23514, not RLS),created_byimmutability against privatize-and-steal and against null-erasure by a non-author (42501) with the author-delete FK cascade still nulling it, theupdated_by/share_requested_byattribution pin (forge, erase, and forged INSERT rejected as 42501; self-claims and share-request clears allowed), thenote_revisionsUPDATE revoke (42501),project_idimmutability, a same-project link insert under FORCE RLS, trashed-slug reuse, thetitle/slug/bodylength CHECKs (oversize writes fail as 23514, not opaque btree/tsvector errors), and a tsvector overflow regression test: a full 200k-char body of distinct hyphenated CJK compounds — content that overflows an unbounded tsvector at ~163k chars — saves under theleft(body, 131072)bound.tests/db/rls-coverage.test.tsauto-covers the four tables.bun run test:db: 501 pass.Notes for reviewer
db:rls:owneris safe to run before this PR's migration (the notes trigger DDL self-guards on table existence) and should be run once before or right after merge; the deploy'sdb:rls:cipreflight anddb:rls:verifytrigger/grant assertions turn any missed step into an actionable failure instead of a raw SQLSTATE.bodyandsearch_tsvlives indocker/storage.sql(Drizzle has no compression API), with STORAGE EXTENDED; it is applied post-migrate byapply-public-rls.tsand mirrored into the push-based test container by the test harness, and a deploy-timepg_attribute.attcompressioncheck inscripts/verify-rls.tsguards it.agent_writable/locked) and visibility transitions are enforced at the call surface (PYZ-249, PYZ-252), not RLS. The append-only DELETE retention cap fornote_revisionslands with PYZ-249.search_tsvis server-side only (GIN-matched, up to hundreds of KB per row): the PYZ-249 data-access layer must read notes through explicit column projections that exclude it — a bareselect()ships it over the wire on every row (flagged in a comment on the column).request_sharesets onlyshare_requested_by(thepending_share_requestboolean was dropped as derivable, desync-prone state).task_edgeskeeps the inheritedIN (SELECT ...)policy shape; converting it to the EXISTS form is a follow-up outside this PR's scope.Docs impact
none