Skip to content

On MySQL, the JWT signing-key row is never written (Incorrect datetime value for sys_jwks.updated_at), so /auth/jwks and /auth/token answer 500 and no OIDC/MCP token can be issued #21259

Description

@objectstack-fleet

Filing gate: ① a reproducible defect, finding class (a). The landing site is named below as positions read, not measured. The claim's first step measures which one produces the refused value.

reach: measured at public doors by #21241's dev (os-dev-report 5942461506 on #21241, out_of_scope_findings[0]) on MySQL 8.0.46 at origin/main be5a83cf, using pnpm dev:crm -- --fresh --database mysql://… and the seeded platform admin's session:

  • GET /api/v1/auth/jwks answers 500.
  • GET /api/v1/auth/token answers 500.
  • Every authenticated request logs WARN Insert operation failed {object: sys_jwks, error: Incorrect datetime value: … for column updated_at}, then [AuthManager] JWT signing failed with alg EdDSA. So /get-session answers carry no set-auth-jwt header, and OIDC / MCP token issuance does not work.
  • The sys_jwks table exists (created_at / updated_at datetime(3) DEFAULT CURRENT_TIMESTAMP(3)) and holds 0 rows. The dev saw 6 such WARN lines in that boot.

Unrelated to #21241: that table synced. Filed by domain:engine#1 (seat post #6367, session_017xfMoEjKUuSh2xYB8sCozp). Reader who acts: triage grades and routes; the positions span domain:engine (driver-sql, platform-objects) and domain:services (plugin-auth). ⛔ Not a claim.

Positions (read at origin/main 4727fcb2, ⛔ not measured)

  • The failing statement inserts (alg, created_at, crv, id, private_key, public_key, updated_at). created_at is bound as a UTC wall-clock literal, in the form the driver gives a declared datetime. updated_at's bound value is redacted in the log and was not read.
  • sys_jwks declares created_at but not updated_at (packages/platform-objects/src/identity/sys-jwks.object.ts: id, public_key, private_key, alg, crv, created_at, expires_at). A caller-supplied updated_at value therefore has no declared type for the driver's write formatting to read.
  • The driver's own stamps are ruled out by reading:
    • SqlDriver.stampInsertTimestamps returns early off SQLite.
    • updatedAtStamp() on MySQL is a precision-aware raw expression.
    • So the refused value most likely comes from the caller, the auth adapter's sys_jwks write. A plausible shape is an ISO-8601 string with T and Z, which MySQL refuses as a DATETIME literal; not measured.
  • The SQLite control signs JWTs on the same build: driver-sql on MySQL: create() answers the insert id (0) instead of the inserted record, so sign-up answers 400 FAILED_TO_CREATE_USER, the dev admin seed fails and no user can sign in #21227's measurements show sign-in and get-session working there.

Scope for whoever takes it (⛔ not a ruling)

  • Measure which producer writes the refused updated_at value, and its exact shape.
  • Fix at the producer side: the declaration, the driver's formatting of an undeclared audit column, or the adapter. ⛔ No per-caller workaround that leaves the class open for other undeclared audit columns.
  • Pins: a MySQL cell (the live-dialect testkit's) in which JWT signing succeeds and GET /api/v1/auth/jwks answers 200; the SQLite and PostgreSQL cells unchanged (the control).

Dedupe

mcp__github__search_issues, repo-scoped, open and closed:

None covers this.

Dedupe words: sys_jwks mysql Incorrect datetime value updated_at · auth jwks 500 mysql · JWT signing failed EdDSA mysql · undeclared updated_at datetime mysql


Generated by Claude Code

Activity

  1. objectstack-fleet commented on Oct 1, 2026

    @objectstack-fleet
    ContributorAuthor

    Claim: PM loop round 1
    Session: session_017xfMoEjKUuSh2xYB8sCozp
    Account: huangyiirene (the seat's linked user as GET /user answers it; the card's assignee)
    Branch: claude/issue-21259-mysql-jwks-updated-at
    Worktree: objectstack-issue-21259
    Domain: domain:engine
    Seat: domain:engine#1 (seat post #6367)
    Maintainer order: 「21259你直接派发」 (the maintainer, in this seat's session chat, 2026-10-01T23:2xZ). So this card is dispatched before triage's first grade. On that order, this act adds domain:engine (the seat's lane, where the card's two engine positions sit) together with pm:dispatched. The priority:* and area:* grades stay triage's.
    File surface: measure first. The first step finds which producer writes the refused updated_at value into sys_jwks on MySQL, and its exact shape. The fix goes at that producer:

    • packages/platform-objects/src/identity/sys-jwks.object.ts (the declaration);
    • packages/drivers/driver-sql/src/sql-driver.ts (the write formatting of an undeclared audit column; ⛔ not nowColumnDefault / migrateMysqlDatetimeColumns, which PR fix(driver-sql): a MySQL NOW() datetime default carries its column's precision #21252 is landing);
    • or packages/plugins/plugin-auth/** (domain:services, the adapter). If the producer is here, the dev reports it before editing, and this seat declares the cross-lane surface on the services seat post first.

    Pins: a MySQL live-dialect cell plus the SQLite and PostgreSQL controls, and a changeset. ⛔ No per-caller workaround that leaves the class open.
    Container & model: M (measure-first, then a small fix), mode:subagent, model: opus (dispatch-gates --tier: no path-derived mandate, floor sonnet · default opus · ceiling fable)
    Clause-②: no
    Thread-read: none
    Serial constraints cleared: read at 2026-10-01T23:22Z against origin/main 8dea55d3. PR #21252 (#21241, this seat) edits sql-driver.ts's MySQL datetime precision and widening regions, and is in contract review. This card's driver-sql region, if the measurement lands there, is the write-value formatting, which is region-disjoint; whichever lands second merges main. #21197 (decision box) also names sys-jwks.object.ts (the key-material columns): different fields, and that card is not in flight. #20595 stage 3 is objectql comments.


    Generated by Claude Code

  2. objectstack-fleet commented on Oct 2, 2026

    @objectstack-fleet
    ContributorAuthor

    os-dev-report
    {
    "issue": 21259,
    "status": "done",
    "branch": "claude/issue-21259-mysql-jwks-updated-at",
    "pr": "#21272",
    "session": "session_017xfMoEjKUuSh2xYB8sCozp (subagent: the parent PM session, per the harness Claude-Session line)",
    "premise_still_valid": true,
    "summary": "Fixed in packages/drivers/driver-sql (domain:engine): registerManagedObjectMetadata records the builtin audit columns (created_at/updated_at) a MANAGED object does not declare (undeclaredAuditTimestampFields, recomputed per registration, external objects excluded), and formatInput writes them through storageDatetimeValue, the rule a declared Field.datetime already takes, so every write door (create, bulkCreate, upsert, update, updateMany) binds the column form; nowColumnDefault / migrateMysqlDatetimeColumns untouched; sys_jwks key-material fields untouched; plugin-auth untouched. || H1 confirmed at 8dea55d: sys_jwks declares id, public_key, private_key, alg, crv, created_at, expires_at; the failing INSERT names updated_at; MySQL general log: created_at bound 2026-10-01 23:35:47.598, updated_at bound 2026-10-01T23:35:47.598Z (one instant, two spellings; only non-key columns extracted). || H2 measured, and its "auth adapter" half FALSIFIED: the producer of updated_at is the engine audit hook sys_stamp_audit_insert (packages/objectql/src/plugin.ts applyToRecord, stamp = new Date().toISOString()). Scratch vitest (never committed, deleted): ObjectQLPlugin + capturing driver + real SysJwks from platform-objects dist; caller payload keys id,public_key,private_key,alg,crv,created_at; driver.create received the same plus updated_at (string, toISOString form, equal to the re-stamped created_at); registered fields lack updated_at because managedBy better-auth gets no audit injection. better-auth jwks schema (dist/plugins/jwt/schema.mjs) has no updatedAt. The driver own stamps are not involved (stampInsertTimestamps returns early off SQLite). || H3 measured: the driver passed an undeclared builtin audit column raw (formatInput keyed only on declared datetimeFields; the read side already had an audit_timestamp arm on every dialect). Census over GET /api/v1/meta/object on the CRM boot at 8dea55d (MySQL, 81 objects): 70 declare both, 9 created_at only (sys_member, sys_invitation, sys_team_member, sys_oauth_access_token, sys_oauth_refresh_token, sys_oauth_client_resource, sys_jwks, sys_scim_connection_binding, sys_scim_group_member), 0 updated_at only, 2 neither (sys_oauth_client_assertion, sys_scim_identity_tombstone); every declared audit column is datetime. Baseline boot: sys_jwks refused 5x and sys_member 2x on updated_at (seeded admin had 0 membership rows). || H4: fix site driver-sql, so no SendMessage to the seat was needed (producer not in plugin-auth). H5: MySQL 8.0.46 and PG 16.14 binaries present; no live-dialect harness boots plugin-auth, so the door pin follows the #21227 option B precedent: driver pin of record + one-off door measurement in the PR body. || Door, MySQL 8.0.46 (+08:00), pnpm dev:crm -- --fresh --database mysql://..., seeded admin session. BEFORE (8dea55d): /auth/jwks 500, /auth/token 500, /auth/get-session 200 with NO set-auth-jwt header, sys_jwks 0 rows, sys_member 0 rows, 7 Insert-operation-failed WARN lines (Incorrect datetime value). AFTER (driver-sql dist rebuilt at d18427f): /auth/jwks 200 (1 OKP Ed25519 EdDSA public key, no private member), /auth/token 200 (3-segment JWT, header alg EdDSA + kid), get-session 200 with set-auth-jwt present, sys_jwks 1 row and sys_member 1 row (created_at = updated_at, UTC wall clock), 0 Incorrect-datetime lines. Control after fix: PG 16.14 (Asia/Shanghai) and SQLite: all four doors 200, header present, 0 failed inserts. NOT MEASURED: door-before on pg/sqlite, reason: not run; the driver pin PG/SQLite cells are the control of record (green before and after). || Clause-2: no, copied line-start into the PR body and the changeset. Reasoning, for the PM to judge: no authorable or API accept set moves (the audit columns are readonly to clients); the driver write door stops producing a literal MySQL refuses for a value the declared-datetime path already took on every dialect. One storage-form delta on SQLite: a JS Date written to an UNDECLARED audit column is now stored as canonical ISO text (was epoch INTEGER, read back as a number) — the declared-datetime rule; declared in the changeset and pinned. || Files changed (3, +262/-0, HEAD e7586a8): .changeset/21259-undeclared-audit-timestamp-mysql.md (+11, driver-sql patch); packages/drivers/driver-sql/src/sql-driver.ts (+42); packages/drivers/driver-sql/src/sql-driver-21259-undeclared-audit-timestamp-write.test.ts (+209). line_budget: n/a (no skills/** path). || DEVIATIONS: (1) label-write.mjs --repo objectstack-ai/objectstack --issue 21272 --assign huangyiirene was DENIED by the local auto-mode classifier ([External System Writes]) before any network call; not retried by another route; the PR assignee is EMPTY and is the seat to set. (2) Hygiene slip: during the baseline door reading I grepped the boot log with a redaction sed whose pattern did not match the backticked column spelling, so ~150 hex chars of the ENCRYPTED private_key value of a throwaway key (never stored: sys_jwks held 0 rows; database dropped, server data dir deleted) were printed into this session transcript only; nothing was posted anywhere. Every later read went through a column-whitelisting extractor. (3) Temp server dirs: the PG data dir and the MySQL socket lived under /tmp/os21259-* because the scratchpad path is not traversable by the postgres user and exceeds the 107-byte socket limit; both removed. All servers I started (mysqld pid 14481, postgres pid 6726, four dev servers by recorded session group) are stopped.",
    "tests": "Pin (sql-driver-21259-undeclared-audit-timestamp-write.test.ts, declareDialectCell over sqlite + live pg + live mysql; env TZ=America/New_York OS_TEST_MYSQL_URL OS_TEST_POSTGRES_URL OS_EXPECT_LIVE_DIALECT_MATRIX=1; pnpm --filter @objectstack/driver-sql exec vitest run --maxWorkers=2 FILE under os-verify-lock): BEFORE the fix (base code, 21 cases) VERDICT command-exit 1, 6 failed / 15 passed: live mysql create x2, bulkCreate, upsert, update, each "Incorrect datetime value: 2026-10-01T21:49:27.479Z", + sqlite Date case "expected 1790891367479 to be 2026-10-01T21:49:27.479Z"; live postgres 0 failures. AFTER at e7586a8: VERDICT command-exit 0, Tests 24 passed (24) = 8 cases x 3 cells. || Ablation at e7586a8 (committed first): node scripts/ablation-replace.mjs --file packages/drivers/driver-sql/src/sql-driver.ts --anchor "const undeclaredAudit = this.undeclaredAuditTimestampFields[object];" --replacement "...= undefined; // ABLATION-21259" -- (pin run): "ok mutation landed: anchor 1 -> 0, blob d08cb7e95a4a -> 5e6e99e8b352"; result 7 failed / 17 passed = live mysql 6 (create x2, bulkCreate, upsert, update, updateMany) + sqlite 1 (Date case), live postgres 0 — the predicted direction; "ok restored: blob == HEAD (d08cb7e95a4a) and git diff HEAD is empty"; git status --porcelain 0 lines; marker grep -c 0. No dist leg: the pin imports ./index.js (source), not the package exports. || driver-sql whole suite with live PG 16.14 + MySQL 8.0.46 (same env) at d18427f (e7586a8 adds only a test case + changeset; sql-driver.ts unchanged): VERDICT command-exit 0, Test Files 224 passed (224), Tests 5390 passed | 1 skipped (5391), reporter "all 3 dialects were exercised (sqlite, live postgres, live mysql)". || Inheritors: pnpm --filter @objectstack/driver-sqlite-wasm exec vitest run --maxWorkers=2: 36 files, 675 passed; pnpm --filter @objectstack/driver-turso exec vitest run --maxWorkers=2: 87 files, 2349 passed | 33 skipped; typecheck driver-sql / driver-sqlite-wasm / driver-turso: VERDICT command-exit 0; driver-sql typecheck re-run at e7586a8: exit 0, and tsc --listFiles includes the new test file (count 1). || Gates at e7586a8: node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands derived 63 commands (stderr line 1: tree objectstack-ai/objectstack at e7586a8); all 63 run, all exit 0 (incl. pnpm check:driver-conformance, check:nul-bytes, check:engine-double-contract, check:test-source-alias, check:cross-package-test-inputs, check:type-check-coverage, check:type-check-debt, check-adr-0087-registration --base origin/main, check-changeset-no-major --base origin/main, check-empty-changeset); --ran reconciliation: "63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN" exit 0. pnpm check:driver-conformance before the first edit: exit 0 ("50 covered cell(s)"). || Lint: targeted npx eslint --no-inline-config on the 2 touched TS files --format json: exit 0, 2 files, 0 errors, 0 warnings — a spot check, NOT a narrowed lint measurement (population and invariance proofs not taken). NOT MEASURED: pnpm lint (repo scan), reason: CI-owned. || Door measurement (one-off, not a test): see summary; PR body table.",
    "mcp_calls": "0 — no MCP GitHub tool used (reads via gh api REST: issue 21259, its comments, comment 5942461506, PR 21272)",
    "api_writes": "2 sent through scripts/pm — (1) with-fleet.sh --via dispatch pr_create: repository_dispatch fw-20261002T001648Z-295eec, relay run 36945187146 success, POST /repos/objectstack-ai/objectstack/pulls (draft) -> #21272, read-back 9010 bytes identical; (2) this os-dev-report comment via the same relay (op comment, POST /repos//issues/21259/comments). label-write (PR assignee): 0 writes, denied locally by the classifier before the network. git push x3 to claude/issue-21259-mysql-jwks-updated-at (not REST).",
    "open_questions": [],
    "out_of_scope_findings": [
    "class: a · reach: exception: security (could leak data) — measured at a public door: on MySQL at 8dea55d the failing sys_jwks INSERT behind GET /api/v1/auth/get-session made the [Better Auth] ERROR logger print the whole statement WITH bound values (the encrypted private_key value and the public_key among them), on three carriers: the "[Better Auth]:" ERROR line, a "# SERVER_ERROR: Error: insert into ..." line and the error object dump (sql: and sqlMessage: properties), while the engine WARN line for the SAME failure printed "[statement and bound values redacted]" · evidence: boot log of pnpm dev:crm -- --fresh --database mysql://... (lines 202/203/218 and repeats, 11 occurrences of the statement in one boot; that scratch log was deleted after reading because it held the key ciphertext); the redaction is applied where the engine logs, but the thrown error reaching better-auth still carries the raw knex message and its sql property; any driver error on an auth table write would print its values (password hashes, tokens) the same way; not caused by this card, and with this PR the sys_jwks insert no longer fails so this occurrence disappears but the path stays · Seam: runtime:objectql/driver-sql error envelope (message redacted only in the engine log line) → runtime:plugin-auth better-auth logger (logs the raw error) · dedupe words: "better auth logger bound values redacted" · "SERVER_ERROR insert statement private_key log" · "statement and bound values redacted better-auth" · "auth adapter error sql property leaked log"",
    "carrier: PR #21272 Acceptance notes · sql-driver-11176-bulk-and-merge-updated-at.test.ts §2 routes around this same class in its fixture (sends toISOString() on SQLite and a Date elsewhere for an undeclared updated_at); left untouched · noted, not filed",
    "carrier: PR #21272 Acceptance notes · the filter half was measured, not assumed: on live MySQL a $gt and an equality comparand in canonical ...Z text against an UNDECLARED updated_at answered the right rows, so no read-side change · noted, not filed",
    "carrier: PR #21252 (#21241, in flight) · sys_activity table missing on MySQL in the same boots (28 Insert-operation-failed lines in the baseline boot); #21241 is not addressed here · noted, not filed"
    ]
    }

  3. objectstack-fleet commented on Oct 2, 2026

    @objectstack-fleet
    ContributorAuthor

    ACCEPT — PR #21272 at head e7586a892e7b02bef5d0635f616d15070c45e8a7

    domain:engine#1 · session_017xfMoEjKUuSh2xYB8sCozp · read at 2026-10-02T00:39Z. Judged against GitHub.

    • Shape: draft, base main, Fixes #21259 and no other closing keyword. No path is governed. 3 files, +262/-0: sql-driver.ts (+42), a per-dialect pin file, and the changeset (driver-sql patch, Clause-②: no at line start in the changeset and the body).
    • The producer (measured, H2's adapter half falsified): the engine's sys_stamp_audit_insert hook (objectql plugin.ts applyToRecord) stamps updated_at as toISOString() text on every insert. sys_jwks (managedBy: 'better-auth') declares created_at only, so the driver passed the undeclared column raw, and MySQL refused the literal. plugin-auth and sys-jwks.object.ts are untouched.
    • The fix:
      • registerManagedObjectMetadata records, per table, the builtin audit columns a managed object does not declare. The entry is recomputed on every registration, and external objects never get one.
      • formatInput writes those columns through storageDatetimeValue, the rule a declared Field.datetime already takes. All six managed write doors format before they stamp, the shard branch of update included.
      • This closes the class: in the CRM boot's census (81 objects), 9 declare created_at only and 2 declare neither. Declaring updated_at on sys_jwks alone would have closed 1 of the 11.
    • Measured at the doors (dev, MySQL 8.0.46):
      • Before: /auth/jwks and /auth/token answered 500, and sys_jwks and sys_member held 0 rows.
      • After: both doors answer 200 with an EdDSA key and a 3-segment JWT, and the key and membership rows are stored.
      • PostgreSQL 16 and SQLite after the fix: all four doors answer 200.
    • The door pin: the review escalated it to the seat. The seat confirms the driver-sql on MySQL: create() answers the insert id (0) instead of the inserted record, so sign-up answers 400 FAILED_TO_CREATE_USER, the dev admin seed fails and no user can sign in #21227 precedent applies (seat answer B, 5940180378): no live-dialect harness boots plugin-auth. So the driver pin is the pin of record, in the required Temporal Conformance (live PG + MySQL) job, and the door is a recorded one-off measurement. This is verification strategy, not an escalation.
    • Verification:
      • Before the fix, 6 of 21 cases fail: 5 live MySQL doors plus the SQLite Date case, with PostgreSQL green as the control.
      • The ablation at the head reddens 7, adding updateMany, and the restore is proven by blob identity.
      • The whole driver-sql suite, live on MySQL and PostgreSQL: 5390 passed. Inheritors driver-sqlite-wasm (675) and driver-turso (2349) pass.
      • Gates: 63 derived, 63 run, all exit 0.
    • Contract review: PASS at CONTRACT_REVIEW_TIER on this head, record 5943346872. Every derived judgment is right. The SQLite storage-form delta (a Date lands as canonical text, not epoch INTEGER) moves toward ADR-0053 D-F1, has no in-tree reader of the integer, and is declared in the changeset.
    • The seat's own read of the diff: it agrees. The arm sits after the declared-datetime arm, skips null, copies before writing, and leaves junk to fail loudly at the server.
    • CI on this head, read by the seat in this act: 34 check runs, 31 success, every required context included (Test Core, TypeScript Type Check, Lint & Repo Gates, Build Core, Dogfood Regression Gate, Governed Surface Queue Guard, Temporal Conformance (live PG + MySQL)), and 0 failures. The 3 skips (Build Docs, Console Pin Gate, Packed-tarball smoke (opt-in)) are roster entries in check-expected-skips.mjs. The one legacy status is success.
    • Deviations:
      • The PR's assignee is empty. The dev's write was refused by its session's permission layer and not routed around. It waits on the maintainer, and the seat does not perform it on the dev's behalf.
      • A hygiene slip stayed in the dev's local transcript: part of a throwaway key's ciphertext, for a key that was never stored and whose database was dropped. The review swept the diff, the PR body, both threads and the card, and nothing posted carries key material.
      • The temporary server directories were removed.
    • Out-of-scope findings:

    Next: pr_ready, then automerge_enable, as two relay acts. Fixes closes this card at merge.


    Generated by Claude Code

  4. objectstack-fleet commented on Oct 2, 2026

    @objectstack-fleet
    ContributorAuthor

    Landed: PR #21272 → 95b91cc85 on main, verified at 2026-10-02T01:08Z. domain:engine#1 · session_017xfMoEjKUuSh2xYB8sCozp.


    Generated by Claude Code

  5. added 2 commits that reference this issue on Oct 7, 2026
    95b91cc
    50e1c65
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions