Skip to content

SQL SELECT * drops dynamic attributes on schema-less tables #2734

Open

Description

@kriszyp

SELECT * over SQL returns only a schema-less table's declared attributes, silently dropping every dynamically-added one, behind a 200. REST and search_by_hash return the full record for the same table, so the same data reads differently depending on the surface.

Repro shape

A non-@sealed (schema-less) table Loose holding four attributes: SELECT * FROM data.Loose returns ["id"], while an identical-data declared control table returns all four. REST and search_by_hash return all four on both tables.

Where it comes from, and the one detail that needs resolving

The SQL engine v2 rewrite (#1285, merged 2026-07-17; default flipped legacy → auto on 2026-06-21, i.e. before this was measured) looks like it should already do the right thing:

  • sqlEngine/optimizer/rules/projectionPushdown.ts leaves a Scan's projection undefined ("fetch all attributes") whenever * appears.
  • sqlEngine/PhysicalProject.ts's star branch forwards Object.keys(row) — the row's actual keys — not a declared list.

But in resources/Table.ts the two relevant helpers disagree when target.select is unset:

  • getColumns() falls back to the declared attributes list (the describe_table-sourced schema) — this is the claimed root cause.
  • transformEntryForSelect()'s per-row transform returns the full raw record, unfiltered.

Which of those governs the observed result is the discriminating detail for whoever picks this up; a source read alone did not settle it.

Why this is not already fixed by the describe work

harper#664 ("describe_table only returns id when schema_defined: true") was closed 2026-07-15 by PR #1804 — but #1804 deliberately declined this fix: it kept schema_defined: true scoping intentional and unchanged, left attributes reporting only the declared schema, and added a new undeclared_attributes diagnostic field instead of merging schema with data-derived attributes. It never changed what SELECT * returns.

Independent confirmation that it is still live

harper#2333 (OPEN, filed 2026-07-29 by a later, independent QA wave) references this exact defect as still-live and unfiled — calling it "F-190 … which drops dynamic attributes on schema-less tables" — and notes it is a distinct code path from #2333's own @computed-scalar bug. That is post-cutover, post-#664 corroboration from a separate source.

Framing note

Do not file this as an "unregistered attribute metadata" bug: PR #1804 shows describe_table's attribute list was left scoped on purpose. The defect is that the SQL star-expansion consumes that deliberately-scoped list as if it were the record's shape.

(qa-explorer campaign, F-190; verified against origin/main 2026-09-04. The finding's original attachment to harper#1163 is a misattachment — #1163 is cross-node shared-structure ID divergence, unrelated.)

Activity

  1. added this to the v5.3 milestone on Sep 22, 2026
  2. added theissue type on Sep 22, 2026
  3. modified the milestones: v5.3, v5.4 on Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    Priority

    P1

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions