Skip to content

Table rows lose their header binding when rendered, making tabular content near-unretrievable #14971

Open

Description

@mrveiss

Problem

Where table content does survive into the knowledge base, each row is rendered without
any binding to its header row. Once chunked and embedded, a row is a sequence of values
whose meaning lives in a different part of the text, or nowhere.

Evidence

  • autobot-backend/utils/document_parser.py, _parse_docx renders rows as
    " | ".join(row) under a --- Tables --- marker. Row 0 (typically the header) is
    rendered identically to every data row, with nothing associating them.
  • autobot-backend/media/document/extraction.py, _normalize_table produces a bare
    List[List[str]] — a grid with no header concept.
  • No header-propagation logic exists anywhere in media/document/.

A row embedded as a bare value sequence is close to unretrievable: the column names
that made it meaningful are not in the vector.

Proposed approach

Heuristic reassembly at render time, no model:

  • Treat row 0 as a header row when it is present and non-empty, and emit each data row
    as header-to-value pairs so each row carries its own column semantics.
  • Record which strategy was applied (header-paired vs raw grid) in chunk metadata.

Risks / constraints

Acceptance criteria

  • A data row's rendered text contains its column names
  • A table with no detectable header falls back to the raw grid rendering
  • The applied strategy is recorded in chunk metadata in both cases
  • A test retrieves a specific cell value by querying its column name plus a row key

Part of #14967

Blocked by: #14970 (needs tables reaching an ingest path before their rendering matters)
Blocks: Nothing.

Implementation order for the whole umbrella is in #14967.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions