Skip to content

A welded co-element filter hangs Typesense when the nested fields hold arrays #798

Description

@ddeboer

The co-element filter #771 was built for – this agent in this role – hangs Typesense 30.2 indefinitely whenever the nested fields hold arrays, which is every document a qualified relation produces in practice.

Reproducer

One document, typesense/typesense:30.2, no LDE code in the path:

curl -XPOST 'http://localhost:8108/collections' -H 'X-TYPESENSE-API-KEY: dev' -d '{
  "name": "weld", "enable_nested_fields": true,
  "fields": [ {"name": "credits", "type": "object[]"},
              {"name": "credits.role", "type": "string[]"},
              {"name": "credits.agent_id", "type": "string[]"} ] }'

# array-valued nested fields
curl -XPOST 'http://localhost:8108/collections/weld/documents' -H 'X-TYPESENSE-API-KEY: dev' \
  -d '{"id": "w1", "credits": [{"role": ["etser"], "agent_id": ["p1"]}]}'

curl -G 'http://localhost:8108/collections/weld/documents/search' -H 'X-TYPESENSE-API-KEY: dev' \
  --data-urlencode 'q=*' --data-urlencode 'query_by=' \
  --data-urlencode 'filter_by=credits.{role:=etser && agent_id:=p1}'

The search never returns. The identical document with scalar nested values – {"role": "etser", "agent_id": "p1"} – answers 1 hit instantly, same schema and same filter.

What we narrowed it to

  • It is a hang, not a slow scan: 0.4 % CPU throughout, inflight_searches: 0 in the log, no response in 300 s. Other queries keep being served normally.
  • The trigger is the array inside each object, not the syntax, the field types or the values. These all answer in milliseconds: one condition inside the braces (either field); the two conditions joined outside the braces (credits.role:=etser && credits.agent_id:=p1); two string[] conditions welded over scalar-valued objects; and Typesense’s own documented example, ingredients.{name:=cheese && concentration:<50}, whose objects also hold scalars.
  • IRIs, backquoting and the partial-match operator make no difference.

The docs’ example holding scalars is presumably why the design work behind #771 cleared the query side against a live 30.2 container.

Why it reaches every real document

A qualified relation declares its edge fields array: true for good reasons – a role may be stated once per language, and the endpoint is multi-valued – so the projected entries hold arrays and the welded filter is unusable across the whole collection. Measured on a real index: creator.role:=fotograaf answers 46 works in 0.0 s, creator.creator_id:=<iri> answers 22 works in 0.0 s, and creator.{role:=fotograaf && creator_id:=<iri>} never returns. Through the GraphQL API that surfaces as Unexpected error, the 5 s Typesense timeout retried once.

So the data is fine – roles, agents and the resolved lookup are all indexed and readable, and either condition alone filters – but the weld itself, the one thing a nested edge buys over two flat fields, cannot be asked. Unwelded, role: fotograaf && creator: X still matches a work where X is the publisher and somebody else is the photographer.

Suggestion

The fix looks like Typesense’s: the syntax is documented and the input valid. What seems worth doing here is a note in #771’s documentation on which value shapes the capability can serve today, and – if it is worth dodging until Typesense fixes it – deciding whether a single-valued edge field should project a scalar rather than a one-element array.

Downstream context: https://codeberg.org/limburg/lol/issues/162.

Activity

  1. added theissue type on Sep 1, 2026
  2. ddeboer commented on Sep 1, 2026

    @ddeboer
    MemberAuthor

    Two follow-ups from tracing which queries can actually reach this, both measured on a fresh typesense/typesense:30.2.

    The dotted-path explanation in compileWelded does not hold

    query-compiler.js states it as a correctness requirement:

    Typesense 30.2 hangs on a dotted path inside a group (creator.{creator.name:=…} never returns – no error, no result), so the compiler must be structurally incapable of emitting one.

    The hang is real, but the dotted path is not what causes it. On a document shaped like a projected edge, with creator.creator a nested object[]:

    filter_by result
    creator.{creator.name_und:=[Leufkens]} – dotted path inside the group 1 hit, 0.0 s
    creator.{role:=[fotograaf] && creator_id:=[p1]} – two leaf conditions hangs

    So a dotted path inside a group answers fine, and the leaf-only form the compiler guarantees is the one that hangs. The trigger is what the first comment describes: two &&-joined conditions inside one group where the nested fields hold arrays. Whatever the dotted-path run showed at the time, keeping conditions leaf-only does not avoid this, and the stated guarantee is load-bearing in a way it cannot deliver.

    Only a multi-criterion where reaches it

    Worth recording, because it bounds the blast radius. compileMembership renders a multi-value in as one condition (role:=[a,b]), so the braces hold two conditions only when a caller puts two criteria inside one edge’s where. Everything else a consumer can send today is fine:

    what the caller writes emitted result
    creator: { in: [iri] } creator_id:=[p1] 1 hit, 0.0 s
    creator: { where: { creator: { in: [iri] } } } creator.{creator_id:=[p1]} 1 hit, 0.0 s
    creator: { where: { role: { in: ["fotograaf"] } } } creator.{role:=[fotograaf]} 1 hit, 0.0 s
    creator: { where: { role: { in: ["fotograaf", "uitgever"] } } } creator.{role:=[fotograaf,uitgever]} 1 hit, 0.0 s
    creator: { where: { creator: {…}, role: {…} } } creator.{role:=… && creator_id:=…} hangs

    So the exposed surface is safe apart from the one query the feature exists for – which is also the example in its own documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions