Skip to content

fix(opensearch): don't merge same-field ranges under OR#3589

Open
chuenchen309 wants to merge 1 commit into
deepset-ai:mainfrom
chuenchen309:fix/opensearch-or-range-merge
Open

fix(opensearch): don't merge same-field ranges under OR#3589
chuenchen309 wants to merge 1 commit into
deepset-ai:mainfrom
chuenchen309:fix/opensearch-or-range-merge

Conversation

@chuenchen309

Copy link
Copy Markdown
Contributor

The bug

Both the top-level _parse_logical_condition and the nested-field grouping call _normalize_ranges for every multi-condition group:

if len(conditions) > 1:
    conditions = _normalize_ranges(conditions)

_normalize_ranges merges range clauses on the same field into one (via dict.update). That's correct for AND — price > 10 AND price < 100 should become one {"gt": 10, "lt": 100} range. But it also runs for OR, where it's wrong:

{
    "operator": "OR",
    "conditions": [
        {"field": "meta.price", "operator": "<", "value": 10},
        {"field": "meta.price", "operator": ">", "value": 100},
    ],
}

renders as:

{"bool": {"should": [{"range": {"price": {"lt": 10, "gt": 100}}}]}}

A single range clause with both lt: 10 and gt: 100 means price < 10 AND price > 100, which matches no document — even though the user asked for price < 10 OR price > 100. The same collapse happens inside nested-field groups (_group_nested_conditions), dropping a bound of an OR of ranges on a nested sub-field.

This is a sibling of the pgvector/elasticsearch range handling; the correct output keeps two separate range clauses in should.

The fix

Only merge ranges when the conditions are AND-combined, at both sites (top level and nested groups). AND and the inner must of NOT keep merging; OR skips it:

if len(conditions) > 1 and operator != "OR":
    conditions = _normalize_ranges(conditions)

Tests

Added two regression tests:

  • test_or_of_ranges_on_same_field_are_not_merged — top-level OR of two same-field ranges stays two separate range clauses.
  • test_or_of_ranges_on_same_nested_field_are_not_merged — same, inside a nested-field group.

Both fail before the change (ranges collapse into one impossible range) and pass after. I verified each fix site is load-bearing (reverting the nested guard alone makes the nested test fail).

Verified locally: hatch run test:unit passes (167 passed), hatch run fmt and hatch run test:types are clean.


Authored with AI assistance (Claude Code); the change and tests were reviewed and verified locally by me.

@github-actions

Copy link
Copy Markdown
Contributor

Coverage report (pgvector)

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  integrations/pgvector/src/haystack_integrations/document_stores/pgvector
  filters.py 230-231
Project Total  

This report was generated by python-coverage-comment-action

@github-actions

Copy link
Copy Markdown
Contributor

Coverage report (chroma)

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  integrations/chroma/src/haystack_integrations/document_stores/chroma
  filters.py 68, 74
Project Total  

This report was generated by python-coverage-comment-action

@github-actions

Copy link
Copy Markdown
Contributor

Coverage report (opensearch)

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  integrations/opensearch/src/haystack_integrations/document_stores/opensearch
  filters.py
Project Total  

This report was generated by python-coverage-comment-action

@github-actions

Copy link
Copy Markdown
Contributor

Coverage report (elasticsearch)

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  integrations/elasticsearch/src/haystack_integrations/document_stores/elasticsearch
  filters.py
Project Total  

This report was generated by python-coverage-comment-action

_parse_logical_condition and the nested-field grouping both ran
_normalize_ranges for every multi-condition group, but merging same-field
range clauses is only valid when they are AND-combined. Under OR,
`price < 10 OR price > 100` collapsed into a single {"lt": 10, "gt": 100}
range clause (meaning price < 10 AND price > 100), matching no document.
Skip range merging for OR at both the top level and inside nested-field
groups; AND and the inner `must` of NOT keep it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@chuenchen309
chuenchen309 force-pushed the fix/opensearch-or-range-merge branch from 7d362c9 to 84d5b6d Compare July 14, 2026 10:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants