Skip to content

Handle empty query stems safely in query side stemming - #36377

Merged
bratseth merged 2 commits into
vespa-engine:masterfrom
SahilKumar75:issue-33540-query-and-stopword-stemming
Apr 10, 2026
Merged

Handle empty query stems safely in query side stemming#36377
bratseth merged 2 commits into
vespa-engine:masterfrom
SahilKumar75:issue-33540-query-and-stopword-stemming

Conversation

@SahilKumar75

Copy link
Copy Markdown
Contributor

Implements issue #33540.

This PR handles empty-stem query terms in query-side stemming with a narrow scope.

Changes:

  • Keep phrase and non-text operator behavior unchanged.
  • Safely drop empty-stem terms only in query-side AND/WeakAND contexts when at least one sibling term remains.
  • Do not allow the whole query to collapse to empty.
  • Preserve connectivity-sensitive terms.

Regression tests added for:

  • Query.And: drop empty-stem term when another term remains.
  • Query.And: do not collapse to empty.
  • userInput path: drop empty-stem term when safe.
  • userInput path: do not collapse to empty.
  • Phrase behavior unchanged.
  • Non-text operator behavior unchanged.

Verification:

  • mvn -pl container-search -Dtest=StemmingSearcherTestCase test
  • mvn -pl container-search -Dtest=StemmingSearcherTestCase,WeakAndTestCase,StopwordTestCase test

I confirm that this contribution is made under the terms of the license found in the root directory of this repository's
source tree and that I have the authority necessary to make this contribution on behalf of its copyright owner.

@SahilKumar75

Copy link
Copy Markdown
Contributor Author

Buildkite is still running.

The Link checker failure appears to be workflow/infrastructure-related, not code-related:
actions/create-github-app-token@v2 failed with [@octokit/auth-app] appId option is required after htmlproofer completed successfully.

Could a maintainer please rerun the Link checker check when convenient?

@SahilKumar75

Copy link
Copy Markdown
Contributor Author

@bratseth @hmusum could you review when convenient? Buildkite is still running; Link checker failed due workflow infra and may need maintainer rerun.

@bratseth bratseth left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider these simplifications, otherwise LGTM!

@bratseth
bratseth merged commit 1691d8f into vespa-engine:master Apr 10, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants