Skip to content

[ES|QL] Add MATCH_PHRASE #127661

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 54 commits into from
Jun 4, 2025
Merged

Conversation

kderusso
Copy link
Member

@kderusso kderusso commented May 2, 2025

Adds MATCH_PHRASE support to ES|QL.

Some examples of how to use MATCH_PHRASE:

POST _query?format=txt
{
  "query": """
  FROM books METADATA _score
  | WHERE MATCH_PHRASE(title, "return of the king")
  | KEEP title, _score
  | SORT _score
  | LIMIT 5
  """
}

POST _query?format=txt
{
  "query": """
  FROM books METADATA _score
  | WHERE MATCH_PHRASE(title, "return of king", {"slop": 5})
  | KEEP title, _score
  | SORT _score
  | LIMIT 5
  """
}

POST _query?format=txt
{
  "query": """
  FROM books METADATA _score
  | WHERE MATCH_PHRASE(title, "return of king", {"analyzer": "whitespace"})
  | KEEP title, _score
  | SORT _score
  | LIMIT 5
  """
}

POST _query?format=txt
{
  "query": """
  FROM books METADATA _score
  | WHERE MATCH_PHRASE(title, "", {"zero_terms_query": "all"})
  | KEEP title, _score
  | SORT _score
  | LIMIT 5
  """
}

@kderusso kderusso force-pushed the kderusso/esql-match-phrase branch from dabef06 to 13ccda7 Compare May 6, 2025 19:32
@kderusso kderusso added :SearchOrg/Relevance Label for the Search (solution/org) Relevance team >enhancement labels May 7, 2025
@elasticsearchmachine
Copy link
Collaborator

Hi @kderusso, I've created a changelog YAML for you.

Copy link
Member

@carlosdelest carlosdelest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's important that we register the match_phrase as snapshot and get some tests ready for release builds.

Besides that, some small nits.

@@ -221,7 +221,7 @@ setup:
- gt: {esql.functions.to_long: $functions_to_long}
- match: {esql.functions.coalesce: $functions_coalesce}
- gt: {esql.functions.categorize: $functions_categorize}
- length: {esql.functions: 134} # check the "sister" test above for a likely update to the same esql.functions length check
- length: {esql.functions: 135} # check the "sister" test above for a likely update to the same esql.functions length check
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully those errors get corrected when registering the function under snapshot functions 🤞

@@ -221,7 +221,7 @@ setup:
- gt: {esql.functions.to_long: $functions_to_long}
- match: {esql.functions.coalesce: $functions_coalesce}
- gt: {esql.functions.categorize: $functions_categorize}
- length: {esql.functions: 134} # check the "sister" test above for a likely update to the same esql.functions length check
- length: {esql.functions: 135} # check the "sister" test above for a likely update to the same esql.functions length check
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you'll need to add match_phrase capability to these tests, otherwise they will fail for release as the function will not be registered

@ioanatia ioanatia removed the test-release Trigger CI checks against release build label Jun 4, 2025
@ioanatia
Copy link
Contributor

ioanatia commented Jun 4, 2025

@elasticmachine test this please

@ioanatia
Copy link
Contributor

ioanatia commented Jun 4, 2025

I checked the release tests failures - there is nothing pertaining to match_phrase - I removed the test-release label since all the match_phrase failures for release tests were fixed.

@kderusso kderusso requested a review from carlosdelest June 4, 2025 13:57
Copy link
Member

@carlosdelest carlosdelest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, great work! 💯

There are other tests that can be added to VerifierTests - look for EsqlCapabilities.Cap.MULTI_MATCH_FUNCTION.isEnabled() and you'll be able to add an equivalent one for match_phrase

@kderusso kderusso merged commit eee423a into elastic:main Jun 4, 2025
18 checks passed
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.19 Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 127661

kderusso added a commit to kderusso/elasticsearch that referenced this pull request Jun 10, 2025
* Initial commit of match_phrase

* Add MatchPhraseQueryTests

* First pass at CSV specs

* Update docs/changelog/127661.yaml

* Refactor so MatchPhrase doesn't use all fulltext test cases, just text only

* Fix tests

* Add some CSV test cases

* Fix test

* Update changelog

* Update tests

* Comment out MATCH_PHRASE in search-functions Markdown

* Minor PR feedback

* PR feedback - refactor/consolidate code

* Add some more tests

* Fix some tests

* [CI] Auto commit changes from spotless

* Fix tests

* PR feedback - add tests, support boost and numeric data

* Revert "PR feedback - add tests, support boost and numeric data"

This reverts commit 4e7a699.

* Apply testing/PR feedback outside numeric support only

* Regenerate docs

* Add negative test

* Update x-pack/plugin/esql/qa/testFixtures/src/main/resources/match-phrase-function.csv-spec

Co-authored-by: Carlos Delgado <6339205+carlosdelest@users.noreply.github.com>

* Update x-pack/plugin/esql/qa/testFixtures/src/main/resources/match-phrase-function.csv-spec

Co-authored-by: Carlos Delgado <6339205+carlosdelest@users.noreply.github.com>

* Update x-pack/plugin/esql/qa/testFixtures/src/main/resources/match-phrase-function.csv-spec

Co-authored-by: Carlos Delgado <6339205+carlosdelest@users.noreply.github.com>

* PR feedback

* Fix auto-commit error

* Regenerate docs

* Update x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/MatchPhrase.java

Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>

* Remove non text field types

* Fake test data

* Remove tests that no longer should pass without ip/date/version support

* Put real data in score tests now that I was able to engineer a failure

* Realized the scoring test might be flakey because how it was written, updated

* PR feedback

* PR feedback

* [CI] Auto commit changes from spotless

* Add check to MatchPhrase tests

* Fix merge errors

* [CI] Auto commit changes from spotless

* Test generated docs

* Add additional verifier tests

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
Co-authored-by: Carlos Delgado <6339205+carlosdelest@users.noreply.github.com>
Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>
(cherry picked from commit eee423a)

# Conflicts:
#	docs/reference/esql/functions/description/match_phrase.md
#	docs/reference/esql/functions/examples/match_phrase.md
#	docs/reference/esql/functions/kibana/definition/match_phrase.json
#	docs/reference/esql/functions/kibana/docs/match_phrase.md
#	docs/reference/query-languages/esql/_snippets/lists/search-functions.md
#	docs/reference/query-languages/esql/functions-operators/search-functions.md
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/EsqlFunctionRegistry.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/FullTextFunction.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/Match.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/QueryString.java
#	x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/analysis/VerifierTests.java
@kderusso
Copy link
Member Author

💚 All backports created successfully

Status Branch Result
8.19

Questions ?

Please refer to the Backport tool documentation

@kderusso
Copy link
Member Author

Backport PR: #129215

kderusso added a commit that referenced this pull request Jun 12, 2025
* [ES|QL] Add MATCH_PHRASE (#127661)

* Initial commit of match_phrase

* Add MatchPhraseQueryTests

* First pass at CSV specs

* Update docs/changelog/127661.yaml

* Refactor so MatchPhrase doesn't use all fulltext test cases, just text only

* Fix tests

* Add some CSV test cases

* Fix test

* Update changelog

* Update tests

* Comment out MATCH_PHRASE in search-functions Markdown

* Minor PR feedback

* PR feedback - refactor/consolidate code

* Add some more tests

* Fix some tests

* [CI] Auto commit changes from spotless

* Fix tests

* PR feedback - add tests, support boost and numeric data

* Revert "PR feedback - add tests, support boost and numeric data"

This reverts commit 4e7a699.

* Apply testing/PR feedback outside numeric support only

* Regenerate docs

* Add negative test

* Update x-pack/plugin/esql/qa/testFixtures/src/main/resources/match-phrase-function.csv-spec

Co-authored-by: Carlos Delgado <6339205+carlosdelest@users.noreply.github.com>

* Update x-pack/plugin/esql/qa/testFixtures/src/main/resources/match-phrase-function.csv-spec

Co-authored-by: Carlos Delgado <6339205+carlosdelest@users.noreply.github.com>

* Update x-pack/plugin/esql/qa/testFixtures/src/main/resources/match-phrase-function.csv-spec

Co-authored-by: Carlos Delgado <6339205+carlosdelest@users.noreply.github.com>

* PR feedback

* Fix auto-commit error

* Regenerate docs

* Update x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/MatchPhrase.java

Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>

* Remove non text field types

* Fake test data

* Remove tests that no longer should pass without ip/date/version support

* Put real data in score tests now that I was able to engineer a failure

* Realized the scoring test might be flakey because how it was written, updated

* PR feedback

* PR feedback

* [CI] Auto commit changes from spotless

* Add check to MatchPhrase tests

* Fix merge errors

* [CI] Auto commit changes from spotless

* Test generated docs

* Add additional verifier tests

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
Co-authored-by: Carlos Delgado <6339205+carlosdelest@users.noreply.github.com>
Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>
(cherry picked from commit eee423a)

# Conflicts:
#	docs/reference/esql/functions/description/match_phrase.md
#	docs/reference/esql/functions/examples/match_phrase.md
#	docs/reference/esql/functions/kibana/definition/match_phrase.json
#	docs/reference/esql/functions/kibana/docs/match_phrase.md
#	docs/reference/query-languages/esql/_snippets/lists/search-functions.md
#	docs/reference/query-languages/esql/functions-operators/search-functions.md
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/EsqlFunctionRegistry.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/FullTextFunction.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/Match.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/QueryString.java
#	x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/analysis/VerifierTests.java

* Take match_phrase out of snapshot and make tech preview (#128925)

* Take match_phrase out of snapshot and make tech preview

* Update docs/changelog/128925.yaml

* PR feedback

* Adding regenerated test data

* Update docs/changelog/128925.yaml

Co-authored-by: Carlos Delgado <6339205+carlosdelest@users.noreply.github.com>

* [CI] Auto commit changes from spotless

* Checkstyle

* Correct docs

* Hopefully fix docs build

* Found one more bad docs link - here's hoping this now fixes the doc build

* OMG bitten by - vs _

---------

Co-authored-by: Carlos Delgado <6339205+carlosdelest@users.noreply.github.com>
Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
Co-authored-by: Aurélien FOUCRET <aurelien.foucret@gmail.com>

* Remove null example for match_phrase (#129173)

* Fix errors from merge

* Fix more errors from merge

* [CI] Auto commit changes from spotless

* Fix compile errors introduced through cherry-pick

* Fix test compilation

* Generate docs

* Register match_phrase as a function not a snapshot function (#129255)

* Register match_phrase as a function not a snapshot function

* Update usage

* Fix usage

---------

Co-authored-by: Carlos Delgado <6339205+carlosdelest@users.noreply.github.com>
Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
Co-authored-by: Aurélien FOUCRET <aurelien.foucret@gmail.com>
Co-authored-by: Ioana Tagirta <ioanatia@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL auto-backport Automatically create backport pull requests when merged >enhancement :SearchOrg/Relevance Label for the Search (solution/org) Relevance team Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:Search - Relevance The Search organization Search Relevance team Team:SearchOrg Meta label for the Search Org (Enterprise Search) v8.19.0 v9.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants