Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Feature] Enable sorting and search_after features in hybrid search (#…
…827) (#829) * Fix jdk version for CI test secure cluster action (#801) (#806) Signed-off-by: Martin Gaievski <gaievski@amazon.com> Co-authored-by: Martin Gaievski <gaievski@amazon.com> * [Part 1] Collector for Sorting Results (#797) * [Part 2] Normalization Phase for Sorting (#802) * Normalization Phase for Sorting Signed-off-by: Varun Jain <varunudr@amazon.com> * Fixing compile test issue Signed-off-by: Varun Jain <varunudr@amazon.com> * Optimize code Signed-off-by: Varun Jain <varunudr@amazon.com> * Add method description Signed-off-by: Varun Jain <varunudr@amazon.com> * [Part 1] Collector for Sorting Results (#797) * HybridSearchSortUtil class Signed-off-by: Varun Jain <varunudr@amazon.com> * Add Integ Tests Signed-off-by: Varun Jain <varunudr@amazon.com> * Add Sorting Integ tests Signed-off-by: Varun Jain <varunudr@amazon.com> * Add integ test for Sorting Signed-off-by: Varun Jain <varunudr@amazon.com> * Refactoring normalization processor workflow Signed-off-by: Varun Jain <varunudr@amazon.com> * Fix Unit Tests Signed-off-by: Varun Jain <varunudr@amazon.com> * Refactoring Signed-off-by: Varun Jain <varunudr@amazon.com> * Refactoring Signed-off-by: Varun Jain <varunudr@amazon.com> * Address Martin Comments Signed-off-by: Varun Jain <varunudr@amazon.com> * Optimising Normalization Signed-off-by: Varun Jain <varunudr@amazon.com> * Address Martin Comments Signed-off-by: Varun Jain <varunudr@amazon.com> * Address Martin Comments Signed-off-by: Varun Jain <varunudr@amazon.com> * Addressing Martin Comments Signed-off-by: Varun Jain <varunudr@amazon.com> * Addressing Vijay comments Signed-off-by: Varun Jain <varunudr@amazon.com> * Address Vijay Comments Signed-off-by: Varun Jain <varunudr@amazon.com> --------- Signed-off-by: Varun Jain <varunudr@amazon.com> * Update bwc workflow to include 2.16.0-SNAPSHOT (#809) (#810) * Increment BWC version * Append 2.16.0-SNAPSHOTn in restart upgrade tests --------- Signed-off-by: Varun Jain <varunudr@amazon.com> * [Part 3] Concurrent segment search bug in Sorting (#808) * Cherry picking Concurrent Segment Search Bug Commit Signed-off-by: Varun Jain <varunudr@amazon.com> * Fix Concurrent Segment Search Bug in Sorting Signed-off-by: Varun Jain <varunudr@amazon.com> * Functional Interface Signed-off-by: Varun Jain <varunudr@amazon.com> * Addressing Martin Comments Signed-off-by: Varun Jain <varunudr@amazon.com> * Removing comments Signed-off-by: Varun Jain <varunudr@amazon.com> * Addressing Martin Comments Signed-off-by: Varun Jain <varunudr@amazon.com> * Addressing Martin Comments Signed-off-by: Varun Jain <varunudr@amazon.com> * Addressing Martin commnents Signed-off-by: Varun Jain <varunudr@amazon.com> * Address Martin Comments Signed-off-by: Varun Jain <varunudr@amazon.com> * Address Martin Comments Signed-off-by: Varun Jain <varunudr@amazon.com> --------- Signed-off-by: Varun Jain <varunudr@amazon.com> Co-authored-by: Martin Gaievski <gaievski@amazon.com> * Rebasing with main (#826) * Adds method_parameters in neural search query to support ef_search (#787) (#814) Signed-off-by: Tejas Shah <shatejas@amazon.com> * Add BWC for batch ingestion (#769) * Add BWC for batch ingestion Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Update Changelog Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Fix spotlessLicenseCheck Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Fix comments Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Reuse the same code Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Rename some functions Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Rename a function Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Minor change to trigger rebuild Signed-off-by: Liyun Xiu <xiliyun@amazon.com> --------- Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Neural sparse query two-phase search processor's bwc test (#777) * Poc of pipeline Signed-off-by: conggguan <congguan@amazon.com> * Complete some settings for two phase pipeline. Signed-off-by: conggguan <congguan@amazon.com> * Change the implement of two-phase from QueryBuilderVistor to custom process funciton. Signed-off-by: conggguan <congguan@amazon.com> * Add It and fix some bug on the state of multy same neuralsparsequerybuilder. Signed-off-by: conggguan <congguan@amazon.com> * Simplify some logic, and correct some format. Signed-off-by: conggguan <congguan@amazon.com> * Optimize some format. Signed-off-by: conggguan <congguan@amazon.com> * Add some test case. Signed-off-by: conggguan <congguan@amazon.com> * Optimize some logic for zhichao-aws's comments. Signed-off-by: conggguan <congguan@amazon.com> * Optimize a line without application. Signed-off-by: conggguan <congguan@amazon.com> * Add some comments, remove some redundant lines, fix some format. Signed-off-by: conggguan <congguan@amazon.com> * Remove a redundant null check, fix a if format. Signed-off-by: conggguan <congguan@amazon.com> * Fix a typo for a comment, camelcase format for some variable. Signed-off-by: conggguan <congguan@amazon.com> * Add some comments to illustrate the influence of the modify on 2-phase search pipeline to neural sparse query builder. Signed-off-by: conggguan <congguan@amazon.com> * Add restart and rolling upgrade bwc test for neural sparse two phase processor. Signed-off-by: conggguan <congguan@amazon.com> * Spotless on qa. Signed-off-by: conggguan <congguan@amazon.com> * Update change log for two-phase BWC test. Signed-off-by: conggguan <congguan@amazon.com> * Remove redundant lines of two-phase BWC test. Signed-off-by: conggguan <congguan@amazon.com> * Add changelog. Signed-off-by: conggguan <congguan@amazon.com> * Add the PR link and number for the CHANGELOG.md. Signed-off-by: conggguan <congguan@amazon.com> * [Fix] NeuralSparseTwoPhaseProcessorIT created wrong ingest pipeline, fix it to correct API. Signed-off-by: conggguan <congguan@amazon.com> --------- Signed-off-by: conggguan <congguan@amazon.com> Signed-off-by: conggguan <157357330+conggguan@users.noreply.github.com> * Enable '.' for nested field in text embedding processor (#811) * Added nested structure for text embed processor mapping Signed-off-by: Martin Gaievski <gaievski@amazon.com> * Fix linux build CI error due to action runner env upgrade node 20 (#821) * Fix linux build CI error due to action runner env upgrade node 20 Signed-off-by: Varun Jain <varunudr@amazon.com> * Fix linux build on additional integ tests Signed-off-by: Varun Jain <varunudr@amazon.com> --------- Signed-off-by: Varun Jain <varunudr@amazon.com> --------- Signed-off-by: Tejas Shah <shatejas@amazon.com> Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: conggguan <congguan@amazon.com> Signed-off-by: conggguan <157357330+conggguan@users.noreply.github.com> Signed-off-by: Martin Gaievski <gaievski@amazon.com> Signed-off-by: Varun Jain <varunudr@amazon.com> Co-authored-by: Tejas Shah <shatejas@amazon.com> Co-authored-by: Liyun Xiu <chishui2@gmail.com> Co-authored-by: conggguan <157357330+conggguan@users.noreply.github.com> Co-authored-by: Martin Gaievski <gaievski@amazon.com> * Add changelog Signed-off-by: Varun Jain <varunudr@amazon.com> --------- Signed-off-by: Martin Gaievski <gaievski@amazon.com> Signed-off-by: Varun Jain <varunudr@amazon.com> Signed-off-by: Tejas Shah <shatejas@amazon.com> Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: conggguan <congguan@amazon.com> Signed-off-by: conggguan <157357330+conggguan@users.noreply.github.com> Co-authored-by: Martin Gaievski <gaievski@amazon.com> Co-authored-by: Tejas Shah <shatejas@amazon.com> Co-authored-by: Liyun Xiu <chishui2@gmail.com> Co-authored-by: conggguan <157357330+conggguan@users.noreply.github.com> (cherry picked from commit d22e1b8) Co-authored-by: Varun Jain <varunudr@amazon.com>
- Loading branch information