Changes related to Meilisearch v1.11.0

[curquiza]

This issue gathers the changes related to the v1.11.0 of Meilisearch that will impact the integrations scope.

📅 Release date: 28 October 2024

Timelines & steps

Pre-release

• With the help of the changelog & this CI, define which integrations should be updated and how (New feature? Update README? Update tests?)
  • Fill in the "What to implement?" section below in this issue 👇. Minial implementation: PHP, JS, Instant-meilisearch.
  • Open implementation issues in repositories that need implementation.
  • Discuss with the Product team if needed. At least share this issue to let them know about the decisions.
• Create a branch by running Octopus script: only open branches for the integrations we choose to update (defined in the previous step) + Kubernetes repository + Cloud provider repository (changing the version)
• Update integrations according to the decisions (cf "What to implement?" section below in this issue 👇)
  ⚠️ If possible, this step is done before pre-release, once the feature is ready thanks to the prototype released by the engine team
  • JS: implementation + fix tests
  • PHP: implementation + fix tests
  • Go: Fix AI tests (maybe implementation is required)
  • Python: fix AI tests (maybe implementation is required)
  • Ruby: fix AI tests
• Add code samples for the chosen up-to-date integrations with the new version of Meilisearch
• Update the library version of the related integrations and prepare the changelogs

Release day

• Release the integrations
  • Release JS
  • Release PHP
  • Go
  • Python
  • Ruby
  • Release Instant-meilisearch with new version of meilisearch-js
• Merge the related PR in K8s repository
• Publish DevOps tools:
  • create the git tag
  • publish images (steps are in CONTRIBUTING.md)
• Remove useless AWS images (by using our internal script)
• Update AI-search related issues with changes in v1.11
• Open issues in the repositories that are not up-to-date with the latest version of Meilisearch (including code samples)
• Update already existing issues in repositories about AI search

What to implement?

Experimental - AI search changes

Related issue in the engine:

• Tracking issue: Vector store stabilization meilisearch#4733
• Binary quantization meilisearch#4873

Breaking:

• When using the semantic or the hybrid search, hybrid.embedder is now a mandatory parameter in GET and POST /indexes/{:indexUid}/search
• As a consequence, it is now mandatory to pass hybrid even for full-vector search (with only vector and not q)
• embedder is now a mandatory parameter in GET and POST /indexes/{:indexUid}/similar
• Ignore non-zero semanticRatio when vector is passed but not q: a semantic search will be performed.

Changes:

• A new sub setting in embedders setting to enable binary quantization and speed up indexing speed.
• The default model for OpenAI is now text-embedding-3-small instead of text-embedding-ada-002.
• Limit the maximum length of a rendered document template: when the source of an embedder is set to huggingFace, openAi, rest or ollama, then documentTemplateMaxBytes is now available as an optional parameter. This parameter describes the number of bytes in which the rendered document template text should fit when trying to embed a document. Longer texts are truncated to fit.
• Add the ability to query whether a field is searchable in documentTemplate: use field.is_searchable: true if the field is a searchable attribute, otherwise false.

TODO:

• JS: implementation + fix tests - [v1.11.0] Apply changes (breaking or not) about AI-powered search meilisearch-js#1714
• PHP: implementation + fix tests - [v1.11.0] Apply changes (breaking or not) about AI-powered search meilisearch-php#682
• Go: fix tests - [v1.11.0] Fix AI-powered search feature following breaking changes meilisearch-go#578
• Python: fix tests - [v1.11.0] Fix AI-powered search feature following breaking changes meilisearch-python#1019
• Ruby: fix tests - no issue, done in Changes related to the next Meilisearch release (v1.11.0) meilisearch-ruby#571

Support facet distribution for federated search

Related issue in the engine: meilisearch/meilisearch#4932

• Request facet distributions and facet stats in your federated search, by using federation.facetsByIndex in the POST POST /multi-search route.
• Merge the returned facets in a single facet distribution and stats that is global to the entire request, by using federation.mergeFacets in the POST POST /multi-search route.

TODO:

• JS: [v1.11.0] Support facet distribution for federated search meilisearch-js#1713
• PHP: [v1.11.0] Support facet distribution for federated search meilisearch-php#681

Experimental - STARTS_WITH operator

Related issue in the engine: meilisearch/meilisearch#4872

Filter with the newly introduced STARTS_WITH operator

TODO:

• JS: nothing to implement 🎉
• PHP: nothing to implement 🎉

Language setting enhancement: support ISO-639-1 languages code

Related issue in the engine: meilisearch/meilisearch#4827

• Add iso-639-1 variants
• Convert iso-639-1 into iso-639-3

TODO:

• JS: nothing to implement 🎉
• PHP: nothing to implement 🎉