Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Allow custom serialization and deserialization for custom TopDocs implementation #9697

Open
martin-gaievski opened this issue Sep 1, 2023 · 1 comment
Labels
enhancement Enhancement or improvement to existing feature or request Search Search query, autocomplete ...etc

Comments

@martin-gaievski
Copy link
Member

Problem definition
It's not possible to use custom TopDocs implementation for scenarios when there and multiple nodes in cluster and query results need to be serialized for sending from data node and deserialized on coordinator node. Current logic supports three types of TopDocs with certain limits on information in those objects.

Code refs for serialization logic and deserialization logic.

It may be required for scenarios when standard TopDocs format needs to be extended. In general any type of info retrieval where processing is required at coordinator level. For example, in case of Hybrid Query we need to send collection of results from multiple queries to coordinator node, where scores are processed by search pipeline processor.

Describe the solution you'd like
A way for developer to provide serialization and serialization logic of their implementation of TopDocs. Existing logic with case/switch on predefined types can be a fallback mechanism in case object doesn't have any specific implementation.

Describe alternatives you've considered
Make new TopDocs implementation part of core (similar to CollapseTopFieldDocs) along with a new case in switch clause specifically for that new type.
That is not very flexible and also custom TopDocs implementation may be not very generic to make it part of core OpenSearch.

@martin-gaievski martin-gaievski added enhancement Enhancement or improvement to existing feature or request untriaged labels Sep 1, 2023
@tlfeng tlfeng added the Search Search query, autocomplete ...etc label Sep 5, 2023
@msfroh msfroh removed the untriaged label Sep 6, 2023
@msfroh
Copy link
Collaborator

msfroh commented Sep 6, 2023

@martin-gaievski -- should we assign this one to you? Would you like some help on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Search Search query, autocomplete ...etc
Projects
Status: Later (6 months plus)
Development

No branches or pull requests

3 participants