[RFC] OpenSearch Relevancy Workbench

[ps48]

OpenSearch Relevancy Workbench

1. Summary

Relevancy Workbench is an OpenSearch Dashboards plugin that comprises of different components: Query Explorer, Rules Manager and Search Analytics. These components allow search relevancy engineers to make their query results more accurate, contextual and relevant.

1.1 Target User:

Search Relevance Engineers who want to create a good search experience for their end-users, by providing them with relevant search results. Also, they want to dive deeper into search analytics like top search queries, top clicks and top queries without result.

1.2 Glossary:

• Search App: An OpenSearch Dashboards UI that allows user to put in a search query, add filters and see results.
• Querqy rewriter types (Details here):
  • Common rules
  • Replace
  • Word-break
  • Number-unit
• PPL Query: OpenSearch Piped Processing Language Details here.

2. Query Explorer

OpenSearch search relevance engineers create and test different DSL queries and Querqy queries for their search app. Query Explorer enables these developers to test these queries and rules.

2.1 Functional requirements

1. Users should be able to see all the Search Apps created, filter them by name and sort them by date modified/date created.
2. Users should be able to create/modify/clone/delete Search Apps.
3. Users should be able to add/edit tags and description for their Search Apps.
4. Users should be able to configure index name used for their Search Apps.
5. Users should be able to configure DSL/Querqy queries used for their Search Apps.
6. Users should be able to preview search results during configuration.
7. In Edit mode, Users should be able to add filters (similar to e-commerce sites) in their Search Apps.
8. In Edit mode, Users should be able to move around filters, search bars and results table.
9. In View mode, Users should be able to type a search query in the search bar and see the search results with all selected filters applied.
10. In View mode, Users should be able to click on each result row to see a detailed view of the selected result.
11. Users should be have the option to allow or not allow the capturing clicks, search queries, results, filters for search analytics.
12. Users should be able to visually compare two result sets.
13. Users should be able to analyze difference in two result sets (permutation metrics).
14. Users should be able to save search test sets.
15. Users should be able to view which Querqy rules were applied along side the query explain functionality.

2.2 Non-Functional requirements

1. The Search App should adhere to all security checks based on tenants, user roles, and permissions.
2. The search queries should be made based on the user roles and permissions.
3. The search queries should have proper circuit breakers, for all search requests made.

3. Rules Manager

Rules Manager allows search relevance engineers to manage querqy rules.

3.1 Functional requirements

1. Users should be able to view all Rules created, filter them by name and sort them by date modified/date created and by rewriter type.
2. Users should be able to create rules for all types of Querqy Rewriters.
3. Users should be restricted from creating two or more rules with the same name.
4. Users should be able to modify/clone/delete rules.
5. Users should be able to view rule contents.
6. Users should be able to add and edit rules using assisted rule table UI.
7. Users should be able to add and edit the rule in a code editor UI.
8. Users should be able to save the rules in the Querqy OpenSearch Index.
9. Users should be able to reload, rename, clone, delete rule while viewing it.
10. Users should be able to view help text/docs on the Querqy Rewriters type while creating/editing a rule.

3.2 Non-Functional requirements

1. The Search App should adhere to all security checks based on user roles, and permissions.

4. Search Analytics

Search Analytics allows search relevance engineers to view summaries of interactions (clicks, search queries, etc) on different search apps.

4.1 Functional requirements

1. Users should be able to view aggregated analytics with data from all Search Apps.
2. Users should be able to filter the analytics view by search app name, tags using PPL Query.
3. Users should be able to filter the analytics view by time interval.
4. Users should be able to see total number of queries, total number of queries with no results, total result-clicks in the analytics, stats on ranks of clicks.
5. Users should be able to see graphs for total queries, queries without results per hour/day/month (based on time interval selected by user).
6. Users should be able to see tables of top queries, top queries with no results.

4.2 Non-Functional requirements

1. The Search Analytics should adhere to all security checks based on tenants, user roles, and permissions.
2. The search queries should be made based on the user roles and permissions.
3. The search queries should have proper circuit breakers, for all search requests made.

5. Future Work

1. Add a query evaluation tool for offline testing with golden queries.
2. Allow users to make a pipeline based configuration for request/response intercepts in Search Apps.
3. Support for 2+ pipelines in a single UI for request/response configs with interleaving results.

6. Appendix

6.1 Related Issues:

1. [RFC] OpenSearch Search Relevance search-processor#1
2. [RFC] OpenSearch Querqy Plugin Design Document search-processor#2

6.2 References:

1. Querqy Docs: https://docs.querqy.org/querqy/index.html
2. SMUI (Querqy Search Management UI): https://docs.querqy.org/smui/index.html
3. Chorus tools playlist: https://www.youtube.com/watch?v=aoWx7KJzvCs&list=PLCoJWKqBHERvkhDoH1JBWfUxjj0H_It5D
4. SMUI usage video:https://www.youtube.com/watch?v=iUFzS7k2OHQ
5. Quepid: https://quepid.com/