msearch deprecation plans

[lizozom]

We are currently working on moving search related code to NP (data\search). This requires transitioning default_search_strategy from using the old es service, to the new search_service @lukasolson and @stacey-gammon have been working on for the past few months.

This however, creates an issue, considering the fact that search_service doesn't have msearch implemented. It's worth mentioning that since 7.5 msearch batching of dashboard requests is disabled by default.

Possible solutions at the moment seem to be:

1. Keep msearch around. Implement msearch and use it to transition to NP. If we want to deprecate it before releasing v8, that's cool. Downsides: requires implementing code we don't really need. Won't allow us deprecating a lot of code ATM.
2. Deprecate msearch on master. Keep it only for v7.x. This would allow us deprecating a lot of code on master while not doing any breaking changes on v7.x. Downdsides: Still requires re-implementing msearch. Will make backports more difficult.
3. Deprecate msearch in 7.6 (or 7.7). This would be the easiest solution technically speaking, allowing us to clean up a bunch of old code and not implement new unneeded functionallity. Downsides: deprecating courier:batchSearches not on a major version.

I would appreciate your quick input, as this issue is ATM meant to be released in 7.6.

@AlonaNadler I think your input is critical here.

[elasticmachine]

Pinging @elastic/kibana-app-arch (Team:AppArch)

[lukasolson]

One of the benefits we have discussed around _msearch is that it bundles up requests so that you don't hit the browser connection limit. As far as I'm aware, this is the only real benefit (other than ensuring that all of your search requests hit the same shards).

Our long term solution is to enable batching in a way similar to how Canvas batches HTTP requests. @streamich has a PR up for this utility (#53711) although it would take some work to integrate it into our new search service (#54207).

I would prefer that we avoid writing new code that actually invokes the _msearch Elasticsearch endpoint. If it means that we need to delay the above linked PR (#53766) then I think it's reasonable.

I would also like to get input from @alexh97 and @streamich.

[AlonaNadler]

Thanks for raising @lizozom
My first impulse is not to have breaking changes in minors,
Few questions:

• @lukasolson where are possible places msearch is still used?
• Which of the solution teams is using msearch?
• For plugin developers who use msearch what should they do after we deprecate it?
• @lizozom how much work will be involved in keeping msearch around until v8?

[lizozom]

@AlonaNadler I think re-implementing and stabilizing a new implementation could take between a week or two. That's a rough estimate.

[lizozom]

Has anyone bench marked various dashboard scenarios with \ without msearch?
@stacey-gammon @lukasolson @AlonaNadler

[lukasolson]

@lukasolson where are possible places msearch is still used?
Which of the solution teams is using msearch?

We're only talking about removing _msearch from Courier/our new search service, which is used by visualize & saved search. I don't believe anyone else or any solutions are using our services at the moment.

It's important to distinguish between removing the use of _msearch in our services vs. deprecating the usage of _msearch everywhere . . . other teams will still be able to manually call _msearch (bypassing our services) if it is desirable.

Has anyone bench marked various dashboard scenarios with \ without msearch?

Not that I'm aware of.

[streamich]

I guess we agreed on how to proceed during our meeting today. I just wanted to note here the few differences between msearch and bfetch, FYI.

• msearch connects directly to Elasticsearch, bfetch talks to Kibana server.
• msearch sends a batch and receives a batch back, bfetch sends a batch but streams the results back as soon as they become available.
• My understanding is that msearch creates batches by buffering for 50ms, bfetch created batches either by buffering for 10ms—by default—or until batch size limit is reached, by default 25 items.
• Both batching solutions don't support cancelling individual requests.