[META] Enable Intra Segment Concurrent Search

[expani]

Please describe the end goal of this project

Lucene introduced Intra Segment Search in version 10 with apache/lucene#13542 giving users the ability to split a segment based on DocIds for concurrent processing in different threads.

This Meta will track all the tasks required for enabling the same in OpenSearch. End goal is to enable it for all queries and aggregations that support ConcurrentSegmentSearch without any reduction in performance.

Supporting References

#18338

Issues

#18849 End User configurations for IntraSegmentConcurrentSearch

#18851 Slicing mechanism changes for equal distribution

#18854 Making the custom collector and collector managers work

[ WIP] There are more tracks of work will update as I create sub-issues

Related component

Search