Use lazy initialization for priority queue of hits and scores to improve latencies by 20%

[martin-gaievski]

Description

In this change we're improving hybrid query latencies by changing allocation of query hits objects for hybrid scores. Currently those objects are pre-populated, then for new element top of the priority queue is updated and heap is rebalanced. We're changing this to lazy initialization - priority queue is initially empty, for each hit we're inserting element. If we reach the capacity then element with low score is pushed out of the heap. As per benchmark results this alone gives 10% improvement.

With several other minor changes like pre calculate and store size instead of calling size() for each doc and each sub-query we can get 10% percent.

Below are metrics for hybrid query latency that I've collected after this change. There are based on 2.x (2.14) version and noaa OSB workload, all times are in ms:

One sub-query that selects 11M documents

Bool: p50 76.5531 | p90 77.8108
Hybrid: p50 133.644 | p90 151.903

One sub-query that selects 1.6K documents

Bool: p50 69.9585 | p90 70.2855
Hybrid: p50 70.1703 | p90 70.5498

Three sub-query that select 15M documents

Bool: p50 87.1001 | p90 87.7131
Hybrid: p50 234.802 | p90 262.876

following are the baseline results

One sub-query that selects 11M documents

Bool: p50 77.8893 | p90 78.1916
Hybrid: p50 186.709 | p90 197.739

One sub-query that selects 1.6K documents

Bool: p50 71.0947 | p90 71.691
Hybrid: p50 71.5156 | p90 72.8801

Three sub-query that select 15M documents

Bool: p50 87.0556 | p90 90.9105
Hybrid: p50 287.255 | p90 313.868

Following framegraph has been captured after this change:

Issues Resolved

#745
#704

Check List

• All tests pass
• Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.86%. Comparing base (7c54c86) to head (4d0c69d).
Report is 2 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #746      +/-   ##
============================================
- Coverage     85.02%   84.86%   -0.16%     
- Complexity      790      792       +2     
============================================
  Files            60       60              
  Lines          2430     2425       -5     
  Branches        410      409       -1     
============================================
- Hits           2066     2058       -8     
- Misses          202      207       +5     
+ Partials        162      160       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.