Should we add a new "BulkScoreVectors" type api to the vector scorer interfaces?

[benwtrent]

Description

With pre-fetching and newer quantization methods (e.g. the currently WIP bit quantization technique), it might be beneficial to have a clean interface indicating you want the scores for a handful of vectors.

I can see this helping with HNSW by allowing exploration to prefetch and then score all neighbors of the current candidate vector.

For higher levels of quantization quantization, since multiple corrective values need to be applied to the scored outputs, its conceivable that these corrections could be vectorized if multiple vectors were scored at the same time.

[benwtrent]

@mccullocht

Given the recent conversation on the Lucene list about making HNSW search faster.

[mccullocht]

Breadcrumb back to the mailing list discussion: https://lists.apache.org/thread/obc84kp3mxmd9nrbpxyj8bt0hbzfpxwv

There's some evidence to suggest that a bulk scoring API would allow us optimize scoring directly or introduce cpu cache prefetching if we call into native code. I'm planning to spend time on this.