Added optimization for OpenSearch engine which were missing

[navneet1v]

Description

This PR added the optimization for Opensearch engine in this benchmarking tool. Along with the optimizations I have also upgraded the OS docker image to latest version(at the time of raising this PR) which is 2.14

Testing

I performed a local testing on a small dataset and a small configuration and I can see improvements in Opensearch results.

Sample run results

Cluster: single node with CI based docker setup

python3 -m run --engines opensearch-default --datasets glove-25-angular

Indexing

{
  "params": {
    "experiment": "opensearch-default",
    "engine": "opensearch",
    "dataset": "glove-25-angular",
    "parallel": 16,
    "method": {
      "parameters": {
        "m": 16,
        "ef_construction": 100
      }
    }
  },
  "results": {
    "post_upload": {},
    "upload_time": 52.828218106064014,
    "total_time": 143.01298881007824
  }
}

Search

{
  "params": {
    "dataset": "glove-25-angular",
    "experiment": "opensearch-default",
    "engine": "opensearch",
    "parallel": 100,
    "config": {
      "knn.algo_param.ef_search": 128
    }
  },
  "results": {
    "total_time": 17.1585195150692,
    "mean_time": 0.013182678549340927,
    "mean_precisions": 0.9283049999999999,
    "std_time": 0.016356453433367828,
    "min_time": 0.0020476150093600154,
    "max_time": 0.7723487680777907,
    "rps": 582.8008640965589,
    "p95_time": 0.030238651664694766,
    "p99_time": 0.07367674464592752
  }
}

Please let me know if I need add this in upstream qadrant repo.

[navneet1v]

In the next PR I will add the support of faiss engine using cosinesiml distance function. I didn't add that functionality because faiss is only getting used when filters are present.

[navneet1v]

@filipecosta90

[fcostaoliveira]

@navneet1v we've missed this one. will review until end of next week. thank you for raising this!

[fcostaoliveira]

Thanks for these OpenSearch optimizations, @navneet1v — closing as superseded by the Rust rewrite (master). The substantive tuning is already in src/bin/vector_db_benchmark/engine/opensearch.rs: refresh_interval: -1 during bulk indexing, force-merge to a single segment after upload, and kNN graph warmup (/_plugins/_knn/warmup). The Rust engine additionally uses the Lucene kNN engine for efficient filtered graph traversal. The only bits not carried over are minor knobs (index_thread_qty, shard/replica count) — happy to track those as a small enhancement if useful.