[META]: SIMD adoption in OpenSearch

[heemin32]

Is your feature request related to a problem? Please describe.
Lucene 9.7.0 introduced new incubating vector APIs from Java 20 which utilize SIMD hardware in x86 AVX2 or later, and ARM NEON platforms. The feature is disabled by default. For a OpenSearch user to enable the feature, the user should pass a command line parameter during launch time. Also, OpenSearch should be running using jdk 20 or 21. To take advantage of SIMD optimization we will need OpenSearch to first run with JDK-21 by default and with SIMD modules enabled. This issue is to track items needed for SIMD enablement and potential area of improvements.

Additional context

• Switching default jdk version to jdk21 in OpenSearch (--add-modules jdk.incubator.vector is already present in config/jvm.options for main, needs backport to 2.x)
• [FEATURE] [k-NN] Lucene Engine with SIMD support k-NN#1062

K-NN performance comparison (https://github.com/opensearch-project/k-NN/tree/main/benchmarks/perf-tool)

• Cluster configuration: 3 leader nodes (c5.xlarge), 3 data nodes (r5.4xlarge)
• OpenSearch version: 2.9.0

1. Dataset: SIFT (128 dimensions, 1M docs)

	jdk17	jdk20-simd	diff
total ingest time (ms)	173861.47249	110505.82751	-36.4%
query latency p50 (ms)	8.9	8.2	-7.8%
query latency p90 (ms)	10.1	9	-10.9%
query latency p99 (ms)	11.1	10	-9.9%

2. Dataset: GIST (960 dimensions, 1M docs)

	jdk17	jdk20-simd	diff
total ingest time (ms)	1114000.82815	919003.77045	-17.5%
query latency p50 (ms)	32.5	18.2	-44.0%
query latency p90 (ms)	36.1	20.1	-44.3%
query latency p99 (ms)	38.8	24.5	-37.3%

• Other lucene optimizations that can be consumed with vector modules.
• Explore other code paths in OpenSearch to use the vector API

[reta]

@heemin32 I think we would be looking into switching main / 2.x to JDK-21 (due Sep 19th) since JDK-20 is not LTS (or whatever the supported long time release means)

[heemin32]

@vamshin JDK-21 GA date is 19th of Sep. OpenSearch code freeze date for 2.10.0 is 5th of Sep. That mean, we might not be able to enable SIMD for OpenSearch 2.10.0.

[reta]

That mean, we might not be able to enable SIMD for OpenSearch 2.10.0.

To note here, bundled JDK for 2.10.0 would still be JDK-17 but users could try to use JDK-20 instead at your own risks (although we have not run 2.x on JDK-20 yet it should work out of the box).

[vamshin]

@reta do you see issues if we bundle jdk-20 by default in 2.10 to take advantage of SIMD out of box for k-NN users? It would go through all the regular tests we do for the release

[reta]

@reta do you see issues if we bundle jdk-20 by default in 2.10 to take advantage of SIMD out of box for k-NN users? It would go through all the regular tests we do for the release

yes, there are at least 3 issues here:

• JDK-20 is not LTS and will officially go away in ~3 weeks
• 2.x have never been tested with JDK-20 (that should include all the plugins)
• JDK-20 sadly (at least the temurin builds) has release gaps (see please https://adoptium.net/temurin/releases/?version=20, specifically ppc64le)

Primarily I think the efforts should be spent on JDK-21 taking into account it is weeks away (not months or years)

[vamshin]

@reta thanks for the details. Looks like we will have to push to 2.11 then

[sohami]

@heemin32 I have repurposed this issue to discuss/explore the generic usage of SIMD in OpenSearch and added KNN related tasks as sub points in description. Let me know if you have any concerns or I am missing anything here

[heemin32]

Have done more testing regarding k-nn feature and result is available in opensearch-project/k-NN#1062

[ketanv3]

Another use-case in the date_histogram aggregation I've been exploring: #10392

[kkhatua]

@heemin32 would it be possible to run with JDK21 and update the numbers here, since JDK21 has LTS ? That'll help us prioritize this.

[heemin32]

@heemin32 would it be possible to run with JDK21 and update the numbers here, since JDK21 has LTS ? That'll help us prioritize this.

Have no bandwidth as of now but will try to get the result with JDK21.

[macohen]

note that the Vector API is still incubating in JDK 22: https://openjdk.org/jeps/460. Looks like potential minor revs in the API. Would we keep this off by default even when it is out of incubation or will it be safe to turn on once out of incubation? I think the Vector API itself determines what to do with or without SIMD present on the processor.

[reta]

Would we keep this off by default even when it is out of incubation or will it be safe to turn on once out of incubation?

Apache Lucene has Vector API support so having it on by default has benefits.

[macohen]

Would we keep this off by default even when it is out of incubation or will it be safe to turn on once out of incubation?

Apache Lucene has Vector API support so having it on by default has benefits.

100%; still wanted to call out the slight risk that the Vector API is incubating. I do know and get that Lucene is accepting that risk so we are, too...