Rehome GPU accelerated vector indexing under vector search

deploy-manage/production-guidance/optimize-performance/approximate-knn-search.md

@@ -210,7 +210,7 @@ A common pattern is to first perform an initial bulk upload, then make an index
 ```{applies_to}
 stack: ga 9.4
 ```
-For indexing-heavy workloads on large vector datasets, GPU acceleration can significantly speed up HNSW index construction and reduce the cost of merging segments into larger ones. See [GPU accelerated vector indexing](elasticsearch://reference/elasticsearch/mapping-reference/gpu-vector-indexing.md) for supported configurations and setup.
+For indexing-heavy workloads on large vector datasets, GPU acceleration can significantly speed up HNSW index construction and reduce the cost of merging segments into larger ones. See [GPU accelerated vector indexing](../../../solutions/search/vector/gpu-vector-indexing.md) for supported configurations and setup.
 
 
 ## Avoid heavy indexing during searches [_avoid_heavy_indexing_during_searches]

solutions/search/vector.md

@@ -197,6 +197,7 @@ These guides provide more direct or customizable approaches to working with vect
 
 - [kNN search in Elasticsearch](vector/knn.md): Perform vector similarity search using the `dense_vector` field type and k-nearest neighbor queries.
 - [Bring your own dense vectors](vector/bring-own-vectors.md): Use this if you already have embeddings and want to index and search them in Elasticsearch.
+- {applies_to}`stack: preview 9.3, ga 9.4+` {applies_to}`serverless: unavailable` [GPU accelerated vector indexing](vector/gpu-vector-indexing.md): Speed up HNSW index construction on nodes with compatible NVIDIA GPUs.
 - [Sparse vector search in Elasticsearch](vector/sparse-vector.md): Perform semantic search using sparse vectors with the ELSER model and the `sparse_vector` field type.
 
 

solutions/search/vector/_snippets/docker-gpu-indexing.md

@@ -0,0 +1,88 @@
+```plaintext
+FROM docker.elastic.co/elasticsearch/elasticsearch:9.3.0
+
+USER root
+
+# See https://gitlab.com/nvidia/container-images/cuda/-/blob/master/dist/12.9.1/ubi9/base/Dockerfile?ref_type=heads
+# and https://gitlab.com/nvidia/container-images/cuda/-/blob/master/dist/12.9.1/ubi9/devel/Dockerfile?ref_type=heads
+# We are installing nvidia/cuda drivers/libraries the same way that nvidia does in their images
+
+ENV CUVS_VERSION=25.12.0
+
+ENV NVARCH=x86_64
+ENV NVIDIA_REQUIRE_CUDA="cuda>=12.9 brand=unknown,driver>=535,driver<536 brand=grid,driver>=535,driver<536 brand=tesla,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=vapps,driver>=535,driver<536 brand=vpc,driver>=535,driver<536 brand=vcs,driver>=535,driver<536 brand=vws,driver>=535,driver<536 brand=cloudgaming,driver>=535,driver<536 brand=unknown,driver>=550,driver<551 brand=grid,driver>=550,driver<551 brand=tesla,driver>=550,driver<551 brand=nvidia,driver>=550,driver<551 brand=quadro,driver>=550,driver<551 brand=quadrortx,driver>=550,driver<551 brand=nvidiartx,driver>=550,driver<551 brand=vapps,driver>=550,driver<551 brand=vpc,driver>=550,driver<551 brand=vcs,driver>=550,driver<551 brand=vws,driver>=550,driver<551 brand=cloudgaming,driver>=550,driver<551 brand=unknown,driver>=560,driver<561 brand=grid,driver>=560,driver<561 brand=tesla,driver>=560,driver<561 brand=nvidia,driver>=560,driver<561 brand=quadro,driver>=560,driver<561 brand=quadrortx,driver>=560,driver<561 brand=nvidiartx,driver>=560,driver<561 brand=vapps,driver>=560,driver<561 brand=vpc,driver>=560,driver<561 brand=vcs,driver>=560,driver<561 brand=vws,driver>=560,driver<561 brand=cloudgaming,driver>=560,driver<561 brand=unknown,driver>=565,driver<566 brand=grid,driver>=565,driver<566 brand=tesla,driver>=565,driver<566 brand=nvidia,driver>=565,driver<566 brand=quadro,driver>=565,driver<566 brand=quadrortx,driver>=565,driver<566 brand=nvidiartx,driver>=565,driver<566 brand=vapps,driver>=565,driver<566 brand=vpc,driver>=565,driver<566 brand=vcs,driver>=565,driver<566 brand=vws,driver>=565,driver<566 brand=cloudgaming,driver>=565,driver<566 brand=unknown,driver>=570,driver<571 brand=grid,driver>=570,driver<571 brand=tesla,driver>=570,driver<571 brand=nvidia,driver>=570,driver<571 brand=quadro,driver>=570,driver<571 brand=quadrortx,driver>=570,driver<571 brand=nvidiartx,driver>=570,driver<571 brand=vapps,driver>=570,driver<571 brand=vpc,driver>=570,driver<571 brand=vcs,driver>=570,driver<571 brand=vws,driver>=570,driver<571 brand=cloudgaming,driver>=570,driver<571"
+ENV NV_CUDA_CUDART_VERSION=12.9.79-1
+ENV CUDA_VERSION=12.9.1
+
+ENV NV_CUDA_LIB_VERSION=12.9.1-1
+ENV NV_NVPROF_VERSION=12.9.79-1
+ENV NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-9-${NV_NVPROF_VERSION}
+ENV NV_CUDA_CUDART_DEV_VERSION=12.9.79-1
+ENV NV_NVML_DEV_VERSION=12.9.79-1
+ENV NV_LIBCUBLAS_DEV_VERSION=12.9.1.4-1
+ENV NV_LIBNPP_DEV_VERSION=12.4.1.87-1
+ENV NV_LIBNPP_DEV_PACKAGE=libnpp-devel-12-9-${NV_LIBNPP_DEV_VERSION}
+ENV NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-devel
+ENV NV_LIBNCCL_DEV_PACKAGE_VERSION=2.27.3-1
+ENV NCCL_VERSION=2.27.3
+ENV NV_LIBNCCL_DEV_PACKAGE=${NV_LIBNCCL_DEV_PACKAGE_NAME}-${NV_LIBNCCL_DEV_PACKAGE_VERSION}+cuda12.9
+ENV NV_CUDA_NSIGHT_COMPUTE_VERSION=12.9.1-1
+ENV NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-9-${NV_CUDA_NSIGHT_COMPUTE_VERSION}
+
+ENV NV_NVTX_VERSION=12.9.79-1
+ENV NV_LIBNPP_VERSION=12.4.1.87-1
+ENV NV_LIBNPP_PACKAGE=libnpp-12-9-${NV_LIBNPP_VERSION}
+ENV NV_LIBCUBLAS_VERSION=12.9.1.4-1
+ENV NV_LIBNCCL_PACKAGE_NAME=libnccl
+ENV NV_LIBNCCL_PACKAGE_VERSION=2.27.3-1
+ENV NV_LIBNCCL_VERSION=2.27.3
+ENV NCCL_VERSION=2.27.3
+ENV NV_LIBNCCL_PACKAGE=${NV_LIBNCCL_PACKAGE_NAME}-${NV_LIBNCCL_PACKAGE_VERSION}+cuda12.9
+
+ENV NVIDIA_VISIBLE_DEVICES=all
+ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
+ENV RAFT_DEBUG_LOG_FILE=/dev/null
+
+# Install nvidia drivers
+RUN microdnf install -y dnf
+RUN dnf install -y 'dnf-command(config-manager)'
+RUN dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo
+
+RUN dnf upgrade -y && dnf install -y \
+    cuda-cudart-12-9-${NV_CUDA_CUDART_VERSION} \
+    cuda-compat-12-9 \
+    && dnf clean all \
+    && rm -rf /var/cache/yum/*
+
+# Set up env vars for various libraries (cuda, libcuvs)
+RUN echo "/usr/local/cuda/lib64" >> /etc/ld.so.conf.d/nvidia.conf
+ENV PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:${PATH}
+ENV LIBCUVS_DIR="/opt/cuvs"
+ENV LD_LIBRARY_PATH=${LIBCUVS_DIR}:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda/lib64
+
+# Install other required nvidia and cuda libraries, as well as tar and gzip
+RUN dnf install -y \
+    cuda-libraries-12-9-${NV_CUDA_LIB_VERSION} \
+    cuda-nvtx-12-9-${NV_NVTX_VERSION} \
+    ${NV_LIBNPP_PACKAGE} \
+    libcublas-12-9-${NV_LIBCUBLAS_VERSION} \
+    ${NV_LIBNCCL_PACKAGE} \
+    tar gzip \
+    && dnf clean all \
+    && rm -rf /var/cache/yum/*
+
+# Grab the libcuvs library from Elastic's gcs archive
+# These are tarballs that contain only the libraries necessary from nvidia's libcuvs builds in conda
+# Note: this is temporary until nvidia begins publishing minimal libcuvs tarballs along with their releases
+RUN mkdir -p "$LIBCUVS_DIR" && \
+    chmod 775 "$LIBCUVS_DIR" && \
+    cd "$LIBCUVS_DIR" && \
+    CUVS_ARCHIVE="libcuvs-$CUVS_VERSION.tar.gz" && \
+    curl -fO "https://storage.googleapis.com/elasticsearch-cuvs-snapshots/libcuvs/$CUVS_ARCHIVE" && \
+    tar -xzf "$CUVS_ARCHIVE" && \
+    rm -f "$CUVS_ARCHIVE" && \
+    if [[ -d "$CUVS_VERSION" ]]; then mv "$CUVS_VERSION/*" ./; fi
+
+# Reset the user back to elasticsearch
+USER 1000:0
+```

solutions/search/vector/dense-vector.md

@@ -34,6 +34,7 @@ To implement dense vector search in {{es}}, you need both an index configuration
      - Refer to the [overview of NLP model options](../semantic-search.md#using-nlp-models)
    - Or [bring your own embeddings](bring-own-vectors.md)
      - Store them using the `dense_vector` field type
+   - {applies_to}`stack: preview 9.3, ga 9.4+` {applies_to}`serverless: unavailable` Optionally [accelerate HNSW indexing with a GPU](gpu-vector-indexing.md)
 
 2. **Query the index with k-NN search**
    - Use the [`knn` query](knn.md) to retrieve results based on vector similarity

solutions/search/vector/gpu-vector-indexing.md

@@ -0,0 +1,181 @@
+---
+navigation_title: GPU vector indexing
+description: Accelerate HNSW index construction for dense vectors with GPU acceleration in Elasticsearch, including hardware setup, Docker configuration, monitoring, and troubleshooting.
+applies_to:
+  stack: preview 9.3, ga 9.4+
+  serverless: unavailable
+products:
+  - id: elasticsearch
+  - id: elastic-stack
+type: how-to
+---
+
+# Accelerate dense vector indexing with a GPU in {{es}} [gpu-vector-indexing]
+
+GPU indexing in {{es}} builds Hierarchical Navigable Small World (HNSW)
+graphs with the [NVIDIA cuVS library](https://developer.nvidia.com/cuvs).
+Offloading that work to a graphics processing unit (GPU) speeds up dense
+vector ingest on large datasets and frees CPU resources for search and
+other tasks.
+
+## Before you begin [gpu-vector-indexing-before-you-begin]
+
+To use GPU indexing, you need:
+
+* An [Enterprise subscription](https://www.elastic.co/subscriptions)
+* A supported NVIDIA GPU (Ampere architecture or better, compute capability
+  \>= 8.0) with a minimum 8 GB of GPU memory
+* GPU driver, CUDA, and
+  [cuVS runtime libraries](https://docs.rapids.ai/api/cuvs/stable/build/)
+  installed on the node. Refer to the
+  [Elastic support matrix](https://www.elastic.co/support/matrix) for
+  supported CUDA and cuVS versions.
+* `LD_LIBRARY_PATH` environment variable configured to include the cuVS
+  libraries path and its dependencies (CUDA, rmm, and so on)
+* Supported platform: Linux x86_64 only, Java 22 or later
+* Supported dense vector configurations: `hnsw` and `int8_hnsw`; `float`
+  element type only
+
+## Configuration
+
+The
+[`vectors.indexing.use_gpu`](elasticsearch://reference/elasticsearch/configuration-reference/node-settings.md#gpu-vector-indexing-settings)
+node-level setting controls GPU vector indexing.
+
+## Elasticsearch Docker image with GPU support
+
+You can extend the official {{es}} Docker image with this example Dockerfile
+to add the dependencies required for GPU support.
+
+::::{warning}
+This Dockerfile serves as an example implementation, and is not fully supported
+like our official Docker images.
+::::
+
+::::{dropdown} Example Dockerfile
+:::{include} _snippets/docker-gpu-indexing.md
+:::
+::::
+
+### Host requirements
+
+The host machine running the Docker container needs
+[NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
+installed and configured.
+
+### Build it
+
+```sh
+docker build -t es-gpu .
+```
+
+### Run it
+
+```sh
+docker run \
+  -p 9200:9200 \
+  -p 9300:9300 \
+  -e "discovery.type=single-node" \
+  -e "xpack.security.enabled=false" \
+  -e "xpack.license.self_generated.type=trial" \
+  -e "vectors.indexing.use_gpu=true" \
+  --user elasticsearch \
+  --gpus all \
+  --rm -it es-gpu
+```
+
+## Monitoring
+
+```{applies_to}
+stack: ga 9.3.2
+```
+
+Use the `GET _xpack/usage` API to monitor GPU vector indexing status and usage
+across all nodes in the cluster:
+
+```console
+GET _xpack/usage?filter_path=gpu_vector_indexing
+```
+
+```console-result
+{
+  "gpu_vector_indexing": {
+    "available": true, <1>
+    "enabled": true, <2>
+    "index_build_count": 30, <3>
+    "nodes_with_gpu": 3, <4>
+    "nodes": [ <5>
+      { "type": "NVIDIA L4", "memory_in_bytes": 24000000000,
+        "enabled": true, "index_build_count": 10 },
+      { "type": "NVIDIA L4", "memory_in_bytes": 24000000000,
+        "enabled": true, "index_build_count": 10 },
+      { "type": "NVIDIA A100", "memory_in_bytes": 80000000000,
+        "enabled": true, "index_build_count": 10 }
+    ]
+  }
+}
+```
+1. Whether the current license permits GPU indexing.
+2. Whether at least one node has GPU hardware configured and has not turned it off through `vectors.indexing.use_gpu=false`.
+3. Total number of GPU index builds across the cluster.
+4. Number of data nodes with GPU support.
+5. Per-node GPU details including type, memory, enabled status, and build count.
+
+## Troubleshooting
+
+By default, {{es}} uses GPU indexing for supported vector types if a
+compatible GPU and required libraries are detected.
+Check server logs for messages indicating whether {{es}} has detected a GPU.
+
+If the following message appears, a GPU was successfully detected and
+GPU indexing is used:
+```
+[o.e.x.g.GPUSupport ] [elasticsearch-0] Found compatible GPU [NVIDIA L4] (id: [0])
+```
+If this message doesn't appear, check for warning messages explaining why GPU
+indexing isn't being used, such as an unsupported environment, missing
+libraries, or an incompatible GPU.
+
+
+### Node fails to start with `vectors.indexing.use_gpu: true`
+
+To enforce GPU indexing, set `vectors.indexing.use_gpu: true` in
+`elasticsearch.yml`.
+The node fails to start if GPU indexing isn't available. For example, if
+a GPU isn't detected by {{es}}, if the runtime isn't supported, or if the
+necessary dependencies aren't correctly configured.
+
+If the node fails to start, check:
+* A supported NVIDIA GPU is present
+* CUDA runtime libraries and drivers are installed (check with `nvidia-smi`)
+* `LD_LIBRARY_PATH` includes paths to the cuVS libraries and to their
+  dependencies (for example, CUDA)
+* Supported platform: Linux x86_64 with Java 22 or later
+
+
+### Performance not improved with GPU indexing
+
+If you're sure that GPU indexing is enabled but performance doesn't improve,
+check the following:
+
+* Use supported vector index types and the `float` element type.
+* Use a dataset large enough to benefit from GPU acceleration.
+* Check for other bottlenecks. GPU indexing accelerates HNSW graph building, but other factors can limit speedups.
+  * Indexing throughput depends on how fast you can get data into {{es}}. Check network speed and client performance. Use multiple clients if needed.
+  * JSON parsing can dominate the computation. Use base64 encoded vectors instead of JSON arrays.
+  * Storage speed also matters. The GPU can process lots of data, so use storage that can keep up. Avoid network-attached storage, and prefer fast NVMe.
+* Monitor CPU usage to confirm work is offloaded to the GPU.
+* Monitor GPU usage (for example, with `nvidia-smi`).
+
+## Next steps
+
+- [Index dense vectors and run kNN search](knn.md)
+- [Tune approximate kNN search](/deploy-manage/production-guidance/optimize-performance/approximate-knn-search.md) for indexing and query performance
+- [Bring your own dense vectors](bring-own-vectors.md) if you already have embeddings
+
+## Related pages
+
+- [GPU vector indexing settings](elasticsearch://reference/elasticsearch/configuration-reference/node-settings.md#gpu-vector-indexing-settings)
+- [`dense_vector` field type](elasticsearch://reference/elasticsearch/mapping-reference/dense-vector.md)
+- [Dense vector search](dense-vector.md)
+- [Vector search in Elasticsearch](../vector.md)

solutions/toc.yml

@@ -1,3 +1,3 @@
 project: "Solutions and use cases"
 toc:
   - file: index.md
@@ -46,6 +46,7 @@
                   - file: search/vector/knn.md
                   - file: search/vector/bring-own-vectors.md
                   - file: search/vector/vector-storage-for-semantic-search.md
+                  - file: search/vector/gpu-vector-indexing.md
               - file: search/vector/sparse-vector.md
           - file: search/hybrid-search.md
             children: