Add Monitoring section to Kubernetes deployment for Grafana dashboard

spiceaidocs/docs/clients/grafana/index.md

@@ -6,7 +6,39 @@ pagination_prev: 'clients/index'
 pagination_next: null
 ---
 
-The Spice runtime has a Prometheus endpoint configured by default on port `9091`. There's also prebuilt Grafana dashboards for out of the box monitoring of all your Spice instances. This tutorial creates and configures Grafana and Prometheus to scrape and display metrics from several Spice instances. It assumes:
+The Spice runtime has a Prometheus endpoint configured by default on port `9000`. There's also prebuilt Grafana dashboards for out of the box monitoring of all your Spice instances.
+
+## Import Grafana Dashboard
+
+Navigate to the Dashboards section in Grafana and click "New" > "Import".
+
+<img width="500" src="/img/grafana/import-dashboard-button.png" />
+
+Copy the dashboard JSON from [monitoring/grafana-dashboard.json](https://github.com/spiceai/spiceai/blob/trunk/monitoring/grafana-dashboard.json) into the Grafana import box.
+
+<img width="500" src="/img/grafana/import-dashboard.png" />
+
+Click "Load".
+
+## Kubernetes
+
+View the [Kubernetes](/deployment/kubernetes) deployment guide for configuring the Prometheus Operator to scrape metrics from the Spice instances in Kubernetes.
+
+## Prometheus
+
+Configure a Prometheus instance to scrape metrics from the Spice runtimes.
+
+```yaml
+global:
+    scrape_interval: 1s
+    scrape_configs:
+    - job_name: spiceai
+        static_configs:
+        - targets: ['127.0.0.1:9000'] # Change to your Spice runtime endpoint + port
+```
+
+## Local Quickstart
+This tutorial creates and configures Grafana and Prometheus locally to scrape and display metrics from several Spice instances. It assumes:
     - Two Spice runtimes, `spiced-main` and `spiced-edge`, are running on `127.0.0.1:9091` and `127.0.0.1:9092` respectively.
 
 

spiceaidocs/docs/deployment/kubernetes/index.md

@@ -48,6 +48,7 @@ spicepod:
 | ------------------- | ---------------------------------------------------------------------------------------------------- | ---------------------- |
 | `image.repository`    | The repository of the Docker image .                                                                           | `spiceai`        |
 | `image.tag`  | Replace with a specific version of Spice.ai to run.                                                                              | `latest` |
+| `monitoring.podMonitoring.enabled` | Enable Prometheus metrics collection for the Spice pods. Requires the [Prometheus Operator](https://prometheus-operator.dev/docs/operator/api/#monitoring.coreos.com/v1.PodMonitor) CRDs. | `false` |
 | `replicaCount`      | Number of Spice.ai replicas to run | `1`                   |
 | `image.pullSecrets` | Specify docker-registry secret names as an array                                                     | `[]`                   |
 | `tolerations`      | List of node taints to tolerate                                        | `[]`                   |
@@ -68,6 +69,33 @@ additionalEnv:
         key: spiceai-key
 ```
 
+### Monitoring
+
+The Spice Helm chart includes compatibility with the [Prometheus Operator](https://prometheus-operator.dev/) for collecting Prometheus metrics that can be visualized in the [Spice Grafana dashboard](../../clients/grafana/index.md). To enable this feature, set the `monitoring.podMonitoring.enabled` value to `true`. This will create a `PodMonitor` resource for the Spice.ai pods that will configure Prometheus to scrape metrics from the Spice.ai pods.
+
+<details>
+  <summary>Install the Prometheus Operator</summary>
+  <div>
+    The easiest way to install the Prometheus Operator along with Grafana is to use the [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/README.md) Helm chart.
+
+    ```bash
+    helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
+    helm repo update
+    helm install prometheus-stack prometheus-community/kube-prometheus-stack \
+          --set prometheus.prometheusSpec.podMonitorSelectorNilUsesHelmValues=false \
+          --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false
+    ```
+  </div>
+</details>
+
+Deploy the Spice.ai Helm chart with monitoring enabled:
+
+```bash
+helm upgrade --install spiceai spiceai/spiceai --set monitoring.podMonitoring.enabled=true
+```
+
+Once the monitoring is enabled, import the [Spice Grafana dashboard](../../clients/grafana/index.md) to visualize the Spice.ai metrics.
+
 ### Example values.yaml
 
 ```yaml
@@ -81,6 +109,9 @@ additionalEnv:
       secretKeyRef:
         name: spice-secrets
         key: spiceai-key
+monitoring:
+  podMonitor:
+    enabled: true
 spicepod:
   name: app
   version: v1beta1