Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

internal instrumentation for k8s cache #1365

Merged
merged 5 commits into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Helm chart
  • Loading branch information
mariomac committed Nov 18, 2024
commit 2ff51dfaad01c5136ed9b89ee2cc0e9157ec323a
2 changes: 1 addition & 1 deletion charts/beyla/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
apiVersion: v2
name: beyla
version: 1.4.11
version: 1.4.12
appVersion: 1.8.6
description: eBPF-based autoinstrumentation HTTP, HTTP2 and gRPC services, as well as network metrics.
home: https://grafana.com/oss/beyla-ebpf/
Expand Down
9 changes: 6 additions & 3 deletions charts/beyla/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# beyla

![Version: 1.4.11](https://img.shields.io/badge/Version-1.4.11-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.8.6](https://img.shields.io/badge/AppVersion-1.8.6-informational?style=flat-square)
![Version: 1.4.12](https://img.shields.io/badge/Version-1.4.12-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.8.6](https://img.shields.io/badge/AppVersion-1.8.6-informational?style=flat-square)

eBPF-based autoinstrumentation HTTP, HTTP2 and gRPC services, as well as network metrics.

Expand Down Expand Up @@ -41,7 +41,8 @@ eBPF-based autoinstrumentation HTTP, HTTP2 and gRPC services, as well as network
| image.registry | string | `"docker.io"` | Beyla image registry (defaults to docker.io) |
| image.repository | string | `"grafana/beyla"` | Beyla image repository. |
| image.tag | string | `nil` | Beyla image tag. When empty, the Chart's appVersion is used. |
| k8sCache | object | `{"env":{},"envValueFrom":{},"image":{"digest":null,"pullPolicy":"IfNotPresent","pullSecrets":[],"registry":"docker.io","repository":"grafana/beyla-k8s-cache","tag":null},"profile_port":0,"replicas":0,"resources":{},"service":{"annotations":{},"labels":{},"name":"beyla-k8s-cache","port":50055}}` | Options to deploy the Kubernetes metadata cache as a separate service |
| k8sCache | object | `{"annotations":{},"env":{},"envValueFrom":{},"image":{"digest":null,"pullPolicy":"IfNotPresent","pullSecrets":[],"registry":"docker.io","repository":"grafana/beyla-k8s-cache","tag":null},"internalMetrics":{"path":"/metrics","port":0,"portName":"metrics"},"podAnnotations":{},"podLabels":{},"profilePort":0,"replicas":0,"resources":{},"service":{"annotations":{},"labels":{},"name":"beyla-k8s-cache","port":50055}}` | Options to deploy the Kubernetes metadata cache as a separate service |
| k8sCache.annotations | object | `{}` | Deployment annotations. |
| k8sCache.env | object | `{}` | extra environment variables |
| k8sCache.envValueFrom | object | `{}` | extra environment variables to be set from resources such as k8s configMaps/secrets |
| k8sCache.image.digest | string | `nil` | K8s Cache image's SHA256 digest (either in format "sha256:XYZ" or "XYZ"). When set, will override `image.tag`. |
Expand All @@ -50,7 +51,9 @@ eBPF-based autoinstrumentation HTTP, HTTP2 and gRPC services, as well as network
| k8sCache.image.registry | string | `"docker.io"` | K8s Cache image registry (defaults to docker.io) |
| k8sCache.image.repository | string | `"grafana/beyla-k8s-cache"` | K8s Cache image repository. |
| k8sCache.image.tag | string | `nil` | K8s Cache image tag. When empty, the Chart's appVersion is used. |
| k8sCache.profile_port | int | `0` | Enables the profile port for the Beyla cache |
| k8sCache.podAnnotations | object | `{}` | Adds custom annotations to the Beyla Kube Cache Pods. |
| k8sCache.podLabels | object | `{}` | Adds custom labels to the Beyla Kube Cache Pods. |
| k8sCache.profilePort | int | `0` | Enables the profile port for the Beyla cache |
| k8sCache.replicas | int | `0` | Number of replicas for the Kubernetes metadata chache service. 0 disables the service. |
| k8sCache.service.annotations | object | `{}` | Service annotations. |
| k8sCache.service.labels | object | `{}` | Service labels. |
Expand Down
24 changes: 24 additions & 0 deletions charts/beyla/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -109,3 +109,27 @@ Calculate name of image ID to use for "beyla-cache".
{{- printf ":%s" .Chart.AppVersion }}
{{- end }}
{{- end }}

{{/*
Common kube cache labels
*/}}
{{- define "beyla.cache.labels" -}}
helm.sh/chart: {{ include "beyla.chart" . }}
{{ include "beyla.cache.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
app.kubernetes.io/part-of: beyla
{{- end }}

{{/*
Selector (pod) labels
*/}}
{{- define "beyla.cache.selectorLabels" -}}
app.kubernetes.io/name: {{ .Values.k8sCache.service.name }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- with .Values.k8sCache.podLabels }}
{{ toYaml . }}
{{- end }}
{{- end }}
24 changes: 17 additions & 7 deletions charts/beyla/templates/cache-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ metadata:
name: {{ .Values.k8sCache.service.name }}
namespace: {{ include "beyla.namespace" .}}
labels:
{{- include "beyla.labels" . | nindent 4 }}
{{- include "beyla.cache.labels" . | nindent 4 }}
app.kubernetes.io/component: workload
{{- with .Values.k8sCache.service.annotations }}
{{- with .Values.k8sCache.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
Expand All @@ -19,11 +19,12 @@ spec:
app.kubernetes.io/name: {{ .Values.k8sCache.service.name }}
template:
metadata:
{{- with .Values.podAnnotations }}
{{- with .Values.k8sCache.podAnnotations }}
annotations:
{{- tpl (toYaml . | nindent 8) $root }}
{{- end }}
labels:
{{- include "beyla.cache.labels" . | nindent 8 }}
app.kubernetes.io/name: {{ .Values.k8sCache.service.name }}
spec:
{{- if .Values.serviceAccount.create }}
Expand All @@ -45,9 +46,14 @@ spec:
- containerPort: {{ .Values.k8sCache.service.port }}
protocol: TCP
name: grpc
{{- if .Values.k8sCache.profile_port }}
{{- if .Values.k8sCache.profilePort }}
- name: profile
containerPort: {{ .Values.k8sCache.profile_port }}
containerPort: {{ .Values.k8sCache.profilePort }}
protocol: TCP
{{- end }}
{{- if .Values.k8sCache.internalMetrics.port }}
- name: {{ .Values.k8sCache.internalMetrics.portName }}
containerPort: {{ .Values.k8sCache.internalMetrics.port }}
protocol: TCP
{{- end }}
{{- with .Values.k8sCache.resources }}
Expand All @@ -57,9 +63,13 @@ spec:
env:
- name: BEYLA_K8S_CACHE_PORT
value: "{{ .Values.k8sCache.service.port }}"
{{- if .Values.k8sCache.profile_port }}
{{- if .Values.k8sCache.profilePort }}
- name: BEYLA_K8S_CACHE_PROFILE_PORT
value: "{{ .Values.k8sCache.profile_port }}"
value: "{{ .Values.k8sCache.profilePort }}"
{{- end }}
{{- if .Values.k8sCache.internalMetrics.port }}
- name: BEYLA_K8S_CACHE_INTERNAL_METRICS_PROMETHEUS_PORT
value: "{{ .Values.k8sCache.internalMetrics.port }}"
{{- end }}
{{- range $key, $value := .Values.k8sCache.env }}
- name: {{ $key }}
Expand Down
7 changes: 7 additions & 0 deletions charts/beyla/templates/cache-service.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,13 @@ spec:
- port: {{ .Values.k8sCache.service.port }}
protocol: TCP
targetPort: grpc
name: grpc
{{ if .Values.k8sCache.internalMetrics.port }}
- port: {{ .Values.k8sCache.internalMetrics.port }}
targetPort: {{ .Values.k8sCache.internalMetrics.portName }}
name: {{ .Values.k8sCache.internalMetrics.portName }}
protocol: TCP
{{- end }}
selector:
app.kubernetes.io/name: {{ .Values.k8sCache.service.name }}
{{- end }}
16 changes: 15 additions & 1 deletion charts/beyla/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@
# -- Number of replicas for the Kubernetes metadata chache service. 0 disables the service.
replicas: 0
# -- Enables the profile port for the Beyla cache
profile_port: 0
profilePort: 0
## Env variables that will override configmap values
## For example:
## BEYLA_K8S_CACHE_LOG_LEVEL: "debug"
Expand Down Expand Up @@ -325,3 +325,17 @@
annotations: {}
# -- Service labels.
labels: {}
internalMetrics:
# 0: disabled by default
port: 0
path: /metrics
portName: metrics
# prometheus:
# port: 6060
# path: /metrics
# -- Deployment annotations.
annotations: { }

Check failure on line 337 in charts/beyla/values.yaml

View workflow job for this annotation

GitHub Actions / lint-helm-chart

[braces] too many spaces inside empty braces
# -- Adds custom annotations to the Beyla Kube Cache Pods.
podAnnotations: {}
# -- Adds custom labels to the Beyla Kube Cache Pods.
podLabels: {}
2 changes: 1 addition & 1 deletion pkg/kubecache/instrument/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (
"github.com/grafana/beyla/pkg/internal/connector"
)

const defaultMetricsPath = "/internal/metrics"
const defaultMetricsPath = "/metrics"

type InternalMetricsConfig struct {
Port int `yaml:"port,omitempty" env:"BEYLA_K8S_CACHE_INTERNAL_METRICS_PROMETHEUS_PORT"`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,5 @@ scrape_configs:
- 'beyla-testserver:8999'
- 'beyla-pinger:8999'
- 'beyla-netolly:8999'
- 'beyla-promscrape:8999'
- 'beyla-promscrape:8999'
- 'k8s-cache:8999'
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,27 @@
package informer_cache

import (
"fmt"
"log/slog"
"os"
"strconv"
"testing"
"time"

Check failure on line 11 in test/integration/k8s/informer_cache/k8s_informer_cache_main_test.go

View workflow job for this annotation

GitHub Actions / test (1.23)

"time" imported and not used

"github.com/stretchr/testify/require"

"github.com/grafana/beyla/test/integration/components/docker"
"github.com/grafana/beyla/test/integration/components/kube"
"github.com/grafana/beyla/test/integration/components/prom"
k8s "github.com/grafana/beyla/test/integration/k8s/common"
otel "github.com/grafana/beyla/test/integration/k8s/netolly"
"github.com/grafana/beyla/test/tools"
)

const (
prometheusHostPort = "localhost:39090"
)

var cluster *kube.Kind

func TestMain(m *testing.M) {
Expand Down Expand Up @@ -70,3 +80,23 @@
func TestInformersCache_NetworkMetrics(t *testing.T) {
cluster.TestEnv().Test(t, otel.FeatureNetworkFlowBytes())
}

func TestInformersCache_InternalMetrics(t *testing.T) {
require.NotZero(t, metricVal(t, "beyla_kube_cache_client_message_submits"))
require.NotZero(t, metricVal(t, "beyla_kube_cache_connected_clients"))
require.NotZero(t, metricVal(t, "beyla_kube_cache_informer_new"))
require.NotZero(t, metricVal(t, "beyla_kube_cache_informer_update"))
require.NotZero(t, metricVal(t, "beyla_kube_cache_internal_build_info"))
}

func metricVal(t *testing.T, promQLQuery string) int {
pq := prom.Client{HostPort: prometheusHostPort}

results, err := pq.Query(promQLQuery)
require.NoError(t, err)
require.Len(t, results, 1)
require.Len(t, results[0].Value, 2)
n, err := strconv.Atoi(fmt.Sprint(results[0].Value[1]))
require.NoError(t, err)
return n
}
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,8 @@ spec:
env:
- name: BEYLA_K8S_CACHE_PORT
value: "50055"
- name: BEYLA_K8S_CACHE_INTERNAL_METRICS_PROMETHEUS_PORT
value: "8999"
- name: GOCOVERDIR
value: "/testoutput"
volumeMounts:
Expand All @@ -152,6 +154,8 @@ spec:
ports:
- containerPort: 50055
name: grpc
- containerPort: 8999
name: internalprom
---
kind: Service
apiVersion: v1
Expand All @@ -164,4 +168,7 @@ spec:
ports:
- port: 50055
name: grpc
protocol: TCP
protocol: TCP
- name: internalprom
port: 8999
protocol: TCP
Loading