Collector service monitor does not scape prometheus exporter

[pavolloffay]

Component(s)

collector

What happened?

Description

  observability:
    metrics:
      enableMetrics: true

Creates service monitor, however only pipeline collector metrics are scraped (port 8888).

Steps to Reproduce

kubectl apply -f - <<EOF
apiVersion: opentelemetry.io/v1beta1
kind: OpenTelemetryCollector
metadata:
  name: simplest
spec:
  observability:
    metrics:
      enableMetrics: true
  config:
    receivers:
      otlp:
        protocols:
          grpc: {}
          http: {}
    processors:
      memory_limiter:
        check_interval: 1s
        limit_percentage: 75
        spike_limit_percentage: 15
      batch:
        send_batch_size: 10000
        timeout: 10s

    exporters:
      debug: {}
      prometheus:
        endpoint: 0.0.0.0:8889
        resource_to_telemetry_conversion:
          enabled: true # by default resource attributes are dropped

    service:
      pipelines:
        metrics:
          receivers: [otlp]
          processors: []
          exporters: [debug, prometheus]
EOF

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  creationTimestamp: "2024-05-17T07:17:43Z"
  generation: 1
  labels:
    app.kubernetes.io/component: opentelemetry-collector
    app.kubernetes.io/instance: default.simplest
    app.kubernetes.io/managed-by: opentelemetry-operator
    app.kubernetes.io/name: simplest-collector
    app.kubernetes.io/part-of: opentelemetry
    app.kubernetes.io/version: latest
  name: simplest-collector
  namespace: default
  ownerReferences:
  - apiVersion: opentelemetry.io/v1beta1
    blockOwnerDeletion: true
    controller: true
    kind: OpenTelemetryCollector
    name: simplest
    uid: 4f7a1341-4c03-45e4-b6f0-6f0036eba791
  resourceVersion: "52828"
  uid: 7e9c6a00-7c95-4b67-8894-7f9a98f661b0
spec:
  endpoints:
  - port: monitoring
  - port: prometheus
  namespaceSelector:
    matchNames:
    - default
  selector:
    matchLabels:
      app.kubernetes.io/component: opentelemetry-collector
      app.kubernetes.io/instance: default.simplest
      app.kubernetes.io/managed-by: opentelemetry-operator
      app.kubernetes.io/part-of: opentelemetry
      operator.opentelemetry.io/collector-monitoring-service: Exists  # <---- this matches only the monitoring service

k get service simplest-collector-monitoring -o yaml                                                                                                                                                                                                                                                                                                           ploffay@fedora
apiVersion: v1
kind: Service
metadata:
  annotations:
  creationTimestamp: "2024-05-17T07:17:43Z"
  labels:
    app.kubernetes.io/component: opentelemetry-collector
    app.kubernetes.io/instance: default.simplest
    app.kubernetes.io/managed-by: opentelemetry-operator
    app.kubernetes.io/name: simplest-collector-monitoring
    app.kubernetes.io/part-of: opentelemetry
    app.kubernetes.io/version: latest
    operator.opentelemetry.io/collector-monitoring-service: Exists
  name: simplest-collector-monitoring
  namespace: default
  ownerReferences:
  - apiVersion: opentelemetry.io/v1beta1
    blockOwnerDeletion: true
    controller: true
    kind: OpenTelemetryCollector
    name: simplest
    uid: 4f7a1341-4c03-45e4-b6f0-6f0036eba791
  resourceVersion: "52825"
  uid: f9805b36-d45e-4014-9408-a22f2d634962
spec:
  clusterIP: 10.96.227.230
  clusterIPs:
  - 10.96.227.230
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: monitoring
    port: 8888
    protocol: TCP
    targetPort: 8888
  selector:
    app.kubernetes.io/component: opentelemetry-collector
    app.kubernetes.io/instance: default.simplest
    app.kubernetes.io/managed-by: opentelemetry-operator
    app.kubernetes.io/part-of: opentelemetry
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2024-05-17T07:17:43Z"
  labels:
    app.kubernetes.io/component: opentelemetry-collector
    app.kubernetes.io/instance: default.simplest
    app.kubernetes.io/managed-by: opentelemetry-operator
    app.kubernetes.io/name: simplest-collector
    app.kubernetes.io/part-of: opentelemetry
    app.kubernetes.io/version: latest
  name: simplest-collector
  namespace: default
  ownerReferences:
  - apiVersion: opentelemetry.io/v1beta1
    blockOwnerDeletion: true
    controller: true
    kind: OpenTelemetryCollector
    name: simplest
    uid: 4f7a1341-4c03-45e4-b6f0-6f0036eba791
  resourceVersion: "52817"
  uid: dcfa3688-b807-4a9a-963c-ae5ef646871f
spec:
  clusterIP: 10.108.22.138
  clusterIPs:
  - 10.108.22.138
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - appProtocol: grpc
    name: otlp-grpc
    port: 4317
    protocol: TCP
    targetPort: 4317
  - appProtocol: http
    name: otlp-http
    port: 4318
    protocol: TCP
    targetPort: 4318
  - name: prometheus
    port: 8889
    protocol: TCP
    targetPort: 8889
  selector:
    app.kubernetes.io/component: opentelemetry-collector
    app.kubernetes.io/instance: default.simplest
    app.kubernetes.io/managed-by: opentelemetry-operator
    app.kubernetes.io/part-of: opentelemetry
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

Expected Result

Actual Result

Kubernetes Version

latest

Operator version

0.99.0

Collector version

0.99.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

Log output

No response

Additional context

No response

[pavolloffay]

IIRC this is caused by #2573

[pavolloffay]

This seems to be resolved by #2878