OBSDOCS-2819: Document how to scrape a custom log file from a pod filesystem

Author: max-cx

modules/otel-collecting-container-logs-with-filelog-sidecar.adoc

@@ -0,0 +1,67 @@
+// Module included in the following assemblies:
+//
+// * observability/otel/otel-sending-traces-logs-and-metrics-to-otel-collector.adoc
+
+:_mod-docs-content-type: PROCEDURE
+[id="collecting-container-logs-with-filelog-sidecar_{context}"]
+= Collecting container logs with the Filelog Receiver and sidecar injection
+
+[role="_abstract"]
+To collect logs from application containers, you can use the Filelog Receiver with sidecar injection. This approach is useful when your application writes logs to files within the container filesystem.
+
+.Prerequisites
+
+* You have access to the cluster as a cluster administrator with the `cluster-admin` role.
+
+The {OTELOperator} allows injecting an OpenTelemetry Collector as a sidecar container into a application pod. To access the generated files, both pods require a shared volume for the application container and the sidecar Collector. The Filelog Receiver can then tail log files and apply Operators to parse and transform the logs.
+
+.Procedure
+
+. Create an OpenTelemetry Collector instance configured with the filelog Receiver in sidecar mode. Ensure a matching `volumeMounts` entry is set to grant the sidecar access to your targeted files.
++
+[source,yaml]
+----
+apiVersion: opentelemetry.io/v1beta1
+kind: OpenTelemetryCollector
+metadata:
+  name: filelog
+  namespace: otel-logging
+spec:
+  mode: sidecar
+  volumeMounts: # <1>
+  - name: logs
+    mountPath: /var/log/app
+  config:
+    receivers:
+      filelog:
+        include: # <2>
+        - /var/log/app/*.log
+        operators:
+        - type: regex_parser
+          regex: '^(?P<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) \[(?P<level>\w+)\] (?P<message>.*)$'
+          timestamp:
+            parse_from: attributes.timestamp
+            layout: '%Y-%m-%d %H:%M:%S'
+    processors: {}
+    exporters:
+      debug:
+        verbosity: detailed
+    service:
+      pipelines:
+        logs:
+          receivers: [filelog]
+          processors: []
+          exporters: [debug]
+----
+<1> Defines the volume mount that the sidecar collector will use to access the log files. This volume must match the volume name defined in the application deployment.
+<2> Specifies file glob patterns for matching the log files to tail. The Filelog Receiver watches these paths for new log entries.
++
+[IMPORTANT]
+====
+The `volumeMounts` field in the `OpenTelemetryCollector` custom resource is critical for the sidecar to access log files. The volume specified here must be defined in the application's `Deployment` or `Pod` specification, and both the application container and the sidecar collector must mount the same volume.
+====
+
+[role="_additional-resources"]
+.Additional resources
+* xref:../../../observability/otel/otel-collector/otel-collector-receivers.adoc#filelog-receiver_otel-collector-receivers[Filelog Receiver]
+* xref:../../../observability/otel/otel-collector/otel-collector-receivers.adoc#otel-collector-receivers[Receivers]

modules/otel-receivers-filelog-receiver.adoc

@@ -32,3 +32,5 @@ include::snippets/technology-preview.adoc[]
 ----
 <1> A list of file glob patterns that match the file paths to be read.
 <2> An array of Operators. Each Operator performs a simple task such as parsing a timestamp or JSON. To process logs into a desired format, chain the Operators together.
+
+When using the Filelog Receiver in sidecar mode to collect logs from application containers, you must configure volume mounts in the `OpenTelemetryCollector` custom resource. The Collector requires access to the log files through a shared volume, such as `emptyDir`, that is mounted in both the application container and the sidecar collector container. For a complete example, see "Collecting container logs with the Filelog Receiver and sidecar injection".

observability/otel/otel-sending-traces-logs-and-metrics-to-otel-collector.adoc

@@ -13,3 +13,5 @@ Sending traces and metrics to the OpenTelemetry Collector is possible with or wi
 include::modules/otel-send-traces-and-metrics-to-otel-collector-with-sidecar.adoc[leveloffset=+1]
 
 include::modules/otel-send-traces-and-metrics-to-otel-collector-without-sidecar.adoc[leveloffset=+1]
+
+include::modules/otel-collecting-container-logs-with-filelog-sidecar.adoc[leveloffset=+1]