k8s-light-logs

A simple logging stack for kubernetes cluster collecting logs from multiple sources:

• container logs
• systemd services
• k8s events

It packs and delivers the logs in opentelemetry logs format enriching the logs with source attributes following the otel semantic convention.

Contents:

• fluent-bit
• victoria-logs
• otel-collecotr

This repo contains an example of a k8s logging stack, which is lightweight but yet performant. There are two key guiding principles in this example:

• Use otel open standard data model for logs
  • OpenTelemetry Logs
  • OpenTelemetry Semantic Conventions
• Use lightweight backend for persisting the cluster logs
  • VictoriaLogs

The example here illustrates the deployment of log shippers, pipelines and backend using respective projects guidelines and brings the necessary additional resources and configurations. Optional components such as prometheus or ingress-controller may be installed to expose the full set of the supported scenarios.

The purpose of the repo, as of today, is to quickly provide a logging stack for k8s clusters that later can be extended to serve production environments. For example at the moment the repo does not provide, out of the box, end to end tls encryption configurations. By default, the ingress endpoints and the service monitoring are not exposed . Those shall be added later in specific deployments.

Components

This repo illustrates the deployment of:

• fluent-operator
• fluent-operator/fluent-bit
• victoria-metrics-operator
• victoria-metrics-operator/victoria-logs

And optionally:

• opentelemetry-operator
• opentelemetry-operator/otel-collector

To setup the stack:

1. Deploy single instance victoria-logs
2. Deploy fluent operator and fluent-bit
3. Optionally deploy the otel-collector

Or simply run

./setup.sh