NETOBSERV-584: Support for writing logs to Loki (distributor) using gRPC #1973

leandroberetta · 2025-09-11T13:17:40Z

Description

Support for writing logs to Loki (distributor) using gRPC.

Ref: https://issues.redhat.com/browse/NETOBSERV-584

Dependencies

netobserv/loki-client-go#3
netobserv/flowlogs-pipeline#1086
netobserv/network-observability-console-plugin#1021

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

openshift-ci · 2025-09-11T13:17:45Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

openshift-ci · 2025-09-11T13:17:47Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from leandroberetta. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

codecov · 2025-09-11T18:13:34Z

Codecov Report

❌ Patch coverage is 26.03550% with 125 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.27%. Comparing base (20eb307) to head (7eea6fb).

Files with missing lines	Patch %	Lines
internal/controller/flp/flp_pipeline_builder.go	26.59%	66 Missing and 3 partials ⚠️
api/flowcollector/v1beta2/zz_generated.deepcopy.go	0.00%	23 Missing and 1 partial ⚠️
internal/pkg/helper/loki_config.go	45.71%	16 Missing and 3 partials ⚠️
.../controller/consoleplugin/consoleplugin_objects.go	18.75%	11 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1973      +/-   ##
==========================================
- Coverage   71.02%   70.27%   -0.75%     
==========================================
  Files          75       75              
  Lines       10011    10143     +132     
==========================================
+ Hits         7110     7128      +18     
- Misses       2513     2622     +109     
- Partials      388      393       +5

Flag	Coverage Δ
unittests	`70.27% <26.03%> (-0.75%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
api/flowcollector/v1beta2/flowcollector_types.go	`100.00% <ø> (ø)`
.../controller/consoleplugin/consoleplugin_objects.go	`82.55% <18.75%> (-2.13%)`	⬇️
internal/pkg/helper/loki_config.go	`79.34% <45.71%> (-20.66%)`	⬇️
api/flowcollector/v1beta2/zz_generated.deepcopy.go	`38.83% <0.00%> (-0.97%)`	⬇️
internal/controller/flp/flp_pipeline_builder.go	`67.80% <26.59%> (-6.22%)`	⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions · 2025-09-11T18:57:59Z

New images:

quay.io/netobserv/network-observability-operator:7c27f48
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-sha-7c27f48
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-sha-7c27f48

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:7c27f48 make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-sha-7c27f48

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-sha-7c27f48
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

github-actions · 2025-09-11T21:15:33Z

New images:

quay.io/netobserv/network-observability-operator:6425002
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-sha-6425002
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-sha-6425002

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:6425002 make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-sha-6425002

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-sha-6425002
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

openshift-ci-robot · 2025-09-23T01:21:50Z

leandroberetta · 2025-09-23T01:25:05Z

This PR adds support for writing logs to Loki using gRPC. In terms of performance, results are pretty similar to the existing http implementation.

I was investigating this and the current implementation uses http but messages are protocol buffers compressed (snappy), so it's already using less memory I guess.

In other hands, gRPC establishes a connection and has mechanisms to reconnect, keep alive. That take more resources (according to what I read).

Probably we can do a better performance testing until we can decide.

github-actions · 2025-09-23T13:21:37Z

New images:

quay.io/netobserv/network-observability-operator:770611c
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-sha-770611c
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-sha-770611c

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:770611c make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-sha-770611c

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-sha-770611c
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

jpinsonneau · 2025-09-23T14:23:30Z

api/flowcollector/v1beta2/flowcollector_types.go

+type LokiGRPCConfig struct {
+	//+kubebuilder:validation:Minimum=1024
+	//+kubebuilder:validation:Maximum=67108864
+	//+kubebuilder:default:=67108864
+	// `maxRecvMsgSize` is the maximum message size in bytes the gRPC client can receive. Default: 64MB.
+	MaxRecvMsgSize int `json:"maxRecvMsgSize,omitempty"`
+
+	//+kubebuilder:validation:Minimum=1024
+	//+kubebuilder:validation:Maximum=67108864
+	//+kubebuilder:default:=16777216
+	// `maxSendMsgSize` is the maximum message size in bytes the gRPC client can send. Default: 16MB.
+	MaxSendMsgSize int `json:"maxSendMsgSize,omitempty"`
+
+	//+kubebuilder:default:="30s"
+	// `keepAlive` is the gRPC keep-alive interval.
+	KeepAlive *metav1.Duration `json:"keepAlive,omitempty"`
+
+	//+kubebuilder:default:="5s"
+	// `keepAliveTimeout` is the gRPC keep-alive timeout.
+	KeepAliveTimeout *metav1.Duration `json:"keepAliveTimeout,omitempty"`
+}


I'm not sure we need those exposed in the API.

On top of that, what's the difference between maxSendMsgSize and the existing writeBatchSize ?
Maybe we can reuse the same field here and put the other options as Env map[string]string in the advanced section

Here is an example for eBPF agent consuming envs: https://github.com/netobserv/network-observability-operator/blob/main/internal/controller/ebpf/agent_controller.go#L33-L75

Yes I think we can probably start with not exposing anything at all (not even the clientType) and have it under a env-based feature gate. Probably users want us to make the choice for them, whether to use it or not.
If after more tests we see that there are no clear winner, just pros and cons, then maybe we will expose it?

bundle/manifests/netobserv-operator.clusterserviceversion.yaml

config/manager/manager.yaml

openshift-ci · 2025-10-09T18:07:44Z

@leandroberetta: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-operator	`8334899`	link	false	`/test e2e-operator`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

leandroberetta self-assigned this Sep 11, 2025

openshift-ci bot added the do-not-merge/work-in-progress label Sep 11, 2025

openshift-merge-robot added needs-rebase and removed needs-rebase labels Sep 11, 2025

leandroberetta added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Sep 11, 2025

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Sep 11, 2025

leandroberetta added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Sep 11, 2025

leandroberetta mentioned this pull request Sep 12, 2025

NETOBSERV-584: Add new grpc related fields to schema netobserv/network-observability-console-plugin#1021

Open

10 tasks

leandroberetta marked this pull request as ready for review September 15, 2025 11:46

openshift-ci bot removed the do-not-merge/work-in-progress label Sep 15, 2025

openshift-merge-robot added the needs-rebase label Sep 22, 2025

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Sep 23, 2025

leandroberetta force-pushed the netobserv-584 branch from 6cd213f to 582bdd9 Compare September 23, 2025 01:16

openshift-merge-robot removed the needs-rebase label Sep 23, 2025

leandroberetta force-pushed the netobserv-584 branch from ce60a96 to 9200b33 Compare September 23, 2025 01:19

leandroberetta requested review from OlivierCazade, jotak and jpinsonneau September 23, 2025 01:19

leandroberetta changed the title ~~Support for writing logs to Loki (distributor) using gRPC~~ NETOBSERV 584: Support for writing logs to Loki (distributor) using gRPC Sep 23, 2025

leandroberetta changed the title ~~NETOBSERV 584: Support for writing logs to Loki (distributor) using gRPC~~ NETOBSERV-584: Support for writing logs to Loki (distributor) using gRPC Sep 23, 2025

openshift-ci-robot added the jira/valid-reference label Sep 23, 2025

leandroberetta added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Sep 23, 2025

jpinsonneau reviewed Sep 23, 2025

View reviewed changes

jotak reviewed Sep 23, 2025

View reviewed changes

bundle/manifests/netobserv-operator.clusterserviceversion.yaml Outdated Show resolved Hide resolved

jotak reviewed Sep 23, 2025

View reviewed changes

config/manager/manager.yaml Outdated Show resolved Hide resolved

openshift-merge-robot added the needs-rebase label Sep 24, 2025

leandroberetta force-pushed the netobserv-584 branch from b3004b4 to 188c7f5 Compare September 24, 2025 17:03

openshift-merge-robot removed the needs-rebase label Sep 24, 2025

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Sep 24, 2025

leandroberetta force-pushed the netobserv-584 branch from 188c7f5 to 7f98af0 Compare September 24, 2025 17:08

openshift-merge-robot added the needs-rebase label Oct 9, 2025

leandroberetta force-pushed the netobserv-584 branch from f944dbe to 6b4e3db Compare October 9, 2025 14:18

openshift-merge-robot removed the needs-rebase label Oct 9, 2025

Support for configure grpc client to send logs to Loki

120a626

leandroberetta force-pushed the netobserv-584 branch from 6b4e3db to 120a626 Compare October 9, 2025 16:36

leandroberetta added 2 commits October 9, 2025 13:38

fix vendors

a828c25

fix flp dep

8334899

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

NETOBSERV-584: Support for writing logs to Loki (distributor) using gRPC #1973

NETOBSERV-584: Support for writing logs to Loki (distributor) using gRPC #1973

Uh oh!

leandroberetta commented Sep 11, 2025 •

edited

Loading

Uh oh!

openshift-ci bot commented Sep 11, 2025

Uh oh!

openshift-ci bot commented Sep 11, 2025

Uh oh!

codecov bot commented Sep 11, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Sep 11, 2025

Uh oh!

github-actions bot commented Sep 11, 2025

Uh oh!

openshift-ci-robot commented Sep 23, 2025 •

edited by openshift-ci bot

Loading

Description

Dependencies

Checklist

Uh oh!

leandroberetta commented Sep 23, 2025

Uh oh!

github-actions bot commented Sep 23, 2025

Uh oh!

jpinsonneau Sep 23, 2025

Uh oh!

jotak Sep 23, 2025

Uh oh!

Uh oh!

Uh oh!

openshift-ci bot commented Oct 9, 2025

Uh oh!

Uh oh!

NETOBSERV-584: Support for writing logs to Loki (distributor) using gRPC #1973

Are you sure you want to change the base?

NETOBSERV-584: Support for writing logs to Loki (distributor) using gRPC #1973

Uh oh!

Conversation

leandroberetta commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Dependencies

Checklist

Uh oh!

openshift-ci bot commented Sep 11, 2025

Uh oh!

openshift-ci bot commented Sep 11, 2025

Uh oh!

codecov bot commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Sep 11, 2025

Uh oh!

github-actions bot commented Sep 11, 2025

Uh oh!

openshift-ci-robot commented Sep 23, 2025 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Dependencies

Checklist

Uh oh!

leandroberetta commented Sep 23, 2025

Uh oh!

github-actions bot commented Sep 23, 2025

Uh oh!

jpinsonneau Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

jotak Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

openshift-ci bot commented Oct 9, 2025

Uh oh!

Uh oh!

leandroberetta commented Sep 11, 2025 •

edited

Loading

codecov bot commented Sep 11, 2025 •

edited

Loading

openshift-ci-robot commented Sep 23, 2025 •

edited by openshift-ci bot

Loading