googlecloudpubsubreceiver: google.cloud.audit.Auth orizationInfo has no known field with JSON name permissionType

[rafaeljesus]

Component(s)

receiver/googlecloudpubsub

What happened?

Description

I have a logging sink exporting writing logs to a pubsub topic that the pubsub receiver is consuming from,

the logging sink inclusion filter is as follows:

NOT (
  (resource.type="k8s_container" OR resource.type="k8s_pod") AND
  NOT resource.labels.namespace_name=~"kube-*"
) AND
NOT resource.type="cloud_run_revision"

below is the screenshot of the data in Cloud Logging, as you can see there is a permissionType: "ADMIN_READ"

Expected Result

data to be exported to the internal logging system

Actual Result

data is not being exported to the internal logging system, and instead, I am having the following error in logs "google.cloud.audit.AuthorizationInfo has no known field with JSON name permissionType"

Collector version

0.95.0

Environment information

Environment

GKE

OpenTelemetry Collector configuration

config:
  exporters:
    otelarrow/logs:
      endpoint: 'ingest.lightstep.com:443'
      headers:
        lightstep-access-token: '${LS_TOKEN}'
  receivers:
    googlecloudpubsub:
      encoding: cloud_logging
      project: org-eu1-staging
      subscription: projects/org-eu1-staging/subscriptions/org-logs-forwarder
  service:
    pipelines:
      logs:
        receivers:
          - googlecloudpubsub
        exporters:
          - otelarrow/logs

Log output

github.com/open-telemetry/opentelemetry-collector-contrib/receiver/googlecloudpubsubreceiver.(*pubsubReceiver).handleCloudLoggingLogEntry
        github.com/open-telemetry/opentelemetry-collector-contrib/receiver/googlecloudpubsubreceiver@v0.99.0/receiver.go:150
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/googlecloudpubsubreceiver.(*pubsubReceiver).createReceiverHandler.func1
        github.com/open-telemetry/opentelemetry-collector-contrib/receiver/googlecloudpubsubreceiver@v0.99.0/receiver.go:299
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/googlecloudpubsubreceiver/internal.(*StreamHandler).responseStream
        github.com/open-telemetry/opentelemetry-collector-contrib/receiver/googlecloudpubsubreceiver@v0.99.0/internal/handler.go:193
2024-06-05T17:05:08.890Z        error   googlecloudpubsubreceiver@v0.99.0/receiver.go:150       got an error    {"kind": "receiver", "name": "googlecloudpubsub", "data_type": "logs", "error": "google.cloud.audit.AuthorizationInfo has no known field with JSON name permissionType"}

Additional context

No response

[github-actions]

Pinging code owners:

• receiver/googlecloudpubsub: @alexvanboxel

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[alexvanboxel]

This is indeed an issue I've saw as well, I have the fix, but first this one needs to go through: #33247 (because I have a fix ontop of the this PR, as it will make the parsing configurable).

The solution will be to disable the Protobuf based parsing (I want to get rid of that, it's not a good idea). The fix will be braking through, as the default will be just JSON based parsing.

[rafaeljesus]

@alexvanboxel thanks for the quick response! I will keep an eye on the other issues then

[alexvanboxel]

Thank you for the issue, now I have something to link the fix to.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/googlecloudpubsub: @alexvanboxel

See Adding Labels via Comments if you do not have permissions to add labels yourself.