Panic in pkg/stanza/adapter/receiver

[ms-jcorley]

Component(s)

pkg/stanza

What happened?

Description

When running with an udplogreceiver -> batch processor -> exporter with very high load for long periods of time we are seeing an occasional panic (see stack below)

I believe the source of the problem is here:
https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/stanza/adapter/receiver.go#L128

Line 128 calls pLogs.LogRecordCount after line 124 calls r.consumer.ConsumeLogs.
Issue 1: If the consumer is mutating the pLogs asynchronously while LogRecordCount is looping, you can get a panic.
Issue 2: If pLogs is modified synchronously in ConsumeLogs, the count could be wrong (assuming that the incoming count is what is desired)

Proposed solution:
I think the count should be taken before ConsumeLogs is called and saved in a local var for EndLogsOp to use later.

Steps to Reproduce

Expected Result

Actual Result

Collector version

v0.88.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

No response

Log output

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xac4840]
goroutine 182 [running]:
go.opentelemetry.io/collector/pdata/plog.ResourceLogsSlice.At(...)
go.opentelemetry.io/collector/pdata@v1.0.0-rcv0017/plog/generated_resourcelogsslice.go:56
go.opentelemetry.io/collector/pdata/plog.Logs.LogRecordCount({0xc002760678?, 0xc001f3cc58?})
go.opentelemetry.io/collector/pdata@v1.0.0-rcv0017/plog/logs.go:48 +0x20
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/adapter.(*receiver).consumerLoop(0xc000591950, {0x3ecf9b8, 0xc000895860})
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.88.0/adapter/receiver.go:128 +0x325
created by github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/adapter.(*receiver).Start in goroutine 1
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.88.0/adapter/receiver.go:68 +0x285

Additional context

No response

[github-actions]

Pinging code owners:

• pkg/stanza: @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.