Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sdk/log: Simple processor may be useful for production #5578

Merged
merged 11 commits into from
Jul 11, 2024
3 changes: 3 additions & 0 deletions exporters/otlp/otlplog/otlploggrpc/exporter.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ type Exporter struct {
var _ log.Exporter = (*Exporter)(nil)

// New returns a new [Exporter].
//
// It is recommended to use it with a [BatchProcessor]
// or other processor exporting records asynchronously.
func New(_ context.Context, options ...Option) (*Exporter, error) {
cfg := newConfig(options)
c, err := newClient(cfg)
Expand Down
3 changes: 3 additions & 0 deletions exporters/otlp/otlplog/otlploghttp/exporter.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ type Exporter struct {
var _ log.Exporter = (*Exporter)(nil)

// New returns a new [Exporter].
//
// It is recommended to use it with a [BatchProcessor]
// or other processor exporting records asynchronously.
func New(_ context.Context, options ...Option) (*Exporter, error) {
cfg := newConfig(options)
c, err := newHTTPClient(cfg)
Expand Down
34 changes: 25 additions & 9 deletions sdk/log/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,28 @@
`go.opentelemetry.io/otel/sdk/log` provides Logs SDK compliant with the
[specification](https://opentelemetry.io/docs/specs/otel/logs/sdk/).

The main and recommended use case is to configure the SDK to use an OTLP
exporter with a batch processor.[^1] Therefore, the design aims to be
high-performant in this scenario.

The prototype was created in
[#4955](https://github.com/open-telemetry/opentelemetry-go/pull/4955).

## Background

The goal is to design the exported API of the SDK would have low performance
overhead. Most importantly, have a design that reduces the amount of heap
allocations and even make it possible to have a zero-allocation implementation.
Eliminating the amount of heap allocations reduces the GC pressure which can
produce some of the largest improvements in performance.[^1]

The main and recommended use case is to configure the SDK to use an OTLP
exporter with a batch processor.[^2] Therefore, the implementation aims to be
high-performant in this scenario. Some users that require high throughput may
also want to use e.g. an [user_events](https://docs.kernel.org/trace/user_events.html),
[LLTng](https://lttng.org/docs/v2.13/#doc-tracing-your-own-user-application)
or [ETW](https://learn.microsoft.com/en-us/windows/win32/etw/about-event-tracing)
exporter with a simple processor. Users may also want to use
pellared marked this conversation as resolved.
Show resolved Hide resolved
[OTLP File](https://opentelemetry.io/docs/specs/otel/protocol/file-exporter/)
or [Standard Output](https://opentelemetry.io/docs/specs/otel/logs/sdk_exporters/stdout/)
exporter in order to emit logs to standard output/error or files.

## Modules structure

The SDK is published as a single `go.opentelemetry.io/otel/sdk/log` Go module.
Expand Down Expand Up @@ -112,10 +127,10 @@ The benchmark results can be found in [the prototype](https://github.com/open-te
Because the [LogRecordProcessor](https://opentelemetry.io/docs/specs/otel/logs/sdk/#logrecordprocessor)
and the [LogRecordProcessor](https://opentelemetry.io/docs/specs/otel/logs/sdk/#logrecordexporter)
abstractions are so similar, there was a proposal to unify them under
single `Expoter` interface.[^2]
single `Expoter` interface.[^3]

However, introducing a `Processor` interface makes it easier
to create custom processor decorators[^3]
to create custom processor decorators[^4]
and makes the design more aligned with the specification.

### Embed log.Record
Expand All @@ -131,6 +146,7 @@ provided via API.
Moreover it is safer to have these abstraction decoupled.
E.g. there can be a need for some fields that can be set via API and cannot be modified by the processors.

[^1]: [OpenTelemetry Logging](https://opentelemetry.io/docs/specs/otel/logs)
[^2]: [Conversation on representing LogRecordProcessor and LogRecordExporter via a single Expoter interface](https://github.com/open-telemetry/opentelemetry-go/pull/4954#discussion_r1515050480)
[^3]: [Introduce Processor](https://github.com/pellared/opentelemetry-go/pull/9)
[^1]: [A Guide to the Go Garbage Collector](https://tip.golang.org/doc/gc-guide)
[^2]: [OpenTelemetry Logging](https://opentelemetry.io/docs/specs/otel/logs)
[^3]: [Conversation on representing LogRecordProcessor and LogRecordExporter via a single Expoter interface](https://github.com/open-telemetry/opentelemetry-go/pull/4954#discussion_r1515050480)
[^4]: [Introduce Processor](https://github.com/pellared/opentelemetry-go/pull/9)
11 changes: 6 additions & 5 deletions sdk/log/simple.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,12 @@ type SimpleProcessor struct {

// NewSimpleProcessor is a simple Processor adapter.
//
// This Processor is not recommended for production use. The synchronous
// nature of this Processor make it good for testing, debugging, or
// showing examples of other features, but it can be slow and have a high
// computation resource usage overhead. [NewBatchProcessor] is recommended
// for production use instead.
// This Processor is not recommended for production use due to its synchronous
pellared marked this conversation as resolved.
Show resolved Hide resolved
// nature, which makes it suitable for testing, debugging, or demonstrating
// other features, but can lead to slow performance and high computational
// overhead. For production environments, it is recommended to use
// [NewBatchProcessor] instead. However, there may be exceptions where certain
// [Exporter] implementations perform better with this Processor.
func NewSimpleProcessor(exporter Exporter, _ ...SimpleProcessorOption) *SimpleProcessor {
if exporter == nil {
// Do not panic on nil exporter.
Expand Down
Loading