Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PoC] Change Processor.OnEmit to return record passed to next registered processor #5470

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

pellared
Copy link
Member

@pellared pellared commented Jun 3, 2024

Towards #5219

Related to open-telemetry/opentelemetry-specification#4065

Alternative proposal: #5469

Changes Processor.OnEmit to:

OnEmit(ctx context.Context, record Record) (Record, error)

Pros:

  • MAY be seen as acceptable by spec (however, probably the spec should still be refined)
  • Almost no performance overhead, zero-allocation is possible. There are more operations on the stack.

Cons:

Personally, I dislike the design of this proposal as I outlined here open-telemetry/opentelemetry-specification#4010 (comment)

Processor benchmark results:

$ benchstat old.txt new.txt
goos: linux
goarch: amd64
pkg: go.opentelemetry.io/otel/sdk/log
cpu: Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz
                                    │   old.txt    │               new.txt                │
                                    │    sec/op    │    sec/op     vs base                │
Processor/Simple-16                   527.3n ± 19%   574.0n ± 15%   +8.86% (p=0.022 n=10)
Processor/Batch-16                    1.013µ ±  6%   1.089µ ±  4%   +7.45% (p=0.003 n=10)
Processor/ModifyTimestampSimple-16    573.2n ± 12%   646.1n ±  5%  +12.72% (p=0.005 n=10)
Processor/ModifyTimestampBatch-16     1.036µ ±  4%   1.106µ ±  8%   +6.71% (p=0.050 n=10)
Processor/ModifyAttributesSimple-16   616.2n ± 12%   688.8n ±  8%  +11.77% (p=0.035 n=10)
Processor/ModifyAttributesBatch-16    1.085µ ±  9%   1.158µ ±  3%   +6.78% (p=0.024 n=10)
geomean                               772.2n         841.8n         +9.02%

                                    │  old.txt   │              new.txt              │
                                    │    B/op    │    B/op     vs base               │
Processor/Simple-16                   416.0 ± 0%   416.0 ± 0%       ~ (p=1.000 n=10)
Processor/Batch-16                    622.5 ± 2%   616.0 ± 1%       ~ (p=0.137 n=10)
Processor/ModifyTimestampSimple-16    417.0 ± 0%   416.0 ± 0%       ~ (p=0.656 n=10)
Processor/ModifyTimestampBatch-16     618.5 ± 3%   609.0 ± 2%       ~ (p=0.137 n=10)
Processor/ModifyAttributesSimple-16   465.0 ± 0%   465.0 ± 0%       ~ (p=1.000 n=10)
Processor/ModifyAttributesBatch-16    652.5 ± 2%   651.0 ± 2%       ~ (p=0.781 n=10)
geomean                               522.1        519.5       -0.51%

                                    │   old.txt    │               new.txt               │
                                    │  allocs/op   │ allocs/op   vs base                 │
Processor/Simple-16                   1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=10) ¹
Processor/Batch-16                    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Processor/ModifyTimestampSimple-16    1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=10) ¹
Processor/ModifyTimestampBatch-16     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Processor/ModifyAttributesSimple-16   2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=10) ¹
Processor/ModifyAttributesBatch-16    1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=10) ¹
geomean                                          ²               +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean


                        │   old.txt   │            new.txt            │
                        │   sec/op    │   sec/op     vs base          │
BatchProcessorOnEmit-16   385.9n ± 8%   385.5n ± 7%  ~ (p=0.493 n=10)

                        │    old.txt     │               new.txt                │
                        │      B/s       │     B/s       vs base                │
BatchProcessorOnEmit-16   1047.85Mi ± 9%   79.17Mi ± 6%  -92.44% (p=0.000 n=10)

                        │  old.txt   │            new.txt             │
                        │    B/op    │    B/op     vs base            │
BatchProcessorOnEmit-16   0.000 ± 0%   0.000 ± 0%  ~ (p=1.000 n=10) ¹
¹ all samples are equal

                        │  old.txt   │            new.txt             │
                        │ allocs/op  │ allocs/op   vs base            │
BatchProcessorOnEmit-16   0.000 ± 0%   0.000 ± 0%  ~ (p=1.000 n=10) ¹
¹ all samples are equal

Copy link

codecov bot commented Jun 3, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.6%. Comparing base (047df28) to head (66cf41b).
Report is 263 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##            main   #5470     +/-   ##
=======================================
- Coverage   84.6%   84.6%   -0.1%     
=======================================
  Files        267     267             
  Lines      17825   17826      +1     
=======================================
- Hits       15086   15085      -1     
- Misses      2426    2428      +2     
  Partials     313     313             
Files with missing lines Coverage Δ
sdk/log/batch.go 100.0% <100.0%> (ø)
sdk/log/logger.go 100.0% <100.0%> (ø)
sdk/log/simple.go 100.0% <100.0%> (ø)

... and 1 file with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant