Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge remote-tracking branch 'upstream/main' #45

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
53ad0a4
[chore]: enable bool-compare rule from testifylint (#34912)
mmorel-35 Aug 29, 2024
240ff76
[pkg/ottl] Remove tracing from OTTL (#34910)
TylerHelmuth Aug 29, 2024
29cd095
[chore]: enable len rule from testifylint (#34921)
mmorel-35 Aug 30, 2024
0ec84e6
[chore][receiver/nginx] Update README for consistency (#34923)
crobert-1 Aug 30, 2024
184e954
[processor/redaction] add support for redacting metrics and logs attr…
bacherfl Aug 30, 2024
499cfd8
[chore] Fix linter issues (#34937)
mx-psi Aug 30, 2024
903eec5
[receiver/gitproviderreceiver] rename to githubreceiver (#34731)
adrielp Aug 30, 2024
6b3237a
[chore][CODEOWNERS] Change account references to match new username (…
crobert-1 Aug 30, 2024
2d63c17
[chore] refactor filter processor test to use generated test harness …
codeboten Aug 30, 2024
ea521f7
[chore] use generated test harness for groupbyattrs processor (#34941)
codeboten Aug 30, 2024
0216163
[chore] update core to pull in telemetry settings changes (#34930)
codeboten Aug 30, 2024
90e75d3
[receiver/splunkhec] fix memory leak (#34911)
atoulme Aug 30, 2024
dbf8982
[chore] Updating @MovieStoreGuy affiliation (#34947)
MovieStoreGuy Sep 1, 2024
fd0e9ef
[chore]: Update README.md (#34725)
CharlieTLe Sep 2, 2024
33362ee
[connector/servicegraph]Fix incorrectly reversed latency settings(res…
Frapschen Sep 2, 2024
9f0cac0
[chore]: enable error-nil and nil-compare rules from testifylint (#34…
mmorel-35 Sep 2, 2024
13f0d14
[connector/count] fix typo in metric.datapoint.count (#34961)
lhns Sep 2, 2024
255020c
[chore] Upgrade arvo package (#34962)
MovieStoreGuy Sep 3, 2024
439e41e
Update module github.com/shirou/gopsutil/v4 to v4.24.8 (#34970)
renovate[bot] Sep 3, 2024
4558429
Update module github.com/ClickHouse/clickhouse-go/v2 to v2.28.2 (#34968)
renovate[bot] Sep 3, 2024
1c9e54e
Update All github.com/datadog packages to v0.56.2 (#34965)
renovate[bot] Sep 3, 2024
ad02711
Update module github.com/snowflakedb/gosnowflake to v1.11.1 (#34971)
renovate[bot] Sep 3, 2024
a967d83
[exporter/loki] Document the migration from the Loki Exporter to the …
cyrille-leclerc Sep 3, 2024
9545957
[connector/spanmetrics] Improve consistency between metrics generated…
iblancasa Sep 3, 2024
5822332
Update module google.golang.org/grpc to v1.66.0 (#34978)
renovate[bot] Sep 3, 2024
d25990b
[vcenterreceiver] Updated units on several metrics (#34946)
BominRahmani Sep 3, 2024
c6cda87
Update module github.com/SAP/go-hdb to v1.12.0 (#34973)
renovate[bot] Sep 3, 2024
b128c46
[chore][pkg/stanza] Speed up file deduplication in finder (#34888)
BinaryFissionGames Sep 3, 2024
510a413
[receiver/mysql] client, convert NULL to int64 (#34411)
knarfli Sep 3, 2024
bfdee42
[testbed] - Add scenarios to handle large files (#34417)
VihasMakwana Sep 3, 2024
640adad
Avoid rendering the whole event to get only the provider name (#34914)
pjanotti Sep 3, 2024
119ad9b
[chore] add Tyler to CODEOWNERS for githubreceiver (#34963)
adrielp Sep 3, 2024
58b5bb9
Update README.md (#34990)
gjtorikian Sep 3, 2024
f6156a9
[extension/opamp] Add support for polling interval in HTTP client (#3…
matej-g Sep 3, 2024
a77faff
[receiver/apache] Invalid endpoint should not cause panic (#34992)
pjanotti Sep 3, 2024
1ec7cfe
[chore]: enable compares and empty rules from testifylint (#34976)
mmorel-35 Sep 3, 2024
dabacbd
[receiver/githubreceiver] promote githubreceiver to alpha status (#34…
adrielp Sep 3, 2024
902756e
[connector/datadog] Improve performance in cases with many peer tags …
songy23 Sep 3, 2024
ec049ab
[chore] upgrading pulsar client v0.13.1 (#34951)
MovieStoreGuy Sep 3, 2024
a4aa4e8
[processor/interval] Update config structure for interval processor (…
ArthurSens Sep 3, 2024
d0e5eec
[extension/solarwindsapmsettingsextension] Added remaining implementa…
jerrytfleung Sep 3, 2024
5258d98
feat: Support Prometheus Created Timestamps (#34596)
ArthurSens Sep 4, 2024
3afb802
[receiver/sqlquery] support attributes for logs (#34599)
Grandys Sep 4, 2024
9be1f80
[receiver/kafkareceiver]: allow tunable fetch sizes (#34431)
cxdy Sep 4, 2024
2de5752
[exporter/splunkhec] drop empty log events (#34871)
atoulme Sep 4, 2024
a088f60
[exporter/signalfx] Fix goroutine leaks (#32781)
crobert-1 Sep 4, 2024
1ba4179
[chore] make update-otel to 48b11ba (#34956)
mx-psi Sep 4, 2024
33687ee
[chore][receiver/filelog] Change plaintext name: filelog -> file log …
crobert-1 Sep 4, 2024
9e3a845
[chore] Fix flaky test by ignoring metrics order (#35002)
djaglowski Sep 4, 2024
255aebe
fix: handle OTLPJSON unmarshal error (#34784)
rogercoll Sep 4, 2024
548d95f
[chore] Fix flaky test in filelog receiver (#35012)
djaglowski Sep 4, 2024
e393e02
Bump github.com/opencontainers/runc from 1.1.13 to 1.1.14 in /cmd/ote…
dependabot[bot] Sep 4, 2024
a0ea89c
Clarify google managed prometheus troubleshooting guide (#34986)
dashpole Sep 4, 2024
fa4cac5
Update module github.com/lestrrat-go/strftime to v1.1.0 (#34974)
renovate[bot] Sep 4, 2024
93ba625
[chore] Upgrading msgpack v4 to v5 (#35019)
MovieStoreGuy Sep 5, 2024
cb71224
[exporter/elasticsearch] Fix dynamic mapping for double values storin…
carsonip Sep 5, 2024
4c490fe
[exporter/elasticsearch] Add exponential histogram support (#34818)
carsonip Sep 5, 2024
53152f0
[processor/deltatocumulative]: drop bad samples (#34979)
sh0rez Sep 5, 2024
fa75b6e
fix: avoid using internal empty attribute.Set pointer (#34903)
rogercoll Sep 5, 2024
ceeb395
Update example in readme for convert_sum_to_gauge and convert_gauge_t…
wildum Sep 5, 2024
9a31997
[exporter/elasticsearch] Add span event to traces OTel mapping mode (…
carsonip Sep 5, 2024
a4393cb
[opampsupervisor] Add HealthCheckPort configuration parameter (#34704)
dpaasman00 Sep 5, 2024
e0e6489
[pkg/ottl]: Add Sort converter (#34283)
kaisecheng Sep 5, 2024
6e5fd62
[chore] update module github.com/google/go-github/v63 to v64 (#35016)
adrielp Sep 5, 2024
2bc5b37
Update All github.com/aws packages (#34964)
renovate[bot] Sep 5, 2024
0055e6e
[chore]: enable error-is-as rule from testifylint (#34995)
mmorel-35 Sep 5, 2024
85a2f4f
fix(deps): update module github.com/rs/cors to v1.11.1 (#34969)
renovate[bot] Sep 5, 2024
eb329f4
Update module google.golang.org/api to v0.195.0 (#34977)
renovate[bot] Sep 5, 2024
0bcac4b
[extension/healthcheckv2] Update readme to reflect current status (#3…
mwear Sep 5, 2024
562c01d
[connector/servicegraph] Fix histogram metrics miss unit (#34511)
Frapschen Sep 5, 2024
6eaf284
OTel-Arrow exporter timeout propagation (#34733)
jmacd Sep 5, 2024
9ad4ae3
[receiver/skywalking] add receiver supports version (#34916)
JaredTan95 Sep 5, 2024
52b6807
[exporter/prometheusremotewriteexporter]chore: log warning about remo…
ArthurSens Sep 5, 2024
145145c
[chore]: enable expected-actual rule from testifylint (#34998)
mmorel-35 Sep 5, 2024
a733b01
[pkg/ottl] Add `Decode` function (#33942)
bacherfl Sep 5, 2024
97533e6
[chore] Fix Supervisor test on Windows (#35035)
evan-bradley Sep 5, 2024
93a6bd9
[chore] Share encoding overrides (#35029)
TylerHelmuth Sep 5, 2024
55482cb
[chore] update logging to debug exporter (#35030)
codeboten Sep 5, 2024
7ec6396
[receiver/googlecloudmonitoringreceiver] Transform GCP Timeseries Dat…
abhishek-at-cloudwerx Sep 5, 2024
e9b835f
[receiver/sqlserver] Emit correct database name resource attribute (#…
crobert-1 Sep 6, 2024
abb7604
[exporter/elasticsearch] Workaround TSDB array dimension limitation f…
carsonip Sep 6, 2024
d5595bb
[connector/servicegraph] Extract the `getDimensionValue` function as …
JaredTan95 Sep 6, 2024
f970421
Operators with silent errors will log errors as debug and won't retur…
SamerJ Sep 6, 2024
af21ce7
[internal/otelarrow] Fix test flake (for 34719) (#34889)
jmacd Sep 6, 2024
642cc35
[pkg/ottl] Change grammar to support expressing statements context vi…
edmocosta Sep 6, 2024
25cb194
[pkg/ottl] Add support for localized time parsing into the timeutils …
edmocosta Sep 6, 2024
95ff5f2
[processor/transform] introduce aggregate_on_attribute_value function…
odubajDT Sep 6, 2024
14aa57a
[chore] go version 1.22.6 -> 1.22.7 (#35059)
crobert-1 Sep 6, 2024
8c08765
Use OTel-Arrow v0.26.0 (#35057)
jmacd Sep 6, 2024
0e2bea5
fix(deps): update module github.com/tencentcloud/tencentcloud-sdk-go/…
renovate[bot] Sep 6, 2024
237a96b
Add admission_blocked span to otel-arrow admission controller (#35031)
jmacd Sep 9, 2024
be7ddc3
[chore] Bump github.com/opencontainers/runc from 1.1.13 to 1.1.14 in …
crobert-1 Sep 9, 2024
cc5889d
[receiver/kafkareceiver] Add encoding extensions support (#33888)
thmshmm Sep 9, 2024
45ffc03
[chore] [receiver/datadog] Add support for Service Checks (#34474)
alexgreenbank Sep 9, 2024
5e26464
[receiver/datadog] Add support for sketches (#34662)
carrieedwards Sep 9, 2024
f81fd7a
Marking instanaexporter module as deprecated (#35052)
Frapschen Sep 9, 2024
248b8ec
[chore] Fix flakiness of tests on Windows requiring ports in the dyna…
pjanotti Sep 9, 2024
0ceefaa
[exporter/datadog] Use correct hostname for logs in logs agent pipeli…
liustanley Sep 9, 2024
b3e952f
[exporter/datadogexporter] Add support for custom log source (#35051)
mackjmr Sep 9, 2024
bd66ceb
[chore] Update test expectation to include processorhelper metrics (#…
djaglowski Sep 9, 2024
71ce97e
[chore] Fixing Prom Compliance tests (#35071)
MovieStoreGuy Sep 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[pkg/ottl] Remove tracing from OTTL (open-telemetry#34910)
**Description:** 
Reverts
open-telemetry#33508
since it was causing performance issues

**Link to tracking Issue:** <Issue number if applicable>
Reopens
open-telemetry#33433
Related to
open-telemetry/opentelemetry-collector#10858
Closes
open-telemetry#34890
  • Loading branch information
TylerHelmuth committed Aug 29, 2024
commit 240ff76cb71917177fc63d79b794c3f80b255b17
27 changes: 27 additions & 0 deletions .chloggen/ottl-remove-tracing.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: breaking

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: ottl

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Remove tracing from OTTL due to performance concerns

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [34910]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:

# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: []
18 changes: 0 additions & 18 deletions pkg/ottl/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,21 +151,3 @@ service:
2024-05-29T16:38:09.600-0600 debug ottl@v0.101.0/parser.go:268 TransformContext after statement execution {"kind": "processor", "name": "transform", "pipeline": "logs", "statement": "set(instrumentation_scope.attributes[\"test\"], [\"pass\"])", "condition matched": true, "TransformContext": {"resource": {"attributes": {"test": "pass"}, "dropped_attribute_count": 0}, "scope": {"attributes": {"test": ["pass"]}, "dropped_attribute_count": 0, "name": "", "version": ""}, "log_record": {"attributes": {"log.file.name": "test.log"}, "body": "test", "dropped_attribute_count": 0, "flags": 0, "observed_time_unix_nano": 1717022289500721000, "severity_number": 0, "severity_text": "", "span_id": "", "time_unix_nano": 0, "trace_id": ""}, "cache": {}}}
2024-05-29T16:38:09.601-0600 debug ottl@v0.101.0/parser.go:268 TransformContext after statement execution {"kind": "processor", "name": "transform", "pipeline": "logs", "statement": "set(attributes[\"test\"], true)", "condition matched": true, "TransformContext": {"resource": {"attributes": {"test": "pass"}, "dropped_attribute_count": 0}, "scope": {"attributes": {"test": ["pass"]}, "dropped_attribute_count": 0, "name": "", "version": ""}, "log_record": {"attributes": {"log.file.name": "test.log", "test": true}, "body": "test", "dropped_attribute_count": 0, "flags": 0, "observed_time_unix_nano": 1717022289500721000, "severity_number": 0, "severity_text": "", "span_id": "", "time_unix_nano": 0, "trace_id": ""}, "cache": {}}}
```

If configured to do so, the collector also emits traces for the execution of OTTL statement sequences.
These traces contain spans for the execution of each statement, including the statement itself and whether it has
been applied or not. To make use of this, enable the self monitoring of the collector by setting the
`--feature-gates=telemetry.useOtelWithSDKConfigurationForInternalTelemetry` flag, and using the following configuration
to export the traces to e.g. an OTLP API endpoint:

```yaml
service:
telemetry:
traces:
processors:
- batch:
exporter:
otlp:
protocol: http/protobuf
endpoint: ${env:OTLP_ENDPOINT}/v1/traces
```
6 changes: 3 additions & 3 deletions pkg/ottl/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,6 @@ require (
go.opentelemetry.io/collector/component v0.108.1
go.opentelemetry.io/collector/pdata v1.14.1
go.opentelemetry.io/collector/semconv v0.108.1
go.opentelemetry.io/otel v1.29.0
go.opentelemetry.io/otel/sdk v1.29.0
go.opentelemetry.io/otel/trace v1.29.0
go.uber.org/goleak v1.3.0
go.uber.org/zap v1.27.0
Expand Down Expand Up @@ -45,11 +43,13 @@ require (
github.com/prometheus/common v0.55.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
go.opentelemetry.io/collector/config/configtelemetry v0.108.1 // indirect
go.opentelemetry.io/otel v1.29.0 // indirect
go.opentelemetry.io/otel/exporters/prometheus v0.50.0 // indirect
go.opentelemetry.io/otel/metric v1.29.0 // indirect
go.opentelemetry.io/otel/sdk v1.28.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.28.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/sys v0.24.0 // indirect
golang.org/x/sys v0.23.0 // indirect
golang.org/x/text v0.17.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240701130421-f6361c86f094 // indirect
google.golang.org/grpc v1.65.0 // indirect
Expand Down
8 changes: 4 additions & 4 deletions pkg/ottl/go.sum

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

64 changes: 4 additions & 60 deletions pkg/ottl/parser.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,9 @@ import (

"github.com/alecthomas/participle/v2"
"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/codes"
"go.opentelemetry.io/otel/trace"
"go.opentelemetry.io/otel/trace/noop"
"go.uber.org/zap"
)

const (
logAttributeTraceID = "trace_id"
logAttributeSpanID = "span_id"
)

// Statement holds a top level Statement for processing telemetry data. A Statement is a combination of a function
// invocation and the boolean expression to match telemetry for invoking the function.
type Statement[K any] struct {
Expand Down Expand Up @@ -240,7 +231,6 @@ type StatementSequence[K any] struct {
statements []*Statement[K]
errorMode ErrorMode
telemetrySettings component.TelemetrySettings
tracer trace.Tracer
}

type StatementSequenceOption[K any] func(*StatementSequence[K])
Expand All @@ -260,10 +250,6 @@ func NewStatementSequence[K any](statements []*Statement[K], telemetrySettings c
statements: statements,
errorMode: PropagateError,
telemetrySettings: telemetrySettings,
tracer: &noop.Tracer{},
}
if telemetrySettings.TracerProvider != nil {
s.tracer = telemetrySettings.TracerProvider.Tracer("ottl")
}
for _, op := range options {
op(&s)
Expand All @@ -276,62 +262,20 @@ func NewStatementSequence[K any](statements []*Statement[K], telemetrySettings c
// When the ErrorMode of the StatementSequence is `ignore`, errors are logged and execution continues to the next statement.
// When the ErrorMode of the StatementSequence is `silent`, errors are not logged and execution continues to the next statement.
func (s *StatementSequence[K]) Execute(ctx context.Context, tCtx K) error {
ctx, sequenceSpan := s.tracer.Start(ctx, "ottl/StatementSequenceExecution")
defer sequenceSpan.End()
s.telemetrySettings.Logger.Debug(
"initial TransformContext",
zap.Any("TransformContext", tCtx),
zap.String(logAttributeTraceID, sequenceSpan.SpanContext().TraceID().String()),
zap.String(logAttributeSpanID, sequenceSpan.SpanContext().SpanID().String()),
)
s.telemetrySettings.Logger.Debug("initial TransformContext", zap.Any("TransformContext", tCtx))
for _, statement := range s.statements {
statementCtx, statementSpan := s.tracer.Start(ctx, "ottl/StatementExecution")
statementSpan.SetAttributes(
attribute.KeyValue{
Key: "statement",
Value: attribute.StringValue(statement.origText),
},
)
_, condition, err := statement.Execute(statementCtx, tCtx)
statementSpan.SetAttributes(
attribute.KeyValue{
Key: "condition.matched",
Value: attribute.BoolValue(condition),
},
)
s.telemetrySettings.Logger.Debug(
"TransformContext after statement execution",
zap.String("statement", statement.origText),
zap.Bool("condition matched", condition),
zap.Any("TransformContext", tCtx),
zap.String(logAttributeTraceID, statementSpan.SpanContext().TraceID().String()),
zap.String(logAttributeSpanID, statementSpan.SpanContext().SpanID().String()),
)
_, condition, err := statement.Execute(ctx, tCtx)
s.telemetrySettings.Logger.Debug("TransformContext after statement execution", zap.String("statement", statement.origText), zap.Bool("condition matched", condition), zap.Any("TransformContext", tCtx))
if err != nil {
statementSpan.RecordError(err)
errMsg := fmt.Sprintf("failed to execute statement '%s': %v", statement.origText, err)
statementSpan.SetStatus(codes.Error, errMsg)
if s.errorMode == PropagateError {
sequenceSpan.SetStatus(codes.Error, errMsg)
statementSpan.End()
err = fmt.Errorf("failed to execute statement: %v, %w", statement.origText, err)
return err
}
if s.errorMode == IgnoreError {
s.telemetrySettings.Logger.Warn(
"failed to execute statement",
zap.Error(err),
zap.String("statement", statement.origText),
zap.String(logAttributeTraceID, statementSpan.SpanContext().TraceID().String()),
zap.String(logAttributeSpanID, statementSpan.SpanContext().SpanID().String()),
)
s.telemetrySettings.Logger.Warn("failed to execute statement", zap.Error(err), zap.String("statement", statement.origText))
}
} else {
statementSpan.SetStatus(codes.Ok, "statement executed successfully")
}
statementSpan.End()
}
sequenceSpan.SetStatus(codes.Ok, "statement sequence executed successfully")
return nil
}

Expand Down
Loading