Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[extension/encoding] support skywalkingencoding extension #36968

Open
wants to merge 36 commits into
base: main
Choose a base branch
from

Conversation

JaredTan95
Copy link
Member

@JaredTan95 JaredTan95 commented Dec 27, 2024

Description

The kafkareceiver supports jaeger and zipkin protocol traces, and can also receive skywalkng traces via skywalkingreceiver.

From the practical use of skywalking, we often use kafka to host large-scale traces data. However, kafka receiver does not support skywalking encoding currently.

with this PR, we can fectch skywalking traces and parse it into otlp traces:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
  kafka:
    encoding: skywalking_encoding
    topic: skywalking-segments
    brokers:
      - "localhost:9092"
    resolve_canonical_bootstrap_servers_only: true
    client_id: otel-collector
    group_id: otel-collector
    metadata:
      retry:
        max: 10
        backoff: 5s

exporters:
  debug:

processors:
  batch:

extensions:
  health_check:
  pprof:
    endpoint: :1888
  zpages:
    endpoint: :55679
  skywalking_encoding:

service:
  extensions: [pprof, zpages, skywalking_encoding, health_check]
  pipelines:
    traces:
      receivers: [kafka]
      processors: [batch]
      exporters: [debug]

Link to tracking issue

Fixes #28693

Testing

Documentation

@JaredTan95 JaredTan95 changed the title [extension/encoding] suuport Skywalkingencoding extension [extension/encoding] support skywalkingencoding extension Dec 27, 2024
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
@JaredTan95 JaredTan95 force-pushed the skywalkingencoding_extension branch from 8d1b0e5 to b01b7a1 Compare December 27, 2024 06:28
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
.github/CODEOWNERS Outdated Show resolved Hide resolved
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
@JaredTan95
Copy link
Member Author

Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Copy link
Contributor

@fatsheep9146 fatsheep9146 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know how to fix the check-owners failed lint?

@JaredTan95
Copy link
Member Author

Do you know how to fix the check-owners failed lint?

logs show that I need to do some changes:

- extension/encoding/                               @open-telemetry/collector-contrib-approvers @atoulme @dao-jun @dmitryax @MovieStoreGuy @VihasMakwana
+ extension/encoding/                               @open-telemetry/collector-contrib-approvers @atoulme @dao-jun @dmitryax @MovieStoreGuy @VihasMakwana @JaredTan95 

in .github/CODEOWNERS file, but @atoulme told me do it in separate PR, so, I'm confused~

@crobert-1
Copy link
Member

Do you know how to fix the check-owners failed lint?

logs show that I need to do some changes:

- extension/encoding/                               @open-telemetry/collector-contrib-approvers @atoulme @dao-jun @dmitryax @MovieStoreGuy @VihasMakwana
+ extension/encoding/                               @open-telemetry/collector-contrib-approvers @atoulme @dao-jun @dmitryax @MovieStoreGuy @VihasMakwana @JaredTan95 

in .github/CODEOWNERS file, but @atoulme told me do it in separate PR, so, I'm confused~

This is the current action failure output:

Generated code is out of date, please apply this diff and commit the changes in this PR.
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
index 004df01..3baf60[9](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/12665959423/job/35296498228?pr=36968#step:5:10) 100644
--- a/.github/CODEOWNERS
+++ b/.github/CODEOWNERS
@@ -95,6 +95,7 @@ extension/encoding/avrologencodingextension/      @open-telemetry/collector-cont
 extension/encoding/jaegerencodingextension/       @open-telemetry/collector-contrib-approvers @MovieStoreGuy @atoulme
 extension/encoding/jsonlogencodingextension/      @open-telemetry/collector-contrib-approvers @VihasMakwana @atoulme
 extension/encoding/otlpencodingextension/         @open-telemetry/collector-contrib-approvers @dao-jun @VihasMakwana
+extension/encoding/skywalkingencodingextension/   @open-telemetry/collector-contrib-approvers @JaredTan95
 extension/encoding/textencodingextension/         @open-telemetry/collector-contrib-approvers @MovieStoreGuy @atoulme
 extension/encoding/zipkinencodingextension/       @open-telemetry/collector-contrib-approvers @MovieStoreGuy @dao-jun
 extension/googleclientauthextension/              @open-telemetry/collector-contrib-approvers @dashpole @aabmass @jsuereth @punya @psx95
Error: Process completed with exit code 1.

It's simply missing a line for the new extension you're adding, it's no longer complaining about the extension/encoding entry 👍

Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
@JaredTan95 JaredTan95 added the ready to merge Code review completed; ready to merge by maintainers label Jan 10, 2025
Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Copy link

codecov bot commented Jan 13, 2025

Codecov Report

Attention: Patch coverage is 73.07692% with 7 lines in your changes missing coverage. Please review.

Project coverage is 79.59%. Comparing base (12551d3) to head (0849205).

Files with missing lines Patch % Lines
.../encoding/skywalkingencodingextension/extension.go 33.33% 4 Missing ⚠️
...encoding/skywalkingencodingextension/skywalking.go 57.14% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #36968      +/-   ##
==========================================
- Coverage   79.59%   79.59%   -0.01%     
==========================================
  Files        2252     2255       +3     
  Lines      211914   211940      +26     
==========================================
+ Hits       168682   168699      +17     
- Misses      37562    37571       +9     
  Partials     5670     5670              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cmd/otelcontribcol otelcontribcol command extension/encoding ready to merge Code review completed; ready to merge by maintainers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[receiver/kafka] support receive skywalking traces from kafka
6 participants