Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/cloudflare] Create logs receiver for Cloudflare LogPush API #19379

Merged
merged 21 commits into from
Mar 13, 2023

Conversation

dehaansa
Copy link
Contributor

@dehaansa dehaansa commented Mar 7, 2023

Description: This pr implements a Cloudflare receiver that will receive logs sent from the LogPush API to allow users to collect logs from the Cloudflare logs aggregation system.

Link to tracking Issue: #19201

Testing: Unit & mock integration tests were created. Additionally, a LogPush job was created in my Cloudflare environment, and logs were written to file output locally to confirm expectations.

image

Documentation: The readme is quite thorough in addressing how to use the receiver and calls out potential pitfalls.

@dehaansa dehaansa requested review from a team and mx-psi March 7, 2023 21:06
@github-actions github-actions bot added the cmd/otelcontribcol otelcontribcol command label Mar 7, 2023
@runforesight
Copy link

runforesight bot commented Mar 7, 2023

Foresight Summary

    
Major Impacts

build-and-test duration(43 minutes 55 seconds) has decreased 21 minutes 15 seconds compared to main branch avg(1 hour 5 minutes 10 seconds).
View More Details

⭕  build-and-test-windows workflow has finished in 6 seconds (43 minutes 5 seconds less than main branch avg.) and finished at 13th Mar, 2023.


Job Failed Steps Tests
windows-unittest-matrix -     🔗  N/A See Details
windows-unittest -     🔗  N/A See Details

✅  telemetrygen workflow has finished in 1 minute 14 seconds (52 seconds less than main branch avg.) and finished at 13th Mar, 2023.


Job Failed Steps Tests
build-dev -     🔗  N/A See Details
publish-latest -     🔗  N/A See Details
publish-stable -     🔗  N/A See Details

✅  check-links workflow has finished in 1 minute 40 seconds and finished at 13th Mar, 2023.


Job Failed Steps Tests
changed files -     🔗  N/A See Details
check-links -     🔗  N/A See Details

✅  changelog workflow has finished in 2 minutes 20 seconds and finished at 13th Mar, 2023.


Job Failed Steps Tests
changelog -     🔗  N/A See Details

✅  build-and-test workflow has finished in 43 minutes 55 seconds (21 minutes 15 seconds less than main branch avg.) and finished at 13th Mar, 2023.


Job Failed Steps Tests
unittest-matrix (1.19, connector) -     🔗  ✅ 113  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, connector) -     🔗  ✅ 113  ❌ 0  ⏭ 0    🔗 See Details
correctness-metrics -     🔗  ✅ 2  ❌ 0  ⏭ 0    🔗 See Details
correctness-traces -     🔗  ✅ 17  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, extension) -     🔗  ✅ 538  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, extension) -     🔗  ✅ 538  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, internal) -     🔗  ✅ 583  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, internal) -     🔗  ✅ 583  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, processor) -     🔗  ✅ 1545  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, processor) -     🔗  ✅ 1545  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, receiver-0) -     🔗  ✅ 2607  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, receiver-0) -     🔗  ✅ 2607  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, exporter) -     🔗  ✅ 2488  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, receiver-1) -     🔗  ✅ 1940  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, receiver-1) -     🔗  ✅ 1940  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, exporter) -     🔗  ✅ 2488  ❌ 0  ⏭ 0    🔗 See Details
integration-tests -     🔗  ✅ 55  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, other) -     🔗  ✅ 4728  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, other) -     🔗  ✅ 4728  ❌ 0  ⏭ 0    🔗 See Details
setup-environment -     🔗  N/A See Details
check-collector-module-version -     🔗  N/A See Details
check-codeowners -     🔗  N/A See Details
checks -     🔗  N/A See Details
lint-matrix (receiver-0) -     🔗  N/A See Details
lint-matrix (receiver-1) -     🔗  N/A See Details
lint-matrix (processor) -     🔗  N/A See Details
lint-matrix (exporter) -     🔗  N/A See Details
lint-matrix (extension) -     🔗  N/A See Details
lint-matrix (connector) -     🔗  N/A See Details
lint-matrix (internal) -     🔗  N/A See Details
lint-matrix (other) -     🔗  N/A See Details
build-examples -     🔗  N/A See Details
unittest (1.20) -     🔗  N/A See Details
unittest (1.19) -     🔗  N/A See Details
lint -     🔗  N/A See Details
cross-compile (darwin, amd64) -     🔗  N/A See Details
cross-compile (darwin, arm64) -     🔗  N/A See Details
cross-compile (linux, 386) -     🔗  N/A See Details
cross-compile (linux, amd64) -     🔗  N/A See Details
cross-compile (linux, arm) -     🔗  N/A See Details
cross-compile (linux, arm64) -     🔗  N/A See Details
cross-compile (linux, ppc64le) -     🔗  N/A See Details
cross-compile (windows, 386) -     🔗  N/A See Details
cross-compile (windows, amd64) -     🔗  N/A See Details
build-package (deb) -     🔗  N/A See Details
build-package (rpm) -     🔗  N/A See Details
windows-msi -     🔗  N/A See Details
publish-check -     🔗  N/A See Details
publish-dev -     🔗  N/A See Details
publish-stable -     🔗  N/A See Details

✅  prometheus-compliance-tests workflow has finished in 12 minutes 2 seconds (⚠️ 4 minutes 30 seconds more than main branch avg.) and finished at 13th Mar, 2023.


Job Failed Steps Tests
prometheus-compliance-tests -     🔗  ✅ 21  ❌ 0  ⏭ 0    🔗 See Details

✅  load-tests workflow has finished in 18 minutes 52 seconds (⚠️ 5 minutes 34 seconds more than main branch avg.) and finished at 13th Mar, 2023.


Job Failed Steps Tests
loadtest (TestTraceAttributesProcessor) -     🔗  ✅ 3  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestIdleMode) -     🔗  ✅ 1  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestMetric10kDPS|TestMetricsFromFile) -     🔗  ✅ 6  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestTraceNoBackend10kSPS|TestTrace1kSPSWithAttrs) -     🔗  ✅ 8  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestTraceBallast1kSPSWithAttrs|TestTraceBallast1kSPSAddAttrs) -     🔗  ✅ 10  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestMetricResourceProcessor|TestTrace10kSPS) -     🔗  ✅ 12  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestBallastMemory|TestLog10kDPS) -     🔗  ✅ 18  ❌ 0  ⏭ 0    🔗 See Details
setup-environment -     🔗  N/A See Details

✅  e2e-tests workflow has finished in 16 minutes 5 seconds and finished at 13th Mar, 2023.


Job Failed Steps Tests
kubernetes-test (v1.26.0) -     🔗  N/A See Details
kubernetes-test (v1.25.3) -     🔗  N/A See Details
kubernetes-test (v1.24.7) -     🔗  N/A See Details
kubernetes-test (v1.23.13) -     🔗  N/A See Details

🔎 See details on Foresight

*You can configure Foresight comments in your organization settings page.

@mx-psi mx-psi requested a review from djaglowski March 8, 2023 09:10
@mx-psi mx-psi assigned djaglowski and unassigned bogdandrutu Mar 8, 2023
@mx-psi mx-psi removed their request for review March 8, 2023 09:11
Copy link
Member

@djaglowski djaglowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks good. Just a bunch of minor cleanup suggestions.

receiver/cloudflarereceiver/config.go Outdated Show resolved Hide resolved
receiver/cloudflarereceiver/README.md Outdated Show resolved Hide resolved
receiver/cloudflarereceiver/logs.go Show resolved Hide resolved
receiver/cloudflarereceiver/logs.go Outdated Show resolved Hide resolved
receiver/cloudflarereceiver/logs.go Outdated Show resolved Hide resolved
receiver/cloudflarereceiver/logs.go Outdated Show resolved Hide resolved
receiver/cloudflarereceiver/logs.go Outdated Show resolved Hide resolved
receiver/cloudflarereceiver/config.go Outdated Show resolved Hide resolved
@djaglowski djaglowski merged commit b031bc7 into open-telemetry:main Mar 13, 2023
@dehaansa dehaansa deleted the cloudflare-logpush branch March 13, 2023 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cmd/otelcontribcol otelcontribcol command
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants