Restructure buildCWMetric logic #1
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kohrapha
force-pushed
the
kohrapha/datapoints
branch
from
f7fccca
to
eecc7b0
Compare
|
Adding @mxiamxia, @hdj630, @shaochengwang as reviewers. |
mxiamxia
reviewed
Oct 19, 2020
|
Overall, LGTM. Can you please help pull the latest 2 PRs we made recently from upstream and update it in your PR? If possible, could you pls help finish the missing test case for TestGetMesurement()? |
|
@mxiamxia Merged upstream code and implemented test cases as mentioned. |
kohrapha
force-pushed
the
kohrapha/datapoints
branch
from
9bb3d3f
to
1c9a56b
Compare
mxiamxia
approved these changes
Oct 21, 2020
|
Can you pls ping Shaocheng to see if he is also fine this PR? I think we can contribute this enhancement to upstream. |
kohrapha
added a commit
that referenced
this pull request
Oct 23, 2020
* Restructure buildCWMetric logic (#1) * Restructure code to remove duplicated logic * Update format * Improve function and variable names * Extract logic for dimension creation and add test * Implement minor fixes * Remove changes to go.sum * Implement tests for getCWMetrics * Implement tests for buildCWMetric * Format metric_translator_test.go * Run with gofmt -s * Disregard ordering of dimensions in test case * Perform dimension equality checking as a helper function
kohrapha
added a commit
that referenced
this pull request
Oct 26, 2020
* Add API key validation (#1216) Adds API key validation to the Datadog metrics exporter. When created, the Datadog metrics exporter now sends a requests to the `/api/v1/validate` endpoint of the Datadog backend to check that the configured API key is valid. If it's not, a warning log is emitted. Tests were amended to take into account that validation call. Test utils were added to mock an HTTP server that performs validation. * sapmexporter: make span source attribute and destination dimension names configurable (#1286) If dimension names are being translated in the signalfxexporter then the map values should be set to the signalfx names. Ideally we can sync to OT dimension names with translation being done on the backend (the default). * Update README (#1294) * Release v0.13.0 (#1295) * Remove duplicate definition of cloud providers with core conventions (#1288) * Remove duplicate definition of cloud providers Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com> * Fix more duplicate usage of the cloud providers semconv Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com> * Splunkhec receiver metrics (#1276) Adds the ability for Splunk HEC to ingest metrics. This is a follow up to #1268 which adds the ability to ingest logs. * Add jpkrohling as an approver (#1296) * Remove pjanotti from maintainers (#1300) * Auto assign approver and maintainers to PRs (#1301) Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com> * Add codeowners to ensure components are assigned to the appropriate reviewers (#1304) This is the initial list extracted from README. * Moved the groupbytrace processor to contrib (#1179) Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de> Co-authored-by: Bogdan Drutu <bogdandrutu@gmail.com> * Add codeowners for interanl components (#1307) Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com> * Small fixes to CODEOWNERS (#1312) Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de> **Description:** This PR changes the CODEOWNERS in a couple of aspects: 1. Fixed the order of the directories, so that 'internal' comes after 'extension' 1. Fixed the name of a few components 1. Added missing components and directories Verified with: ``` for component in exporter extension processor receiver; do ls ${component}/ -1 > /tmp/${component}.txt grep ${component} .github/CODEOWNERS | awk -F\/ '{print $2}' > /tmp/${component}-codeowners.txt diff /tmp/${component}.txt /tmp/${component}-codeowners.txt done ``` Result of the script before this PR: ```diff 11d10 < loadbalancingexporter 2c2 < jmxmetricsextension --- > jmxmetrics 1d0 < groupbytraceprocessor 5c4 < routingprocessor --- > routing ``` * Update collector version in groupbytraceprocessor (#1309) Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com> * Update dependabot to ensure all projects are added (#1303) * Update dependabot to ensure all projects are added Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com> * Update dependabot.yml * Do not run tests/lint/etc for all component tags (e.g. tag testbed/v0.13.0) (#1298) Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com> * tests: increase TestTrace10kSPS memory limits (#1314) * Bump k8s.io/client-go in /receiver/k8sclusterreceiver (#1323) Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.19.2 to 0.19.3. - [Release notes](https://github.com/kubernetes/client-go/releases) - [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md) - [Commits](kubernetes/client-go@v0.19.2...v0.19.3) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/aliyun/aliyun-log-go-sdk (#1321) Bumps [github.com/aliyun/aliyun-log-go-sdk](https://github.com/aliyun/aliyun-log-go-sdk) from 0.1.13 to 0.1.14. - [Release notes](https://github.com/aliyun/aliyun-log-go-sdk/releases) - [Commits](aliyun/aliyun-log-go-sdk@v0.1.13...v0.1.14) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump go.opencensus.io in /processor/groupbytraceprocessor (#1320) Bumps [go.opencensus.io](https://github.com/census-instrumentation/opencensus-go) from 0.22.4 to 0.22.5. - [Release notes](https://github.com/census-instrumentation/opencensus-go/releases) - [Commits](census-instrumentation/opencensus-go@v0.22.4...v0.22.5) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump k8s.io/client-go from 0.19.2 to 0.19.3 in /internal/k8sconfig (#1318) Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.19.2 to 0.19.3. - [Release notes](https://github.com/kubernetes/client-go/releases) - [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md) - [Commits](kubernetes/client-go@v0.19.2...v0.19.3) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add more @elastic folks to codeowners (#1313) In answer to open-telemetry#1304 (comment) add two more codeowners for the Elastic exporter. * Add contrib approvers as owners to all the components. (#1325) Without this change if there is a listed owner with write permission in the component owners list, the contrib approvers will lose their power see #1316. Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com> * Bump github.com/aws/aws-sdk-go in /internal/awsxray (#1316) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.35.9 to 1.35.10. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md) - [Commits](aws/aws-sdk-go@v1.35.9...v1.35.10) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/aws/aws-sdk-go in /internal/awsxray/testdata/sampleapp (#1317) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.35.9 to 1.35.10. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md) - [Commits](aws/aws-sdk-go@v1.35.9...v1.35.10) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Clarify PR reviewing and facilitating (#1315) We recently introduced the automatic assignments of PRs to reviewers and to facilitators. This change explains the process. * Handle nil references from the kubelet API (#1326) * Update to latest collector, update deprecated calls (#1308) Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com> * signalfx Receiver: Better Pipeline Error Handling (#1329) If logs aren't configured and events are sent, return a clear error response instead of panicing. Vice-versa for metrics. * [datadogexporter] Improve hostname resolution (#1285) Improve system hostname detection for the Datadog exporter. This PR: - Moves config and host code to their own packages to avoid dependency cycles - Adds hostname validation - Adds fully qualified domain name hostname resolution on some platforms - Adds support for caching hostname Added unit tests, tested on an end to end test environment with the component activated. Documentation was added to all public functions. * Temporarily remove dmitryax from PR facilitators (#1330) dmitryax will be unavailable for a while, removing him from the list of PR facilitators. * Update otel collector, fix breaking change for renaming TracesConsumer (#1328) * Update otel collector, fix breaking change for renaming TracesConsumer Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com> * More fixes of usages Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com> * Add batchpertrace library (#1257) Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de> Adds a library that will split the incoming batch into several batches, one per trace. **Link to tracking Issue:** Closes #1235. * Fix the link to the release notes (#1327) * Datadog trace flushing/export (#1266) This PR adds flushing+export of traces and trace-related statistics to the `datadogexporter`, as well as some very minor changes to the translation of internal traces into Datadog format. It represents the second of two PRs for the work contained in open-telemetry#1203. It builds on top of current master branch, and follows up to the work [done here](open-telemetry#1208). The final PR explicitly enabling The Datadog exporter will follow, and will allow users to export traces to Datadog's API Intake. This PR Split was requested by @tigrannajaryan and hopefully should make code review a bit less cumbersome. However if there are any questions or changes to the PR format needed, please let me know. **Testing:** There are unit tests for the different methods and helper methods within the export code. **Documentation:** Appropriate usage, including best practices for which processors to also enable, has been documented in the README, `testdata/config.yaml` and `example/config.yaml` samples. **Notes**: This PR includes a trace exporter for non-windows environments only (metrics are fine in windows, just traces that are the issue), due to reasons explained in this pr open-telemetry#1274 . tl;dr is our trace export code for windows env would rely on CGO for now, which is not permitted in the collector * Logzio exporter impl (#1161) Added a logz.io traces exporter **Link to tracking Issue**: #686 **Testing**: Added test for each of the components in the new exporter **Documentation**: Added a readme specifying how to use the exporter and its parameters with an example. * Add the notion of unstable components and unstable executable (#1299) The list of experimental components is defined in unstable_components_enabled.go. These components are only enabled when enable_unstable build tag is defined. We define this tag and produce an executable named otelcontribcol_unstable_$(GOOS)_$(GOARCH)$(EXTENSION) when `make otelcontribcol-unstable` is invoked. For now the new executable is not used anywhere. Next I will look into modifying the testbed to call the new unstable executable for certain tests. To verify that the unstable build functionality is enabled I added stanzareceiver to the list of unstable components and manually verified that it is indeed enabled in the unstable executable but is not available in the regular otelcontribcol executable. Contributes to open-telemetry#873 * JMX Metric Extension: Initial implementation (#1182) * Add JMX Metric Extension implementation * rename package to jmxmetricextension * jmxmetricextension s/metrics/metric * jmx metrics: fix prometheus typo * jmx metrics: capitalize acronyms * jmx metrics: clarify interval * Enable stale PR action (#1341) To help reviewers and authors remember to make progress on PR this action will mark PRs as stale after inactivity of 7 days and will close the PR after 7 more days of inactivity. * [datadogexporter] Enable traces on Windows (#1340) * Re-enable traces code on Windows Use a custom-made version of the Datadog Agent repository that greatly reduces the number of dependencies needed and removes the osext one that depends on CGo * Address linter issue * Empty commit to retrigger CI * Build traces flush/export code on Windows * Add kind type to root span to fix the empty parentID problem (#1338) * Add kind type to root span to fix the empty parentID problem * Set kind type for root span in Xray receiver * Update receiver/awsxrayreceiver/internal/translator/translator.go Co-authored-by: Anuraag Agrawal <anuraaga@gmail.com> Co-authored-by: Bogdan Drutu <lazy@splunk.com> Co-authored-by: Anuraag Agrawal <anuraaga@gmail.com> * [awsecscontainermetrics] receiver- Update README (#1358) * [awsecscontainermetrics] receiver- Update README Signed-off-by: Rayhan Hossain <hossain.rayhan@outlook.com> * Use full form of metric units Signed-off-by: Rayhan Hossain <hossain.rayhan@outlook.com> * Add timer support for statsD receiver (#1335) * [datadogexporter] Add Datadog exporter to the otelcontribcol binary (#1352) * Add datadogexporter to the binary * Disable environment variables They don't work; we will revisit it in the future * [datadogexporter] Update go-datadog-api.v2 dependency to v2.30.0 (#1365) * [signalfx_correlation] Add signalfx_correlation exporter skeleton (#1332) * [signalfx_correlation] Add signalfx_correlation exporter skeleton This is for moving the correlation out of sapmexporter into a dedicated exporter so that the correlation can be used even when sapm isn't (for example, on an agent that is exporting in otlp to a gateway instead of sapm.) * fix readme * [awsemfexporter] Restructure Metric Translator Logic (#1353) * Restructure buildCWMetric logic (#1) * Restructure code to remove duplicated logic * Update format * Improve function and variable names * Extract logic for dimension creation and add test * Implement minor fixes * Remove changes to go.sum * Implement tests for getCWMetrics * Implement tests for buildCWMetric * Format metric_translator_test.go * Run with gofmt -s * Disregard ordering of dimensions in test case * Perform dimension equality checking as a helper function * Setting the tlsconfig InsecureSkipVerify using NoVerifySSL (#1350) Co-authored-by: Kylian Serrania <kylian.serrania@datadoghq.com> Co-authored-by: Jay Camp <jay.r.camp@gmail.com> Co-authored-by: Steve Flanders <sflanders@splunk.com> Co-authored-by: Jeff Cheng <jcheng@signalfx.com> Co-authored-by: Bogdan Drutu <bogdandrutu@gmail.com> Co-authored-by: Antoine Toulme <atoulme@users.noreply.github.com> Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Co-authored-by: Paulo Janotti <pjanotti@splunk.com> Co-authored-by: Juraci Paixão Kröhling <juraci@kroehling.de> Co-authored-by: Eric Mustin <mustin.eric@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Andrew Wilkins <axw@elastic.co> Co-authored-by: Pablo Collins <pablo.collins@gmail.com> Co-authored-by: Ben Keith <benkeith@splunk.com> Co-authored-by: Pablo Baeyens <pablo.baeyens@datadoghq.com> Co-authored-by: Yogev Mets <yogev.metzuyanim@logz.io> Co-authored-by: Ryan Fitzpatrick <rmfitzpatrick@users.noreply.github.com> Co-authored-by: John <59711343+JohnWu20@users.noreply.github.com> Co-authored-by: Bogdan Drutu <lazy@splunk.com> Co-authored-by: Anuraag Agrawal <anuraaga@gmail.com> Co-authored-by: Rayhan Hossain (Mukla.C) <hossain.rayhan@outlook.com> Co-authored-by: Gavin Zhang (Kunyuan Zhang) <31523962+gavindoudou@users.noreply.github.com> Co-authored-by: shreyas Darwhatkar <67069455+darwhs@users.noreply.github.com>
kohrapha
added a commit
that referenced
this pull request
Oct 29, 2020
* Restructure buildCWMetric logic (#1) * Restructure code to remove duplicated logic * Update format * Improve function and variable names * Extract logic for dimension creation and add test * Implement minor fixes * Remove changes to go.sum * Implement tests for getCWMetrics * Implement tests for buildCWMetric * Format metric_translator_test.go * Run with gofmt -s * Disregard ordering of dimensions in test case * Perform dimension equality checking as a helper function
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR restructures the code for building CW Metrics from
pdata.Metricto remove duplicated logic. To do this, we had to define an interfaceDataPointsthat wrapspdata.(Int|Double|DoubleHistogram)DataPointSliceand an interfaceDataPointthat wrapspdata.(Int|Double|DoubleHistogram)DataPoint.However, since the
At(i)method forDataPointsrequires returning aDataPointwhich is not the same as whatpdata.(Int|Double|DoubleHistogram)DataPointreturns, we have to define a light conversion struct that provides the correct method signature.Performance
I ran the benchmarks implemented in #5 and compared the performance of this new code change against the old code, and got the following results:
We note that not only is there no negative impact on the performance, we see a minor improvement across the board. This is great news because, not only did we improve code readability for developers, performance was also not impacted.