Cloud Monitoring (Stackdriver) receiver

[ocervell]

Is your feature request related to a problem? Please describe.
Query metrics from Cloud Monitoring (ex-Stackdriver) and forward push them into any export destination that OpenTelemetry support.

Describe the solution you'd like
Implementation of a Cloud Monitoring receiver scraping Cloud Monitoring API every X seconds.

Describe alternatives you've considered
Exporting metrics included in Cloud Monitoring product (not planned).

[weyert]

Yes, this would be nice to have

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[weyert]

Yes, still relevant

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[jpkrohling]

On behalf of the OpenTelemetry Squad at Grafana Labs, I would like to volunteer as a sponsor for this component. In addition to the component's author, the code owners (as sponsors) will be:

• @jpkrohling
• @mar4uk

While we would very much prefer the affected vendor to support this receiver, we understand they might not be interested in translating their proprietary formats into OTLP, only the other way around.

We believe that the community wants this receiver, and if community members are willing to contribute code for this, we are willing to support the community.

[dashpole]

While we would very much prefer the affected vendor to support this receiver, we understand they might not be interested in translating their proprietary formats into OTLP, only the other way around.

Speaking for Google Cloud Monitoring, I would be happy to sponsor this receiver. We fully support customers’ decisions to use other monitoring solutions, and are willing to help with the design and review for the component to ensure it works well. It won’t be an officially-supported GCP product (hopefully I'll have more to share in the future), but we will do what we can to enable the community.

A few pointers, if anyone is interested in picking this up (the top section of this GCM export architecture may also be helpful):

• Use projects.metricDescriptors.list to get metadata (descriptions, units, etc.) about metrics. Do this once at the beginning and cache it, or call again as needed.
• Use projects.timeseries.list (go client) to query the API. * The configuration of the component should probably mirror the query parameters
• Recommended query interval: 5 minutes
• Convert the returned TimeSeries to pdata using the reverse of the conversion from our exporter.

[fatsheep9146]

@ocervell do you have time to contribute this?

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.

[weyert]

Yes, still relevant

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.

[dashpole]

I've spoken with someone who is interested in implementing this.

[datsabk]

@dashpole - Thanks for bringing this thread alive. I'd like to start working on this receiver and would love to collaborate with the sponsors and any others contributors alongside.

I understand that in the past things might have changed for how GCP - We would prefer to call the component gcpcloudmetricsreceiver. If I have the go-ahead from the maintainers and sponsors herewith, I'd like to proceed.

[dashpole]

Other google-related components in contrib use "google" or "googlecloud" rather than "gcp", and include the name of the product if needed (googlecloudexporter, googlecloudpubsubexporter, googlemanagedprometheusexporter, googlecloudpubsubreceiver, googlecloudspannerreceiver), so please try to stay consistent with existing names if possible. googlecloudreceiver or googlecloudmonitoringreceiver are probably the closest matches.

[datsabk]

googlecloudmetricsreceiver seems more relevant as Monitoring is a broader term in general. Let me know your thoughts.

[dashpole]

The reason I suggested "monitoring" is because the product name is "Google Cloud Monitoring". E.g. the receiver for "Azure Monitor" is named azuremonitorreceiver. I'm asking around to see if there are any naming conventions that can help us here.

[dashpole]

It doesn't seem like there are any existing component naming conventions.

[datsabk]

Ack @dashpole - We'll go with googlecloudmonitoringreceiver.

[dashpole]

See https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#adding-new-components for the new component process. For the first PR, see the "First PR should include the overall structure of the new component" section.

[dashpole]

Closing in favor of #33762