Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[chore] updating scrapers to use default method from scraperhelper #22138

Merged
merged 4 commits into from
May 23, 2023
Merged

[chore] updating scrapers to use default method from scraperhelper #22138

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d2ed9d9
Adding updates to scraper intervals for upcoming changes in upstream
MovieStoreGuy May 20, 2023
77ac225
Fixing up lint
MovieStoreGuy May 20, 2023
3794295
Using pull request id for changelog
MovieStoreGuy May 20, 2023
b070e52
Using components defaults where aplicable
MovieStoreGuy May 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .chloggen/msg_ensure-standardised-collector-intervals.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Use this changelog template to create an entry for release notes.
# If your change doesn't affect end users, such as a test fix or a tooling change,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: breaking
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still a breaking change?

If I understand correctly, the change in components' behavior is that they start scraping after 1 second and not after their default collection interval, is this right? In that case, I believe we should call it out in the release notes (using the subtext below?).

Copy link
Contributor Author

@MovieStoreGuy MovieStoreGuy May 23, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏽 , that makes sense but at this point this is a transparent change until the upstream is merged.

However, I think this missed the v0.78.0 release so we have time to fix this up.
I can correct it in the next release with more details.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, the change in components' behavior is that they start scraping after 1 second and not after their default collection interval, is this right?

This comment of mine is actually incorrect 🙂 Not until the core PR gets merged:

Looking at this PR again in detail, it wasn't (meant as) a breaking change, rather a refactoring. The only (accidental, I suppose) breaking change is that the Apache receiver's default collection interval has changed from 10s to 60s. This should probably be fixed in a future PR.


# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: receivers

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Updating receivers that run intervals to use standard interval by default

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [22138]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
12 changes: 2 additions & 10 deletions receiver/activedirectorydsreceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,13 @@
package activedirectorydsreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/activedirectorydsreceiver"

import (
"time"

"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/receiver"
"go.opentelemetry.io/collector/receiver/scraperhelper"

"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/activedirectorydsreceiver/internal/metadata"
)

const (
defaultCollectionInterval = 10 * time.Second
)

func NewFactory() receiver.Factory {
return receiver.NewFactory(
metadata.Type,
Expand All @@ -27,9 +21,7 @@ func NewFactory() receiver.Factory {

func createDefaultConfig() component.Config {
return &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: defaultCollectionInterval,
},
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
}
}
12 changes: 5 additions & 7 deletions receiver/aerospikereceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,12 +61,10 @@ func createMetricsReceiver(

func createDefaultConfig() component.Config {
return &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: time.Minute,
},
Endpoint: defaultEndpoint,
Timeout: defaultTimeout,
CollectClusterMetrics: defaultCollectClusterMetrics,
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
Endpoint: defaultEndpoint,
Timeout: defaultTimeout,
CollectClusterMetrics: defaultCollectClusterMetrics,
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
}
}
4 changes: 1 addition & 3 deletions receiver/apachereceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,7 @@ func NewFactory() receiver.Factory {

func createDefaultConfig() component.Config {
return &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: 10 * time.Second,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this was missed and effectively the Apache receiver's default collection interval has changed from 10s to 60s?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shit, I can revert in a follow up PR.

},
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
HTTPClientSettings: confighttp.HTTPClientSettings{
Endpoint: defaultEndpoint,
Timeout: 10 * time.Second,
Expand Down
5 changes: 1 addition & 4 deletions receiver/azuremonitorreceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ package azuremonitorreceiver // import "github.com/open-telemetry/opentelemetry-
import (
"context"
"errors"
"time"

"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/consumer"
Expand All @@ -28,9 +27,7 @@ func NewFactory() receiver.Factory {

func createDefaultConfig() component.Config {
return &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: 10 * time.Second,
},
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
CacheResources: 24 * 60 * 60,
CacheResourcesDefinitions: 24 * 60 * 60,
Expand Down
5 changes: 1 addition & 4 deletions receiver/azuremonitorreceiver/factory_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ package azuremonitorreceiver // import "github.com/open-telemetry/opentelemetry-
import (
"context"
"testing"
"time"

"github.com/stretchr/testify/require"
"go.opentelemetry.io/collector/component"
Expand Down Expand Up @@ -35,9 +34,7 @@ func TestNewFactory(t *testing.T) {
factory := NewFactory()

var expectedCfg component.Config = &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: 10 * time.Second,
},
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
Services: monitorServices,
CacheResources: 24 * 60 * 60,
Expand Down
6 changes: 2 additions & 4 deletions receiver/couchdbreceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,8 @@ func NewFactory() receiver.Factory {

func createDefaultConfig() component.Config {
return &Config{
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: 1 * time.Minute,
},
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
HTTPClientSettings: confighttp.HTTPClientSettings{
TLSSetting: configtls.TLSClientSetting{},
Endpoint: defaultEndpoint,
Expand Down
4 changes: 1 addition & 3 deletions receiver/elasticsearchreceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,7 @@ func NewFactory() receiver.Factory {
// createDefaultConfig creates the default elasticsearchreceiver config.
func createDefaultConfig() component.Config {
return &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: defaultCollectionInterval,
},
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
HTTPClientSettings: confighttp.HTTPClientSettings{
Endpoint: defaultEndpoint,
Timeout: defaultHTTPClientTimeout,
Expand Down
7 changes: 2 additions & 5 deletions receiver/filestatsreceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ package filestatsreceiver // import "github.com/open-telemetry/opentelemetry-col

import (
"context"
"time"

"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/consumer"
Expand All @@ -25,10 +24,8 @@ func NewFactory() receiver.Factory {

func newDefaultConfig() component.Config {
return &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: 1 * time.Minute,
},
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
}
}

Expand Down
4 changes: 1 addition & 3 deletions receiver/flinkmetricsreceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,7 @@ func NewFactory() receiver.Factory {

func createDefaultConfig() component.Config {
return &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: 10 * time.Second,
},
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
HTTPClientSettings: confighttp.HTTPClientSettings{
Endpoint: defaultEndpoint,
Timeout: 10 * time.Second,
Expand Down
4 changes: 1 addition & 3 deletions receiver/flinkmetricsreceiver/factory_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,7 @@ func TestNewFactory(t *testing.T) {
factory := NewFactory()

var expectedCfg component.Config = &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: 10 * time.Second,
},
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
HTTPClientSettings: confighttp.HTTPClientSettings{
Endpoint: defaultEndpoint,
Timeout: 10 * time.Second,
Expand Down
4 changes: 1 addition & 3 deletions receiver/googlecloudspannerreceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,7 @@ func NewFactory() receiver.Factory {

func createDefaultConfig() component.Config {
return &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: defaultCollectionInterval,
},
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The defaultCollectionInterval const defined above is now left unused. This could trick someone trying to change it and seeing no effect. We should remove that const.

TopMetricsQueryMaxRows: defaultTopMetricsQueryMaxRows,
BackfillEnabled: defaultBackfillEnabled,
HideTopnLockstatsRowrangestartkey: defaultHideTopnLockstatsRowrangestartkey,
Expand Down
7 changes: 2 additions & 5 deletions receiver/haproxyreceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ package haproxyreceiver // import "github.com/open-telemetry/opentelemetry-colle

import (
"context"
"time"

"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/consumer"
Expand All @@ -25,10 +24,8 @@ func NewFactory() receiver.Factory {

func newDefaultConfig() component.Config {
return &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: 1 * time.Minute,
},
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
}
}

Expand Down
4 changes: 1 addition & 3 deletions receiver/httpcheckreceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,7 @@ func NewFactory() receiver.Factory {

func createDefaultConfig() component.Config {
return &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: 10 * time.Second,
},
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
HTTPClientSettings: confighttp.HTTPClientSettings{
Endpoint: defaultEndpoint,
Timeout: 10 * time.Second,
Expand Down
4 changes: 1 addition & 3 deletions receiver/httpcheckreceiver/factory_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,7 @@ func TestNewFactory(t *testing.T) {
factory := NewFactory()

var expectedCfg component.Config = &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: 10 * time.Second,
},
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
HTTPClientSettings: confighttp.HTTPClientSettings{
Endpoint: defaultEndpoint,
Timeout: 10 * time.Second,
Expand Down
8 changes: 2 additions & 6 deletions receiver/iisreceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@
package iisreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/iisreceiver"

import (
"time"

"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/receiver"
"go.opentelemetry.io/collector/receiver/scraperhelper"
Expand All @@ -23,9 +21,7 @@ func NewFactory() receiver.Factory {

func createDefaultConfig() component.Config {
return &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: time.Minute,
},
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
}
}
3 changes: 2 additions & 1 deletion receiver/memcachedreceiver/config_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ package memcachedreceiver
import (
"path/filepath"
"testing"
"time"

"github.com/stretchr/testify/require"
"go.opentelemetry.io/collector/component"
Expand All @@ -18,7 +19,7 @@ func TestDefaultConfig(t *testing.T) {
cfg := NewFactory().CreateDefaultConfig().(*Config)
require.Equal(t, defaultEndpoint, cfg.Endpoint)
require.Equal(t, defaultTimeout, cfg.Timeout)
require.Equal(t, defaultCollectionInterval, cfg.CollectionInterval)
require.Equal(t, time.Minute, cfg.CollectionInterval)
}

func TestLoadConfig(t *testing.T) {
Expand Down
11 changes: 4 additions & 7 deletions receiver/memcachedreceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,8 @@ import (
)

const (
defaultEndpoint = "localhost:11211"
defaultTimeout = 10 * time.Second
defaultCollectionInterval = 10 * time.Second
defaultEndpoint = "localhost:11211"
defaultTimeout = 10 * time.Second
)

// NewFactory creates a factory for memcached receiver.
Expand All @@ -32,10 +31,8 @@ func NewFactory() receiver.Factory {

func createDefaultConfig() component.Config {
return &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: defaultCollectionInterval,
},
Timeout: defaultTimeout,
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
Timeout: defaultTimeout,
NetAddr: confignet.NetAddr{
Endpoint: defaultEndpoint,
},
Expand Down
7 changes: 3 additions & 4 deletions receiver/memcachedreceiver/factory_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,14 @@ package memcachedreceiver
import (
"context"
"testing"
"time"

"github.com/stretchr/testify/require"
"go.opentelemetry.io/collector/component/componenttest"
"go.opentelemetry.io/collector/consumer/consumertest"
"go.opentelemetry.io/collector/receiver/receivertest"
"go.opentelemetry.io/collector/receiver/scraperhelper"

"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/memcachedreceiver/internal/metadata"
)

func TestType(t *testing.T) {
Expand All @@ -33,9 +34,7 @@ func TestCreateMetricsReceiver(t *testing.T) {
context.Background(),
receivertest.NewNopCreateSettings(),
&Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: 10 * time.Second,
},
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
},
consumertest.NewNop(),
)
Expand Down
2 changes: 1 addition & 1 deletion receiver/memcachedreceiver/testdata/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
memcached:
endpoint: "localhost:11211"
collection_interval: 10s
collection_interval: 1m
6 changes: 2 additions & 4 deletions receiver/mongodbreceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,8 @@ func NewFactory() receiver.Factory {

func createDefaultConfig() component.Config {
return &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: time.Minute,
},
Timeout: time.Minute,
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
Timeout: time.Minute,
Hosts: []confignet.NetAddr{
{
Endpoint: "localhost:27017",
Expand Down
9 changes: 3 additions & 6 deletions receiver/mysqlreceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ package mysqlreceiver // import "github.com/open-telemetry/opentelemetry-collect

import (
"context"
"time"

"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/config/confignet"
Expand All @@ -25,11 +24,9 @@ func NewFactory() receiver.Factory {

func createDefaultConfig() component.Config {
return &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: 10 * time.Second,
},
AllowNativePasswords: true,
Username: "root",
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
AllowNativePasswords: true,
Username: "root",
NetAddr: confignet.NetAddr{
Endpoint: "localhost:3306",
Transport: "tcp",
Expand Down
4 changes: 1 addition & 3 deletions receiver/nginxreceiver/factory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,7 @@ func NewFactory() receiver.Factory {

func createDefaultConfig() component.Config {
return &Config{
ScraperControllerSettings: scraperhelper.ScraperControllerSettings{
CollectionInterval: 10 * time.Second,
},
ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(metadata.Type),
HTTPClientSettings: confighttp.HTTPClientSettings{
Endpoint: "http://localhost:80/status",
Timeout: 10 * time.Second,
Expand Down
2 changes: 1 addition & 1 deletion receiver/nginxreceiver/testdata/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
nginx:
endpoint: "http://localhost:80/status"
collection_interval: 10s
collection_interval: 1m
Loading