Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move the lifecycle tests with to each component module #27849

Closed
Tracked by #27826
dmitryax opened this issue Oct 18, 2023 · 3 comments
Closed
Tracked by #27826

Move the lifecycle tests with to each component module #27849

dmitryax opened this issue Oct 18, 2023 · 3 comments
Assignees
Labels
cmd/mdatagen mdatagen command help wanted Extra attention is needed test coverage Improve test coverage

Comments

@dmitryax
Copy link
Member

Move lifecycle tests from cmd/otelcolcontrib to each module as individual generated tests.

mdatagen tool can be used for to generate the tests. We would need another section in the metadata.yaml as a source for the generated tests. Sample section:

tests:
  config: // {} by default
    endpoint: http://127.0.0.1
    region: local
    queue:
      enabled: false
    retry:
      enabled: false
  skip_lifecycle: false // false by default
  expect_consumer_error: true // false by default

Moving the tests to the components provides us exposure to the internals of the component, which can help us improve the test coverage and eliminate the need for the 'skip_lifecycle' option. Presently, certain components cannot accommodate test configurations that pass lifecycle tests, hence the necessity of retaining the 'skip_lifecycle' option. Looking ahead, we should be able to utilize internal helpers, not accessible through the public config interface, to ensure proper lifecycle validation and consequently eliminate the need for 'skip_lifecycle'."

@atoulme
Copy link
Contributor

atoulme commented Nov 16, 2023

#29284 has shown how to do this and we can now extend this work to all exporters, for starters.

Adding a task list for all exporters:

  • alertmanagerexporter
  • alibabacloudlogserviceexporter
  • awscloudwatchlogsexporter
  • awsemfexporter
  • awskinesisexporter
  • awss3exporter
  • awsxrayexporter
  • azuredataexplorerexporter
  • azuremonitorexporter
  • carbonexporter
  • cassandraexporter
  • clickhouseexporter
  • coralogixexporter
  • datadogexporter
  • datasetexporter
  • dynatraceexporter
  • elasticsearchexporter
  • f5cloudexporter
  • fileexporter
  • googlecloudexporter
  • googlecloudpubsubexporter
  • googlemanagedprometheusexporter
  • honeycombmarkerexporter
  • influxdbexporter
  • instanaexporter
  • kafkaexporter
  • kineticaexporter
  • loadbalancingexporter
  • logicmonitorexporter
  • logzioexporter
  • lokiexporter
  • mezmoexporter
  • opencensusexporter
  • opensearchexporter
  • prometheusexporter
  • prometheusremotewriteexporter
  • pulsarexporter
  • sapmexporter
  • sentryexporter
  • signalfxexporter
  • skywalkingexporter
  • splunkhecexporter
  • sumologicexporter
  • syslogexporter
  • tanzuobservabilityexporter
  • tencentcloudlogserviceexporter
  • zipkinexporter

For each exporter, you need to take the configuration they expose in cmd/otelcontribcol/exporters_test.go, and move it to a yaml fragment in each of the metadata.yaml of the component under tests/config. Same for skip_lifecycle and expect_consumer_error.

RoryCrispin pushed a commit to ClickHouse/opentelemetry-collector-contrib that referenced this issue Nov 24, 2023
dmitryax pushed a commit that referenced this issue Nov 28, 2023
Continuation of the work on #27849 - now opening a way to create
lifecycle tests for processors.
dmitryax pushed a commit that referenced this issue Dec 7, 2023
Continuation of the work on
#27849
- now opening a way to create lifecycle tests for receivers.
jayasai470 pushed a commit to jayasai470/opentelemetry-collector-contrib that referenced this issue Dec 8, 2023
Continuation of the work on
open-telemetry#27849
- now opening a way to create lifecycle tests for receivers.
TylerHelmuth pushed a commit that referenced this issue Dec 11, 2023
dmitryax pushed a commit that referenced this issue Dec 14, 2023
Continue work on #27849

---------

Co-authored-by: Evan Bradley <11745660+evan-bradley@users.noreply.github.com>
@atoulme atoulme mentioned this issue Dec 22, 2023
3 tasks
cparkins pushed a commit to AmadeusITGroup/opentelemetry-collector-contrib that referenced this issue Jan 10, 2024
Continue work on open-telemetry#27849

---------

Co-authored-by: Evan Bradley <11745660+evan-bradley@users.noreply.github.com>
@atoulme
Copy link
Contributor

atoulme commented Jan 11, 2024

Task list for remaining receivers:

  • activedirectorydsreceiver
  • awsecscontainermetricsreceiver
  • awsxrayreceiver
  • chronyreceiver
  • datadogreceiver
  • dockerstatsreceiver
  • elasticsearchreceiver
  • filelogreceiver
  • filereceiver
  • gitproviderreceiver
  • iisreceiver
  • jmxreceiver
  • journaldreceiver
  • k8sclusterreceiver
  • k8sobjectsreceiver
  • kubeletstatsreceiver
  • lokireceiver
  • memcachedreceiver
  • mongodbatlasreceiver
  • mongodbreceiver
  • namedpipereceiver
  • podmanreceiver
  • receivercreator
  • saphanareceiver
  • solacereceiver
  • sqlserverreceiver
  • sshcheckreceiver
  • windowseventlogreceiver
  • windowsperfcountersreceiver

XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
…elemetry#30512)

Relates to open-telemetry#27849

---------

Co-authored-by: Antoine Toulme <antoine@lunar-ocean.com>
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
…ry#30523)

Relates to
open-telemetry#27849

---------

Co-authored-by: Antoine Toulme <antoine@lunar-ocean.com>
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
…try#30537)

Relates to
open-telemetry#27849

---------

Co-authored-by: Antoine Toulme <antoine@lunar-ocean.com>
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
…etry#30516)

Relates to
open-telemetry#27849

---------

Co-authored-by: Antoine Toulme <antoine@lunar-ocean.com>
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
…elemetry#30540)

Relates to
open-telemetry#27849

---------

Co-authored-by: Antoine Toulme <antoine@lunar-ocean.com>
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
**Description:**
Add generated tests for chrony receiver; move initialization of the
chrony client to the start function

**Link to tracking Issue:**
Relates to open-telemetry#27849
MovieStoreGuy pushed a commit that referenced this issue Mar 14, 2024
Relates to
#27849

---------

Co-authored-by: Antoine Toulme <antoine@lunar-ocean.com>
andrzej-stencel pushed a commit that referenced this issue Apr 10, 2024
**Description:**
Change the lifecycle of the extension so that the creation of the AWS
client happens in the Start method, which helps with supporting
component lifecycle.

**Link to tracking Issue:**
Relates to #27849
@atoulme
Copy link
Contributor

atoulme commented Apr 13, 2024

I would like to mark this issue as closed. TODO tags exist for each component that needs to satisfy their test requirements, which will become a requirement down the road for component maturation. Please reopen if you see more work is needed, I'd rather follow up individually on each remaining component.

@atoulme atoulme closed this as completed Apr 13, 2024
lapis-lark pushed a commit to lapis-lark/otel-cassandra-exporter-rework that referenced this issue Nov 4, 2024
Relates to
open-telemetry/opentelemetry-collector-contrib#27849

duo to `cluster.CreateSession()` func, event if set `skip_lifecycle:
true` the Lifecycle test still fail:

https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/1de0e36f76a5e6a9a9ced094b7b90529ea3beae6/exporter/cassandraexporter/exporter_logs.go#L26-L28
test log:
```
=== RUN   TestComponentLifecycle
=== RUN   TestComponentLifecycle/logs-shutdown
    generated_component_test.go:75: 
        	Error Trace:	/Users/fraps/daocloud/github-code/opentelemetry-collector-contrib/exporter/cassandraexporter/generated_component_test.go:75
        	Error:      	Received unexpected error:
        	            	cannot configure cassandra traces exporter: gocql: unable to create session: unable to discover protocol version: dial tcp 127.0.0.1:9042: connect: connection refused
        	Test:       	TestComponentLifecycle/logs-shutdown
=== RUN   TestComponentLifecycle/logs-lifecycle
=== RUN   TestComponentLifecycle/traces-shutdown
    generated_component_test.go:75: 
        	Error Trace:	/Users/fraps/daocloud/github-code/opentelemetry-collector-contrib/exporter/cassandraexporter/generated_component_test.go:75
        	Error:      	Received unexpected error:
        	            	cannot configure cassandra traces exporter: gocql: unable to create session: unable to discover protocol version: dial tcp 127.0.0.1:9042: connect: connection refused
        	Test:       	TestComponentLifecycle/traces-shutdown
```

we need more discuss for it @atoulme

---------

Co-authored-by: Antoine Toulme <antoine@lunar-ocean.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cmd/mdatagen mdatagen command help wanted Extra attention is needed test coverage Improve test coverage
Projects
None yet
Development

No branches or pull requests

2 participants