Skip to content

Separate test suites for different configurations #14128

@jaydeluca

Description

@jaydeluca

As part of #13468 we collect and analyze the telemetry emitted by each module's tests in order to document the metrics, span kinds, and attributes. Many of our modules will have individual test suites configured for different configurations, and this works well with our tooling, because we can then tag each test run with the associated configs. For example:

testing {
  suites {
    val testStableSemconv by registering(JvmTestSuite::class) {
      targets {
        all {
          testTask.configure {
            jvmArgs("-Dotel.semconv-stability.opt-in=database")
            systemProperty("metadataConfig", "otel.semconv-stability.opt-in=database")
          }
        }
      }
    }
  }
}

tasks {
  withType<Test>().configureEach {
    systemProperty("collectMetadata", findProperty("collectMetadata")?.toString() ?: "false")
  }

  check {
    dependsOn(testing.suites)
  }
}

While going through and enabling telemetry collection for #14098 and #14104, there are modules that test everything at once, making it difficult to identify the "default" behavior if no configurations are enabled.

This may not make sense for every module, but this issue is to track modules that should be evaluated, and if it would be valuable, separate out the test suites. This will also involve updating the test assertions so that they are conditionally checking for attributes based on system properties as well.

Instrumentation List:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    In Progress

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions