Issue: [detector: gcp] "cannot merge resource due to conflicting Schema URL" in 1.18.0

[utsushiiro]

Description

After upgrading go.opentelemetry.io/contrib/detectors/gcp to v1.18.0, an error occurred in the application running on Cloud Run: "cannot merge resource due to conflicting Schema URL".

go.opentelemetry.io/contrib/detectors/gcp@v1.18.0 uses go.opentelemetry.io/otel/semconv/v1.17.0, while its dependency go.opentelemetry.io/otel/sdk@v1.17.0 uses go.opentelemetry.io/otel/semconv/v1.21.0. Is this version mismatch the cause of the error?

Related PRs:

• Upgrade all use of semconv to v1.21.0 opentelemetry-go#4408

Environment

My Application is running on Cloud Run. Docker image is based on gcr.io/distroless/static-debian11.

• Go Version: 1.21
• opentelemetry.io/contrib/detectors/gcp version: 1.18.0
• go.opentelemetry.io/otel/sdk version: 1.17.0

Code

	res, err := resource.New(
		ctx,
		resource.WithDetectors(gcp.NewDetector()), // <- semconv/v1.17.0 is used 
		resource.WithTelemetrySDK(), // <- semconv/v1.21.0 is used
		resource.WithAttributes(
			semconv.ServiceNameKey.String(serviceName),
			attribute.String("environment", env),
		),
	)
	...

[MrAlias]

This is a known issue: open-telemetry/opentelemetry-go#2341

https://github.com/MrAlias/otel-schema-utils was created to help while the specification and this project come to define its own translation approach.

[maximkosov]

We also hit this issue yesterday.

@MrAlias how can we use https://github.com/MrAlias/otel-schema-utils to resolve this issue?

Also, fyi, it means that gcp detector 1.18 is broken because it requires sdk 1.17 in go.mod and apparently it is not compatible with that version of sdk.

[MrAlias]

cc @dashpole

[dashpole]

This will be fixed by #4263

[MrAlias]

@MrAlias how can we use https://github.com/MrAlias/otel-schema-utils to resolve this issue?

module main

go 1.21.0

require (
	github.com/MrAlias/otel-schema-utils v0.2.1-alpha
	go.opentelemetry.io/contrib/detectors/gcp v1.18.0
	go.opentelemetry.io/otel v1.17.0
	go.opentelemetry.io/otel/sdk v1.17.0
)

require (
	cloud.google.com/go/compute v1.14.0 // indirect
	cloud.google.com/go/compute/metadata v0.2.3 // indirect
	github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.18.0 // indirect
	github.com/Masterminds/semver/v3 v3.2.1 // indirect
	github.com/go-logr/logr v1.2.4 // indirect
	github.com/go-logr/stdr v1.2.2 // indirect
	go.opentelemetry.io/otel/metric v1.17.0 // indirect
	go.opentelemetry.io/otel/schema v0.0.5 // indirect
	go.opentelemetry.io/otel/trace v1.17.0 // indirect
	golang.org/x/sys v0.11.0 // indirect
	gopkg.in/yaml.v2 v2.4.0 // indirect
)

package main

import (
	"context"
	"log"

	"github.com/MrAlias/otel-schema-utils/schema"
	"go.opentelemetry.io/contrib/detectors/gcp"
	"go.opentelemetry.io/otel/attribute"
	"go.opentelemetry.io/otel/sdk/resource"
	semconv "go.opentelemetry.io/otel/semconv/v1.17.0"
)

const (
	serviceName = "my-service"
	env         = "prod"
)

func must(res *resource.Resource, err error) *resource.Resource {
	if err != nil {
		log.Fatal(err)
	}
	return res
}

func main() {
	conv := schema.NewConverter(schema.NewLocalClient())

	ctx := context.Background()
	res := must(conv.MergeResources(
		ctx, semconv.SchemaURL,
		must(gcp.NewDetector().Detect(ctx)),
		must(resource.New(
			ctx,
			resource.WithTelemetrySDK(),
			resource.WithAttributes(
				semconv.ServiceNameKey.String(serviceName),
				attribute.String("environment", env),
			),
		),
		),
	))
	log.Println(res.SchemaURL())
}

$ go run .
2023/08/31 13:48:23 https://opentelemetry.io/schemas/1.17.0

[dustinmoris]

Is there any other way of getting this to work right now without having to import yet another third party dependency like https://github.com/MrAlias/otel-schema-utils?

Any workarounds, etc.?

[maximkosov]

@dustinmoris I just removed resource.WithTelemetrySDK(). It is only adding version and language of open telemetry library, which is always the same in our case.

[dustinmoris]

@maximkosov Thanks, that's probably better than what I did just now. I copied the entire source code from detectors/gcp into my own project, changed the semconv to 1.21.0 and renamed FaaSIDKey to FaaSInstanceKey in 4 places as per a comment in another issue and that worked for me too.

[anuraaga]

@dustinmoris Also can consider manually running contrib detectors and only using their attributes, it allows all the data to still be present without copying code

	gcpRes, err := gcp.NewDetector().Detect(ctx)
	if err != nil {
		panic(err)
	}

	res, err := resource.New(ctx,
		resource.WithAttributes(gcpRes.Attributes()...),
		resource.WithTelemetrySDK(),
		resource.WithHost(),
		resource.WithContainer(),
		resource.WithProcess(),
		resource.WithOS(),
		resource.WithFromEnv(),
	)

It's too bad the original issue open-telemetry/opentelemetry-go#2341 doesn't have more traction as it does seem it's creating a lot of unhappy users.