Ensure semantic conventions are up to date

[bogdandrutu]

We should build a tool or share a tool with opentelemetry-go for all semantic conventions.

/cc @MrAlias @punya

[bogdandrutu]

@MrAlias @punya so to understand there is no such thing :)) Maybe we should start talking about building this.

[MrAlias]

@MrAlias @punya so to understand there is no such thing :)) Maybe we should start talking about building this.

Sure, I'm onboard. What does this entail? Should we schedule a meeting, or can it be done asynchronously here?

[MrAlias]

Currently we use the hand-crafted semconv package to contain all the semantic conventions. There are some curated comments in there that help relate the semantic conventions to OTel Go implementation concepts. It would be ideal if the thing we build could preserve these types of comments.

[punya]

There are some existing tools that process the semconv yaml files: https://github.com/open-telemetry/build-tools/blob/main/semantic-conventions/README.md. It looks like they are already being used for Java codegen. @bogdandrutu were you thinking about extending that tooling with a new Jinja template, or building something else?

[alolita]

@rakyll are you looking at this - can you share your doc / thoughts here?

[rakyll]

There is a spec change related to this: open-telemetry/opentelemetry-specification#1515

[alolita]

@bogdandrutu please advise on next steps given this would be important to implement before GA.

@Aneurysm9 can we reuse the Go semantic conventions that you've been adding support for in the Go API/SDK as a common package in Go that the Collector can use too?

[bogdandrutu]

@Aneurysm9 I think we should share the generated semconv if go library has already this. We can do the following:

1. Rename the go-proto repo to go-shared and expose multiple modules there like:
   a. protos
   b. semconv
   c. pdata (for collector and other users)
2. Change the vanity URL so we don't break the current proto package.

Otherwise we can ask for a different repo as well.

[MrAlias]

I like the idea of having a new repository dedicated to the semconv package. I'm in favor of something like opentelemetry-semantic-conventions-go being created to house the package and generation tooling and pointing at that with go.opentelemetry.io/semconv.

I think isolating the purpose of each repository (i.e. protos and semantic conventions) will promote cohesive and single-focused Go packages.

[alolita]

@Aneurysm9 - can you provide your recommendations used for the Go API/SDK which we can reuse here? Thx.

[Aneurysm9]

I'm with @MrAlias in thinking that a new repo with a new vanity URL would be the way to go. I think the generation tooling we recently added to the Go repo can easily be moved elsewhere.

The one problem I see that we'll need to work through is that the semconv package currently depends on the attribute package from the API as all of the conventions are defined as attribute.Keys or attribute.KeyValues. The attribute package has a dependency on an internal package, but it's just for a few conversion helpers that can easily be moved/duplicated. Moving that package will cause a lot of churn, but probably less than the two times we've previously renamed it.

I've put this topic on the agenda for the Go SIG meeting today.

[MrAlias]

From the Go SIG:

Currently the semconv package represents the semantic conventions with our attribute package KeyValue type. Would it be okay to keep the generation in this type? Would the collector be okay transforming from these to the pdata format?

Would it make more sense to just move the generation code to a separate repo?

@bogdandrutu @tigrannajaryan

[alolita]

@MrAlias Having the generation code to a separate repo would help in both the Go library and the Collector being able to leverage the same package. @bogdandrutu had mentioned this strategy could be reused in the long run for all Go components.

[MrAlias]

@MrAlias Having the generation code to a separate repo would help in both the Go library and the Collector being able to leverage the same package. @bogdandrutu had mentioned this strategy could be reused in the long run for all Go components.

I'm not sure I understand what is being said here.

I agree that unifying the location of shared code is a good idea.

Do we want to put the generation code in the repository or the generated code? If it is the generated code, would the collector be okay with that code using "go.opentelemetry.io/otel/attribute".KeyValue type to represent the conventions?

[alolita]

Based on the Collector SIG discussion today, the Go library and Collector maintainers will sync to finalize next steps in setting up a common repo as proposed by @Aneurysm9 to reuse common packages between Go based code bases.

[alolita]

@bogdandrutu Have you signed off on all the items that need to be completed for this task? I can close the issue if you sign off. Thx.