Discuss possible stable+incubating solutions for semconv artifacts

[lmolkova]

Extracted from #45

Repeating the question I left on the python PR open-telemetry/opentelemetry-python#3586 (comment)

Option 1: ship 2 artifacts (semconv and semconv-incubating) - current solution

It's aligned with semantic versioning, easy to maintain.
Still having stable attributes as deprecated in the incubating artifact is not great.

Option 2: ship 1 artifact, but two different versions: stable and preview. Stable contains only stable semconv. Preview contains all of them.

Ideal in terms of semantic versioning. When attribute stabilizes, it remains in the same namespace and all libs/apps that depend on it don't need to change
The only downside is that it could be harder to maintain and release.

Option 3: Probably not applicable to Java - ship 1 artifact, 1 version containing stable and experimental semconv. Emphasize that some parts are experimental via namespace (import semconv.experimental.foo_attributes, perhaps we should call it _experimental )?

It's easy to do, but violates semantic versioning. When attribute stabilizes, it moves to a different namespace, i.e. libs/apps that depend on it need to change.

[lmolkova]

I think the cleanest solution from user perspective would be option 2:

we'd have semantic-conventions-java:1.0.0 with stable only, semantic-conventions-java:1.0.0-beta.1 with all semconvs. Experimental -> stable transition requires no action on the user side.
It's also one of the common patterns to ship experimental features.

We can also discuss at the SIG meeting, but I always have a conflict discussing the same thing with Python SIG at the same time:)

Once we pick the ideal solution I'll make another pass on build-tools to add convenience for this specific case if anything else is necessary.

[jack-berg]

We discussed in the 3/14/24 Java SIG. Decided that a single artifact is probably not the right approach based on the following example:

• App depends on libraryA which depends on :semantic-conventions-java:1.0.0-beta
• App depends on libraryB which depends on :semantic-conventions-java:1.0.0
• App depends on libraryC which depends on :semantic-conventions-java:1.1.0

The normal way to resolve dependency version conflicts is to update to the latest of all dependencies. In this case, 1.1.0. But doing so would expose the app to runtime errors since libraryA was compiled against 1.0.0-beta, which contains experimental attributes not available in 1.1.0. To resolve, the app owner would have to intervene and set the version to 1.1.0-beta.

The consensus of the discussion was to:

• Continue with two artifacts, opentelemetry-semconv and opentelemetry-semconv-incubating (as merged to main currently)
• opentelemetry-semconv only contains stable semantic conventions.
  • Classes live in io.opentelemetry.semconv package
  • Attributes which are deprecated are retaining and annotated with @deprecated
  • Goal is to eventually mark this artifact as stable and have strong compatibility guarantees
  • Goal is to encourage library authors to depend on this artifact
• opentelemetry-semconv-incubating contains stable and incubating semantic conventions
  • Classes live in io.opentelemetry.semconv.incubating package. Note this is different than stable artifact to make java module system work.
  • Stable attributes will be annotated @deprecated with javadoc pointing to the equivalent location in the stable artifact
  • The semantic-conventions repo will keep incubating attributes around indefinitely (or for a long time) and mark as deprecated. These deprecated, experimental attributes will be annotated @deprecated.
  • This artifact will always be marked alpha, since we may eventually remove some deprecated experimental attributes, and since the types of experimental attributes are subject to change.
  • We will discourage library authors from using this artifact. They can use it for making assertions in tests, but should make copies of any experimental attributes they depend on in instrumentation.

Note that #45 is aligned with this strategy, except in that it does not currently retain deprecated experimental attributes. Will address that in a followup.