diff --git a/oteps/0227-separate-semantic-conventions.md b/oteps/0227-separate-semantic-conventions.md new file mode 100644 index 00000000000..ff144b58da1 --- /dev/null +++ b/oteps/0227-separate-semantic-conventions.md @@ -0,0 +1,182 @@ +# Separate Semantic Conventions + +Move Semantic Conventions outside of the main Specifications and version them +separately. + +## Motivation + +We need to allow semantic conventions to evolve mostly independent of the +overall OpenTelemetry specification. Today, any breaking change in a semantic +convention would require bumping the version number of the entirety of the +OpenTelemetry specification. + +## Explanation + +A new GitHub repository called `semantic-conventions` would be created in the +OpenTelemetry organization. + +This would *initially* have the following structure: + +- Boilerplate files, e.g. `README.md`, `LICENSE`, `CODEOWNERS`, `CONTRIBUTING.md` +- `Makefile` that allows automatic generation of documentation from model. +- `semantic_conventions/` The set of YAML files that exist in + `{specification}/semantic_conventions` today. +- `docs/` A new directory that contains human readable documentation for how to + create instrumentation compliant with semantic conventions. + - `resource/` - Contents of `{specification}/resource/semantic_conventions` + - `trace/` - Contents of `{specification}/trace/semantic_conventions` + - `metrics/` - Contents of `{specification}/metrics/semantic_conventions` + - `logs/`- Contents of `{specification}/logs/semantic_conventions` + - `schemas/` - A new location for [Telemetry Schemas](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/schemas/README.md) + to live. This directory will be hosted at + `https://opentelemetry.io/schemas/` + +Existing semantic conventions in the specification would be marked as +moved, with documentation denoting the move, but preserving previous contents. + +Additionally, if the semantic conventions eventually move to domain-specific +directory structure (e.g. `docs/{domain}/README.md`, with trace, metrics, events +in the same file), then this can be refactored in the new repository, preserving +git history. + +There will also be the following exceptions in the specification: + +- Semantic conventions used to implement API/SDK details will be fully specified in the `opentelemetry-specification` repo + and will not be allowed to change in the Semantic Convention directory. + - Error/Exception handling will remain in the specification. + - SDK configuration interaction w/ semantic convention will remain in the + specification. Specifically `service.name`. +- The specification may elevate some semantic conventions as necessary for + compatibility requirements, e.g. `service.instance.id` and + [Prometheus Compatibility](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/compatibility/prometheus_and_openmetrics.md). + +These exceptions exist because: + +- Stable portions of the specification already rely on these conventions. +- These conventions are required to implement an SDK today. + +As such, the Specification should define the absolute minimum of reserved or +required attribute names and their interaction with the SDK. + +## Internal details + +The following process would be used to ensure semantic conventions are +seamlessly moved to their new location. This process lists steps in order: + +- A moratorium will be placed on Semantic Convention PRs to the specififcation + repository. (Caveat that PRs related to this proposal would be allowed). +- Interactions between Semantic Conventions and the Specification will be + extracted such that the Specification can place requirements on Semantic + Conventions and *normative* specification language will remain in the + core specification directories. +- A new repository `open-telemetry/semantic-conventions` will be constructed with + the proposed format and necessary `Makefile` / tooling. + - The new repository would be created by using `git filter-branch` to preserve + all existing semantic convention history. *This means all existing + semantic conventions will be in the new repository*. + - GitHub Actions, `Makefile` tooling and contributing / readmes would be + updated for the separate repository. + - **Note: At this point, the new location for semantic conventions should + be adoptable/usable.** +- Semantic conventions in the Specification will be marked as moved with + links to the new location. + - The semconv YAML files in the specification repository *will be deleted*. + - All semconv markdown files will be updated such that: + - They no longer generate from YAML files. + - They include a header denoting deprecation and move to the new repository. +- Instrumentation authors will update their code generation to pull from the new + semconv repository instead of the specification repository. + +## Trade-offs and mitigations + +This proposal has a few drawbacks: + +- The semantic conventions will no longer be easily referenceable form the specification. + - This is actually a benefit. We can ensure isolation of convention from + specification and require the Specification to use firm language for + attributes it requires, like `service.name`. + - We will provide links from existing location to the new location. +- Semantic Convention version will no longer match the specification version. + - Instrumentation authors will need to consume a separate semantic-convention + bundle from Specification bundle. What used to be ONE upgrade effort will + now be split into two (hopefully smaller) efforts. + - We expect changes from Semantic Conventions and Specification to be + orthogonal, so this should not add significant wall-clock time. +- Existing PRs against semantic conventions will need to be regenerated. + +Initially this repository would have the following ownership: + +- Approvers + - [Christian Neumüller](https://github.com/Oberon00), Dynatrace + - [James Moessis](https://github.com/jamesmoessis), Atlassian + - [Joao Grassi](https://github.com/joaopgrassi), Dynatrace + - [Johannes Tax](https://github.com/pyohannes), Microsoft + - [Liudmila Molkova](https://github.com/lmolkova), Microsoft + - [Sean Marcinak](https://github.com/MovieStoreGuy), Atlassian + - [Ted Young](https://github.com/tedsuo), Lightstep +- Approvers (HTTP Only) + - [Trask Stalnaker](github.com/trask) +- Approvers (SchemaUrl Files) + - [Tigran Najaryan](github.com/tigrannajaryan) +- Maintainers + - [Josh Suereth](github.com/jsuereth) + - [Armin Ruech](github.com/arminru) + - [Reiley Yang](github.com/reyang) + +That is, Maintenance would initially continue to fall on (a subset of) the +Technical Committee. Approvers would start with existing semconv approvers in +addition to targeted at HTTP semantic convention stability approvers and +expand rapidly as we build momentum on semantic conventions. + +## Prior art and alternatives + +When we evaluate equivalent communities and efforts we see the following: + +- `OpenTracing` - had specification and [semantics](https://github.com/opentracing/specification/blob/master/semantic_conventions.md) + merged. +- `OpenCensus` - had specification and semantics merged. However, OpenCensus + merged with OpenTelemetry prior to mass adoption or stable release of its + specification. +- `Elastic Common Schema` - the schema is its own project / document. +- `Prometheus` - Prometheus does not define rigid guidelines for telemetry, like + semantic conventions, instead relying on naming conventions and + standardization through mass adoption. + +## Open questions + +This OTEP doesn't address the full needs of tooling and codegen that will be +needed for the community to shift to a separate semantic convention directory. +This will require each SIG that uses autogenerated semantic conventions to +adapt to the new location. + +The first version of the new location for semantic conventions may not follow +the latest of the specification. There is reasoning to desire a `2.0` but the +details will be discussed in the new repository location upon execution of this +OTEP. + +## Future possibilities + +This OTEP paves way for the following desirable features: + +- Semantic Conventions can decide to bump major version numbers to accommodate + new signals or hard-to-resolve new domains without breaking the Specification + version number. +- Semantic Conventions can have dedicated maintainers and approvers. +- Semantic Conventions can restructure to better enable subject matter experts + (SMEs) to have approver/CODEOWNER status on relevant directories. +- Semantic Conventions can adopt semantic versioning for itself, helping clearly + denote breaking changes to users. + +There is a desire to move semantic conventions to domain-specific directories +instead of signal-specific. This can occur after the separation of the repository +and will be proposed and discussed separately from this OTEP. + +For example: + +- `docs/` + - `signals/` - Conventions for metrics, traces + logs + - `http/` + - `db/` + - `messaging/` + - `client/` + - `resource/` - We still need resource-specific semantic conventions