From 1ed5e828009e22fc6b06c6ba0f9fa9abc67fb8ac Mon Sep 17 00:00:00 2001 From: David Mirza Date: Thu, 28 Jul 2022 07:54:58 -0700 Subject: [PATCH 1/7] add process.parent_pid attribute (#2691) Co-authored-by: Armin Ruech --- CHANGELOG.md | 2 ++ semantic_conventions/resource/process.yaml | 5 +++++ specification/resource/semantic_conventions/process.md | 1 + 3 files changed, 8 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3b49b8972d0..c36531d0c22 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -59,6 +59,8 @@ release. ([#2650](https://github.com/open-telemetry/opentelemetry-specification/pull/2650)). - Improve the definition of `state` attribute for metric `system.network.connections` ([#2663](https://github.com/open-telemetry/opentelemetry-specification/pull/2663)) +- Add `process.parent_pid` attribute for use in reporting parent process id (PID) + ([#2691](https://github.com/open-telemetry/opentelemetry-specification/pull/2691)). ### Compatibility diff --git a/semantic_conventions/resource/process.yaml b/semantic_conventions/resource/process.yaml index bc387dea199..d7d3b25b564 100644 --- a/semantic_conventions/resource/process.yaml +++ b/semantic_conventions/resource/process.yaml @@ -10,6 +10,11 @@ groups: brief: > Process identifier (PID). examples: [1234] + - id: parent_pid + type: int + brief: > + Parent Process identifier (PID). + examples: [111] - id: executable.name type: string requirement_level: diff --git a/specification/resource/semantic_conventions/process.md b/specification/resource/semantic_conventions/process.md index dbfa1a7e38f..b741128b5ed 100644 --- a/specification/resource/semantic_conventions/process.md +++ b/specification/resource/semantic_conventions/process.md @@ -28,6 +28,7 @@ | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| | `process.pid` | int | Process identifier (PID). | `1234` | Recommended | +| `process.parent_pid` | int | Parent Process identifier (PID). | `111` | Recommended | | `process.executable.name` | string | The name of the process executable. On Linux based systems, can be set to the `Name` in `proc/[pid]/status`. On Windows, can be set to the base name of `GetProcessImageFileNameW`. | `otelcol` | Conditionally Required: See alternative attributes below. | | `process.executable.path` | string | The full path to the process executable. On Linux based systems, can be set to the target of `proc/[pid]/exe`. On Windows, can be set to the result of `GetProcessImageFileNameW`. | `/usr/bin/cmd/otelcol` | Conditionally Required: See alternative attributes below. | | `process.command` | string | The command used to launch the process (i.e. the command name). On Linux based systems, can be set to the zeroth string in `proc/[pid]/cmdline`. On Windows, can be set to the first parameter extracted from `GetCommandLineW`. | `cmd/otelcol` | Conditionally Required: See alternative attributes below. | From 3de977792acf35f46d656b3665e888a65de56760 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Robert=20Paj=C4=85k?= Date: Thu, 28 Jul 2022 17:08:32 +0200 Subject: [PATCH 2/7] Clarify "automatic instrumentation" term (#2700) --- specification/glossary.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/specification/glossary.md b/specification/glossary.md index 30dde00b739..6de6aaf419d 100644 --- a/specification/glossary.md +++ b/specification/glossary.md @@ -105,7 +105,8 @@ Coding against the OpenTelemetry API such as the [Tracing API](trace/api.md), [M ### Automatic Instrumentation -Refers to telemetry collection methods that do not require the end-user to write or access application code to use the OpenTelemetry APIs. Methods vary by programming language, and examples include bytecode injection or monkey patching. +Refers to telemetry collection methods that do not require the end-user to modify application's source code. +Methods vary by programming language, and examples include bytecode injection or monkey patching. Synonym: *Auto-instrumentation*. From 87e45d3be83d291eee8e4320c4f9292890984721 Mon Sep 17 00:00:00 2001 From: Alex Boten Date: Fri, 29 Jul 2022 08:42:32 -0700 Subject: [PATCH 3/7] Add schema definition for direction change (#2688) This is a follow up to https://github.com/open-telemetry/opentelemetry-specification/pull/2617 now that https://github.com/open-telemetry/opentelemetry-specification/pull/2653 has been merged. --- internal/tools/schema_check.sh | 2 +- schemas/1.13.0 | 105 +++++++++++++++++++++++++++++++++ 2 files changed, 106 insertions(+), 1 deletion(-) create mode 100644 schemas/1.13.0 diff --git a/internal/tools/schema_check.sh b/internal/tools/schema_check.sh index f9b1826a629..86c8e253be3 100755 --- a/internal/tools/schema_check.sh +++ b/internal/tools/schema_check.sh @@ -6,7 +6,7 @@ set -e -BUILD_TOOL_SCHEMAS_VERSION=0.12.1 +BUILD_TOOL_SCHEMAS_VERSION=0.13.0 # List of vesions that do not require or have a schema. declare -a skip_versions=("1.0.0" "1.0.1" "1.1.0" "1.2.0" "1.3.0" "1.6.0") diff --git a/schemas/1.13.0 b/schemas/1.13.0 new file mode 100644 index 00000000000..6202271c0dc --- /dev/null +++ b/schemas/1.13.0 @@ -0,0 +1,105 @@ +file_format: 1.1.0 +schema_url: https://opentelemetry.io/schemas/1.13.0 +versions: + 1.13.0: + metrics: + changes: + # https://github.com/open-telemetry/opentelemetry-specification/pull/2617 + - split: + apply_to_metric: system.paging.operations + by_attribute: direction + metrics_from_attributes: + system.paging.operations.in: in + system.paging.operations.out: out + # https://github.com/open-telemetry/opentelemetry-specification/pull/2617 + - split: + apply_to_metric: system.disk.io + by_attribute: direction + metrics_from_attributes: + system.disk.io.read: read + system.disk.io.write: write + # https://github.com/open-telemetry/opentelemetry-specification/pull/2617 + - split: + apply_to_metric: system.disk.operations + by_attribute: direction + metrics_from_attributes: + system.disk.operations.read: read + system.disk.operations.write: write + # https://github.com/open-telemetry/opentelemetry-specification/pull/2617 + - split: + apply_to_metric: system.disk.operation_time + by_attribute: direction + metrics_from_attributes: + system.disk.operation_time.read: read + system.disk.operation_time.write: write + # https://github.com/open-telemetry/opentelemetry-specification/pull/2617 + - split: + apply_to_metric: system.disk.merged + by_attribute: direction + metrics_from_attributes: + system.disk.merged.read: read + system.disk.merged.write: write + # https://github.com/open-telemetry/opentelemetry-specification/pull/2617 + - split: + apply_to_metric: system.network.dropped + by_attribute: direction + metrics_from_attributes: + system.network.dropped.receive: receive + system.network.dropped.transmit: transmit + # https://github.com/open-telemetry/opentelemetry-specification/pull/2617 + - split: + apply_to_metric: system.network.packets + by_attribute: direction + metrics_from_attributes: + system.network.packets.receive: receive + system.network.packets.transmit: transmit + # https://github.com/open-telemetry/opentelemetry-specification/pull/2617 + - split: + apply_to_metric: system.network.errors + by_attribute: direction + metrics_from_attributes: + system.network.errors.receive: receive + system.network.errors.transmit: transmit + # https://github.com/open-telemetry/opentelemetry-specification/pull/2617 + - split: + apply_to_metric: system.network.io + by_attribute: direction + metrics_from_attributes: + system.network.io.receive: receive + system.network.io.transmit: transmit + # https://github.com/open-telemetry/opentelemetry-specification/pull/2617 + - split: + apply_to_metric: process.disk.io + by_attribute: direction + metrics_from_attributes: + process.disk.io.read: read + process.disk.io.write: write + # https://github.com/open-telemetry/opentelemetry-specification/pull/2617 + - split: + apply_to_metric: process.network.io + by_attribute: direction + metrics_from_attributes: + process.network.io.receive: receive + process.network.io.transmit: transmit + + 1.12.0: + spans: + changes: + - rename_attributes: + attribute_map: + net.peer.ip: net.sock.peer.addr + net.host.ip: net.sock.host.addr + 1.11.0: + 1.10.0: + 1.9.0: + 1.8.0: + spans: + changes: + - rename_attributes: + attribute_map: + db.cassandra.keyspace: db.name + db.hbase.namespace: db.name + 1.7.0: + 1.6.1: + 1.5.0: + 1.4.0: From 39bacbbb15812126290b795efa0c44cbcf064b40 Mon Sep 17 00:00:00 2001 From: Foad Nosrati Habibi Date: Sat, 30 Jul 2022 04:31:23 +1000 Subject: [PATCH 4/7] Fix copy-paste error (#2699) There is an extra `}` character in the python method call. Probably it is there from copy-paste of the other method. Co-authored-by: Bogdan Drutu --- specification/metrics/api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index 102bd901f8b..af32b8f7683 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -709,7 +709,7 @@ API](../overview.md#api) authors might consider: # Python http_server_duration.Record(50, {"http.method": "POST", "http.scheme": "https"}) -http_server_duration.Record(100, http_method="GET", http_scheme="http"}) +http_server_duration.Record(100, http_method="GET", http_scheme="http") ``` ```csharp From 43ae6ed2d26d9770897aa0a961e934a6d4c658f1 Mon Sep 17 00:00:00 2001 From: Chengzhong Wu Date: Sat, 30 Jul 2022 13:29:43 +0800 Subject: [PATCH 5/7] Update JS metrics status in spec compliance matrix (#2662) --- spec-compliance-matrix.md | 176 +++++++++++++++++++------------------- 1 file changed, 88 insertions(+), 88 deletions(-) diff --git a/spec-compliance-matrix.md b/spec-compliance-matrix.md index 13f5317b066..cbcbcffc582 100644 --- a/spec-compliance-matrix.md +++ b/spec-compliance-matrix.md @@ -96,103 +96,103 @@ formats is required. Implementing more than one format is optional. | Feature | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift | |------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|----|------|----|--------|------|--------|-----|------|-----|------|-------| -| The API provides a way to set and get a global default `MeterProvider`. | X | + | + | | + | | | | | + | - | | -| It is possible to create any number of `MeterProvider`s. | X | + | + | | + | | | | | | + | | -| `MeterProvider` provides a way to get a `Meter`. | | + | + | | + | | | | | + | - | | -| `get_meter` accepts name, `version` and `schema_url`. | | + | + | | + | | | | | + | - | | +| The API provides a way to set and get a global default `MeterProvider`. | X | + | + | + | + | | | | | + | - | | +| It is possible to create any number of `MeterProvider`s. | X | + | + | + | + | | | | | | + | | +| `MeterProvider` provides a way to get a `Meter`. | | + | + | + | + | | | | | + | - | | +| `get_meter` accepts name, `version` and `schema_url`. | | + | + | + | + | | | | | + | - | | | `get_meter` accepts `attributes`. | | | | | | | | | | | | | -| When an invalid `name` is specified a working `Meter` implementation is returned as a fallback. | | + | + | | + | | | | | + | - | | -| The fallback `Meter` `name` property keeps its original invalid value. | X | - | - | | + | | | | | - | - | | -| Associate `Meter` with `InstrumentationScope`. | | | + | | + | | | | | | | | -| The meter provides functions to create a new `Counter`. | | + | + | | + | | | | | + | + | | -| The meter provides functions to create a new `AsynchronousCounter`. | | + | + | | + | | | | | + | + | | -| The meter provides functions to create a new `Histogram`. | | + | + | | + | | | | | + | + | | -| The meter provides functions to create a new `AsynchronousGauge`. | | + | + | | + | | | | | + | + | | -| The meter provides functions to create a new `UpDownCounter`. | | + | + | | + | | | | | + | - | | -| The meter provides functions to create a new `AsynchronousUpDownCounter`. | | + | + | | + | | | | | + | - | | -| Instruments have `name` | | + | + | | + | | | | | + | + | | -| Instruments have kind. | | + | + | | + | | | | | | + | | -| Instruments have an optional unit of measure. | | + | + | | + | | | | | + | + | | -| Instruments have an optional description. | | + | + | | + | | | | | + | + | | -| A valid instrument MUST be created and warning SHOULD be emitted when multiple instruments are registered under the same `Meter` using the same `name`. | | | + | | + | | | | | | | | -| It is possible to register two instruments with same `name` under different `Meter`s. | | + | + | | + | | | | | | + | | +| When an invalid `name` is specified a working `Meter` implementation is returned as a fallback. | | + | + | + | + | | | | | + | - | | +| The fallback `Meter` `name` property keeps its original invalid value. | X | - | - | + | + | | | | | - | - | | +| Associate `Meter` with `InstrumentationScope`. | | | + | + | + | | | | | | | | +| The meter provides functions to create a new `Counter`. | | + | + | + | + | | | | | + | + | | +| The meter provides functions to create a new `AsynchronousCounter`. | | + | + | + | + | | | | | + | + | | +| The meter provides functions to create a new `Histogram`. | | + | + | + | + | | | | | + | + | | +| The meter provides functions to create a new `AsynchronousGauge`. | | + | + | + | + | | | | | + | + | | +| The meter provides functions to create a new `UpDownCounter`. | | + | + | + | + | | | | | + | - | | +| The meter provides functions to create a new `AsynchronousUpDownCounter`. | | + | + | + | + | | | | | + | - | | +| Instruments have `name` | | + | + | + | + | | | | | + | + | | +| Instruments have kind. | | + | + | + | + | | | | | | + | | +| Instruments have an optional unit of measure. | | + | + | + | + | | | | | + | + | | +| Instruments have an optional description. | | + | + | + | + | | | | | + | + | | +| A valid instrument MUST be created and warning SHOULD be emitted when multiple instruments are registered under the same `Meter` using the same `name`. | | | + | + | + | | | | | | | | +| It is possible to register two instruments with same `name` under different `Meter`s. | | + | + | + | + | | | | | | + | | | Instrument names conform to the specified syntax. | | - | + | | + | | | | | | + | | | Instrument units conform to the specified syntax. | | - | + | | + | | | | | | + | | | Instrument descriptions conform to the specified syntax. | | - | + | | - | | | | | | + | | -| `create_counter` returns a `Counter`. | | + | + | | + | | | | | + | + | | -| The API for `Counter` accepts the name, unit and description of the instrument. | | + | + | | + | | | | | + | + | | -| `Counter` has an `add` method. | | + | + | | + | | | | | + | + | | -| The `add` method returns no (or dummy) value. | X | + | + | | + | | | | | + | + | | -| The `add` method accepts optional attributes. | | + | + | | + | | | | | + | + | | -| The `add` method accepts the increment amount. | | + | + | | + | | | | | + | + | | -| The `add` method of `Counter` accepts only positive amounts. | | + | + | | + | | | | | | - | | -| `create_asynchronous_counter` creates an `AsynchronousCounter`. | | + | + | | + | | | | | + | + | | -| The API for `AsynchronousCounter` accepts the name, unit and description of the instrument. | | + | + | | + | | | | | + | + | | -| The API for `AsynchronousCounter` accepts a callback. | | + | + | | + | | | | | + | + | | -| `create_up_down_counter` returns an `UpDownCounter`. | | + | + | | + | | | | | + | - | | -| The API for `UpDownCounter` accepts the name, unit and description of the instrument. | | + | + | | + | | | | | + | - | | -| `UpDownCounter` has an `add` method. | | + | + | | + | | | | | + | - | | -| The `add` method returns no (or dummy) value. | X | + | + | | + | | | | | + | - | | -| The `add` method accepts optional attributes. | | + | + | | + | | | | | + | - | | -| The `add` method accepts the increment amount. | | + | + | | + | | | | | + | - | | -| `create_asynchronous_up_down_counter` creates an `AsynchronousUpDownCounter`. | | + | + | | + | | | | | + | - | | -| The API for `AsynchronousUpDownCounter` accepts the name, unit and description of the instrument. | | + | + | | + | | | | | + | - | | -| The API for `AsynchronousUpDownCounter` accepts a callback. | | + | + | | + | | | | | + | - | | -| `create_histogram` returns a `Histogram`. | | + | + | | + | | | | | + | + | | -| The API for `Histogram` accepts the name, unit and description of the instrument. | | + | + | | + | | | | | + | + | | -| `Histogram` has a `record` method. | | + | + | | + | | | | | + | + | | -| The `record` method return no (or dummy) value. | X | + | + | | + | | | | | + | + | | -| The `record` method accepts optional attributes. | | + | + | | + | | | | | + | + | | -| The `record` method accepts a value. | | + | + | | + | | | | | + | + | | -| The `record` method of `Histogram` accepts only positive amounts. | | - | + | | + | | | | | | + | | -| `create_asynchronous_gauge` creates an `Asynchronous Gauge`. | | + | + | | + | | | | | + | + | | -| The API for `AsynchronousGauge` accepts the name, unit and description of the instrument. | | + | + | | + | | | | | + | + | | -| The API for `AsynchronousGauge` accepts a callback. | | + | + | | + | | | | | + | + | | -| The callback function of an `Asynchronous` instrument does not block indefinitely. | X | - | - | | - | | | | | | ? | | -| The callback function reports `Measurement`s. | | + | + | | + | | | | | + | + | | +| `create_counter` returns a `Counter`. | | + | + | + | + | | | | | + | + | | +| The API for `Counter` accepts the name, unit and description of the instrument. | | + | + | + | + | | | | | + | + | | +| `Counter` has an `add` method. | | + | + | + | + | | | | | + | + | | +| The `add` method returns no (or dummy) value. | X | + | + | + | + | | | | | + | + | | +| The `add` method accepts optional attributes. | | + | + | + | + | | | | | + | + | | +| The `add` method accepts the increment amount. | | + | + | + | + | | | | | + | + | | +| The `add` method of `Counter` accepts only positive amounts. | | + | + | + | + | | | | | | - | | +| `create_asynchronous_counter` creates an `AsynchronousCounter`. | | + | + | + | + | | | | | + | + | | +| The API for `AsynchronousCounter` accepts the name, unit and description of the instrument. | | + | + | + | + | | | | | + | + | | +| The API for `AsynchronousCounter` accepts a callback. | | + | + | + | + | | | | | + | + | | +| `create_up_down_counter` returns an `UpDownCounter`. | | + | + | + | + | | | | | + | - | | +| The API for `UpDownCounter` accepts the name, unit and description of the instrument. | | + | + | + | + | | | | | + | - | | +| `UpDownCounter` has an `add` method. | | + | + | + | + | | | | | + | - | | +| The `add` method returns no (or dummy) value. | X | + | + | + | + | | | | | + | - | | +| The `add` method accepts optional attributes. | | + | + | + | + | | | | | + | - | | +| The `add` method accepts the increment amount. | | + | + | + | + | | | | | + | - | | +| `create_asynchronous_up_down_counter` creates an `AsynchronousUpDownCounter`. | | + | + | + | + | | | | | + | - | | +| The API for `AsynchronousUpDownCounter` accepts the name, unit and description of the instrument. | | + | + | + | + | | | | | + | - | | +| The API for `AsynchronousUpDownCounter` accepts a callback. | | + | + | + | + | | | | | + | - | | +| `create_histogram` returns a `Histogram`. | | + | + | + | + | | | | | + | + | | +| The API for `Histogram` accepts the name, unit and description of the instrument. | | + | + | + | + | | | | | + | + | | +| `Histogram` has a `record` method. | | + | + | + | + | | | | | + | + | | +| The `record` method return no (or dummy) value. | X | + | + | + | + | | | | | + | + | | +| The `record` method accepts optional attributes. | | + | + | + | + | | | | | + | + | | +| The `record` method accepts a value. | | + | + | + | + | | | | | + | + | | +| The `record` method of `Histogram` accepts only positive amounts. | | - | + | + | + | | | | | | + | | +| `create_asynchronous_gauge` creates an `Asynchronous Gauge`. | | + | + | + | + | | | | | + | + | | +| The API for `AsynchronousGauge` accepts the name, unit and description of the instrument. | | + | + | + | + | | | | | + | + | | +| The API for `AsynchronousGauge` accepts a callback. | | + | + | + | + | | | | | + | + | | +| The callback function of an `Asynchronous` instrument does not block indefinitely. | X | - | - | + | - | | | | | | ? | | +| The callback function reports `Measurement`s. | | + | + | + | + | | | | | + | + | | | There is a way to pass state to the callback. | X | + | + | | + | | | | | | + | | -| All methods of `MeterProvider` are safe to be called concurrently. | | + | + | | - | | | | | + | + | | -| All methods of `Meter` are safe to be called concurrently. | | + | + | | - | | | | | | + | | -| All methods of any instrument are safe to be called concurrently. | | + | + | | - | | | | | | + | | -| `MeterProvider` allows a `Resource` to be specified. | | + | + | | + | | | | | + | + | | -| A specified `Resource` can be associated with all the produced metrics from any `Meter` from the `MeterProvider`. | | + | + | | + | | | | | + | + | | +| All methods of `MeterProvider` are safe to be called concurrently. | | + | + | + | - | | | | | + | + | | +| All methods of `Meter` are safe to be called concurrently. | | + | + | + | - | | | | | | + | | +| All methods of any instrument are safe to be called concurrently. | | + | + | + | - | | | | | | + | | +| `MeterProvider` allows a `Resource` to be specified. | | + | + | + | + | | | | | + | + | | +| A specified `Resource` can be associated with all the produced metrics from any `Meter` from the `MeterProvider`. | | + | + | + | + | | | | | + | + | | | The supplied `name`, `version` and `schema_url` arguments passed to the `MeterProvider` are used to create an `InstrumentationLibrary` instance stored in the `Meter`. | | + | - | | + | | | | | + | - | | -| The supplied `name`, `version` and `schema_url` arguments passed to the `MeterProvider` are used to create an `InstrumentationScope` instance stored in the `Meter`. | | | + | | | | | | | | | | -| Configuration is managed solely by the `MeterProvider`. | | + | + | | + | | | | | + | + | | -| The `MeterProvider` provides methods to update the configuration | X | - | - | | + | | | | | | + | | -| The updated configuration applies to all already returned `Meter`s. | if above | - | - | | - | | | | | | + | | -| There is a way to register `View`s with a `MeterProvider`. | | - | + | | + | | | | | + | + | | -| The `View` instrument selection criteria is as specified. | | | + | | + | | | | | + | + | | -| The `View` instrument selection criteria supports wildcards. | X | | + | | + | | | | | + | + | | -| The `View` instrument selection criteria supports the match-all wildcard. | | | + | | + | | | | | + | + | | -| The name of the `View` can be specified. | | | + | | + | | | | | + | + | | -| The `View` allows configuring the name description, attributes keys and aggregation of the resulting metric stream. | | | + | | + | | | | | + | - | | +| The supplied `name`, `version` and `schema_url` arguments passed to the `MeterProvider` are used to create an `InstrumentationScope` instance stored in the `Meter`. | | | + | + | | | | | | | | | +| Configuration is managed solely by the `MeterProvider`. | | + | + | + | + | | | | | + | + | | +| The `MeterProvider` provides methods to update the configuration | X | - | - | - | + | | | | | | + | | +| The updated configuration applies to all already returned `Meter`s. | if above | - | - | - | - | | | | | | + | | +| There is a way to register `View`s with a `MeterProvider`. | | - | + | + | + | | | | | + | + | | +| The `View` instrument selection criteria is as specified. | | | + | + | + | | | | | + | + | | +| The `View` instrument selection criteria supports wildcards. | X | | + | + | + | | | | | + | + | | +| The `View` instrument selection criteria supports the match-all wildcard. | | | + | + | + | | | | | + | + | | +| The name of the `View` can be specified. | | | + | + | + | | | | | + | + | | +| The `View` allows configuring the name description, attributes keys and aggregation of the resulting metric stream. | | | + | + | + | | | | | + | - | | | The `View` allows configuring the exemplar reservoir of resulting metric stream. | X | | - | | - | | | | | | - | | -| The SDK allows more than one `View` to be specified per instrument. | X | | + | | + | | | | | + | + | | -| The `Drop` aggregation is available. | | + | + | | - | | | | | + | + | | -| The `Drop` aggregation drops all measurements and does not produce a metric stream. | | + | + | | - | | | | | | + | | -| The `Default` aggregation is available. | | + | + | | - | | | | | + | + | | -| The `Default` aggregation uses the specified aggregation by instrument. | | + | + | | - | | | | | + | + | | -| The `Sum` aggregation is available. | | + | + | | + | | | | | + | + | | -| The `Sum` aggregation performs as specified. | | + | + | | + | | | | | + | + | | -| The `LastValue` aggregation is available. | | + | + | | + | | | | | + | + | | -| The `LastValue` aggregation performs as specified. | | + | + | | + | | | | | + | + | | -| The `Histogram` aggregation is available. | | + | + | | + | | | | | + | + | | +| The SDK allows more than one `View` to be specified per instrument. | X | | + | + | + | | | | | + | + | | +| The `Drop` aggregation is available. | | + | + | + | - | | | | | + | + | | +| The `Drop` aggregation drops all measurements and does not produce a metric stream. | | + | + | + | - | | | | | | + | | +| The `Default` aggregation is available. | | + | + | + | - | | | | | + | + | | +| The `Default` aggregation uses the specified aggregation by instrument. | | + | + | + | - | | | | | + | + | | +| The `Sum` aggregation is available. | | + | + | + | + | | | | | + | + | | +| The `Sum` aggregation performs as specified. | | + | + | + | + | | | | | + | + | | +| The `LastValue` aggregation is available. | | + | + | + | + | | | | | + | + | | +| The `LastValue` aggregation performs as specified. | | + | + | + | + | | | | | + | + | | +| The `Histogram` aggregation is available. | | + | + | + | + | | | | | + | + | | | The `Histogram` aggregation performs as specified. | | + | + | | + | | | | | + | + | | -| The explicit bucket `Histogram` aggregation is available. | | - | + | | + | | | | | | + | | -| The explicit bucket `Histogram` aggregation performs as specified. | | - | + | | + | | | | | | + | | -| The metrics Reader implementation supports registering metric Exporters | | | + | | + | | | | | | + | | +| The explicit bucket `Histogram` aggregation is available. | | - | + | + | + | | | | | | + | | +| The explicit bucket `Histogram` aggregation performs as specified. | | - | + | + | + | | | | | | + | | +| The metrics Reader implementation supports registering metric Exporters | | | + | + | + | | | | | | + | | | The metrics Reader implementation supports configuring the default aggregation on the basis of instrument kind. | | | - | | + | | | | | | - | | -| The metrics Reader implementation supports configuring the default temporality on the basis of instrument kind. | | | + | | + | | | | | | | | -| The metrics Exporter has access to the aggregated metrics data (aggregated points, not raw measurements). | | + | + | | + | | | | | | + | | -| The metrics Exporter `export` function can not be called concurrently from the same Exporter instance. | | + | + | | - | | | | | | + | | -| The metrics Exporter `export` function does not block indefinitely. | | + | + | | - | | | | | + | + | | -| The metrics Exporter `export` function receives a batch of metrics. | | + | + | | + | | | | | + | + | | -| The metrics Exporter `export` function returns `Success` or `Failure`. | | + | + | | + | | | | | + | + | | -| The metrics Exporter provides a `ForceFlush` function. | | - | + | | - | | | | | + | + | | -| The metrics Exporter `ForceFlush` can inform the caller whether it succeeded, failed or timed out. | | | + | | - | | | | | + | + | | -| The metrics Exporter provides a `shutdown` function. | | + | + | | + | | | | | + | + | | -| The metrics Exporter `shutdown` function do not block indefinitely. | | + | + | | + | | | | | + | + | | +| The metrics Reader implementation supports configuring the default temporality on the basis of instrument kind. | | | + | + | + | | | | | | | | +| The metrics Exporter has access to the aggregated metrics data (aggregated points, not raw measurements). | | + | + | + | + | | | | | | + | | +| The metrics Exporter `export` function can not be called concurrently from the same Exporter instance. | | + | + | + | - | | | | | | + | | +| The metrics Exporter `export` function does not block indefinitely. | | + | + | + | - | | | | | + | + | | +| The metrics Exporter `export` function receives a batch of metrics. | | + | + | + | + | | | | | + | + | | +| The metrics Exporter `export` function returns `Success` or `Failure`. | | + | + | + | + | | | | | + | + | | +| The metrics Exporter provides a `ForceFlush` function. | | - | + | + | - | | | | | + | + | | +| The metrics Exporter `ForceFlush` can inform the caller whether it succeeded, failed or timed out. | | | + | + | - | | | | | + | + | | +| The metrics Exporter provides a `shutdown` function. | | + | + | + | + | | | | | + | + | | +| The metrics Exporter `shutdown` function do not block indefinitely. | | + | + | + | + | | | | | + | + | | | The metrics SDK samples `Exemplar`s from measurements. | | | + | | - | | | | | | - | | | Exemplar sampling can be disabled. | | | - | | - | | | | | | - | | | The metrics SDK samples measurements in the context of a sampled trace by default. | | | + | | - | | | | | | - | | From 1ef7871c02170f07ce6e8bf3ec16c1d88cdb65a4 Mon Sep 17 00:00:00 2001 From: Ted Young Date: Sat, 30 Jul 2022 00:44:47 -0500 Subject: [PATCH 6/7] Specification project management proposal (#2621) --- .github/ISSUE_TEMPLATE/project.md | 51 ++++++++++++++++ specification/project-management.md | 95 +++++++++++++++++++++++++++++ 2 files changed, 146 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE/project.md create mode 100644 specification/project-management.md diff --git a/.github/ISSUE_TEMPLATE/project.md b/.github/ISSUE_TEMPLATE/project.md new file mode 100644 index 00000000000..de98cac6cce --- /dev/null +++ b/.github/ISSUE_TEMPLATE/project.md @@ -0,0 +1,51 @@ +--- +name: Project Tracking Issue +about: Proposals for large-scale specification work which requires a group effort. +labels: spec:project-tracking +--- + +### Description + +Describe the goals, objectives, and requirements for the project. This include the motivations for starting the project now, as opposed to later. + +### Project Board + +Once approved by TC, a project should be managed using a GitHub project board. This project board should be pre-populated with issues that cover all known deliverables, organized by timeline milestones. + +A [Technical Committee](https://github.com/open-telemetry/community/blob/main/community-members.md#technical-committee) (TC) member associated with the project can create the board, along with a new project-specific GitHub label to automatically associate issues and PRs with the project. The project lead and all other relevant project members should have edit access to the board. + +Once created, please link to the project board here. + +### Deliverables + +A description of what this project is planning to deliver, or is in the process of delivering. This includes all OTEPs and their associated prototypes. + +In general, OTEPs are not accepted unless they come with working prototypes available to review in at least two languages. Please discuss these requirements with a TC member before submitting an OTEP. + +### Staffing / Help Wanted + +Who is currently planning to work on the project? If a project requires specialized domain expertise, please list it here. If a project is missing a critical mass of people in order to begin work, please clarify. + +#### Required staffing + +Projects cannot be started until they the following participants have been identified: +* Every project needs a project lead, who is willing to bottom line the project and address any issues which are not handled by other project members. +* At least two sponsoring TC members. TC sponsors are dedicated to attending meetings, reviewing proposals, and in general being aware of the state of the project and it's technical details. TC sponsors guide the project through the spec process, keep the tracking issue up to date, and help to ensure that relevant community members provide input at the appropriate times. +* Engineers willing to write prototypes in at least two languages (if relevant to project). Languages should be fairly different from each other (for example: Java and Python). +* Maintainers or approvers from those languages committed to reviewing the prototypes. + +### Meeting Times + +Once a project is started, the working group should meet regularly for discussion. These meeting times should be posted on the OpenTelemetry public calendar. + +### Timeline + +What is the expected timeline the project will aim to adhere to, and what resources and deliverables will be needed for each portion of the timeline? If the project has not been started, please describe this timeline in relative terms (one month in, two weeks later, etc). If a project has started, please include actual dates. + +### Labels + +The tracking issue should be properly labeled to indicate what parts of the specification it is focused on. + +### Linked Issues and PRs + +All PRs, Issues, and OTEPs related to the project should link back to the tracking issue, so that they can be easily found. \ No newline at end of file diff --git a/specification/project-management.md b/specification/project-management.md new file mode 100644 index 00000000000..fdc211ae7ac --- /dev/null +++ b/specification/project-management.md @@ -0,0 +1,95 @@ +# Project Management + +Some specification changes are small enough in scope such that they can be resolved with a single PR or an OTEP written by a single person. However, this is rarely the case for large, meaningful changes. Most spec work ends up being organized into projects. + +At minimum, specification projects require the following resources to be successful: + +* A group of designers and subject matter experts need to dedicate a significant amount of their work time to the project. These participants are needed to design the spec, write a set of OTEPs, and create multiple prototypes. This group needs to meet with each other (and with TC members) on a regular basis to develop a successful set of proposals. +* A portion of the TC needs to be aware of and participate in the development of the project, to review the proposals and help guide the participants through the specification process. +* Spec approvers and the broader community need to be aware of progress being made on the project, so they can be prepared to participate when proposals are ready for review. + +## Project Tracking Issue + +Every project has a high level **Project Tracking Issue**, which describes the project. This issue is frequently edited and kept up to date by the working group. To create a tracking issue, please use the Project Tracking Issue template. + +The project tracking issue must contain the following information: + +### Description + +Describes the goals, objectives, and requirements for the project. This include the motivations for starting the project now, as opposed to later. + +### Project Board + +Projects should be managed using a github project board. The project board should be pre-populated with issues that cover all known deliverables, organized by timeline milestones. The project board should be linked to in the tracking issue. + +A TC member associated with the project can create the board, along with a new github label to automatically associate issues and PRs with the project. The project lead and all other relevant project members should have edit access to the board. +The project lead is responsible for maintaining the board and keeping it up-to-date. + +### Deliverables + +A description of what this project is planning to deliver, or is in the process of delivering. This includes all OTEPs and their associated prototypes. + +In general, OTEPs are not accepted unless they come with working prototypes available to review in at least two languages. Please discuss these requirements with a TC member before submitting an OTEP. + +### Staffing / Help Wanted + +Who is currently planning to work on the project? If a project requires specialized domain expertise, please list it here. If a project is missing a critical mass of people in order to begin work, please clarify. + +#### Required staffing + +Projects cannot be started until the following participants have been identified: + +* Every project needs a project lead, who is willing to bottom line the project and address any issues which are not handled by other project members. +* At least two sponsoring TC members. TC sponsors are dedicated to attending meetings, reviewing proposals, and in general being aware of the state of the project and it’s technical details. TC sponsors guide the project through the spec process, keep the tracking issue up to date, and help to ensure that relevant community members provide input at the appropriate times. +* Engineers willing to write prototypes in at least two languages (if relevant to project). Languages should be fairly different from each other (for example. Java and Python). +* Maintainers or approvers from those languages committed to reviewing the prototypes. + +### Meeting Times + +Once a project is started, the working group should meet regularly for discussion. These meeting times should be posted on the OpenTelemetry public calendar. + +### Timeline + +What is the expected timeline the project will aim to adhere to, and what resources and deliverables will be needed for each portion of the timeline? If the project has not been started, please describe this timeline in relative terms (one month in, two weeks later, etc). If a project has started, please include actual dates. + +### Labels + +The tracking issue should be properly labeled to indicate what parts of the specification it is focused on. + +### Linked Issues and PRs + +All PRs, Issues, and OTEPs related to the project should link back to the tracking issue, so that they can be easily found. + +## Specification Project Lifecycle + +All specification projects have the same lifecycle, and are tracked on the Specification Project Board, which the community can use to get a high-level view of the specification roadmap. + +The project lifecycle is as follows: + +* A **Project Tracking Issue** is created. The tracking issue includes all the necessary information for the TC and spec community to evaluate the breadth and depth of the work being proposed. +* If a project is approved, it is added to the list of **Potential Projects**. This list is roughly ordered in the order we expect we will start the project. +* Potential projects are moved to the list of **Scheduled Projects** once they have a planned start date. Having a planned start date lets potential contributors know when they need to make themselves available, and get prepared to begin their work. Subject matter experts and participants who plan to do a lot of work – such as building prototypes – benefit greatly from having a start date, as they can plan for their participantion with their employers and coworkers. +* Once a project is begun, it is moved to the list of **Current Projects**. Projects are only started when the necessary resources are available to move them quickly to completion. This means that the necessary subject matter experts have been identified, and at least two TC members are committed to review and guide the project through the specification process. +* Once all OTEPs have been approved and integrated into the spec, and the working group is no longer meeting, projects are moved to **Completed Projects**. + +## Project Board + +To track our specification projects, we use a GitHub project board. This board only contains Project Tracking Issues, and is organized into the following columns: + +### Current Projects + +Projects that are actively being moved to completion. Projects may be in one of several different states: design, proposal review, and implementation. + +### Scheduled Projects + +Many projects require people (such as subject matter experts) and other resources whose participation/availability must be planned out in advance. In general, projects may be able to move faster when their start date is scheduled and known in advance, so participants can prepare their schedules and do preliminary research. + +Scheduled projects are projects which have not started yet, but have a scheduled start date. + +### Potential Projects + +Any project which has a tracking issue and has been approved by the TC as a needed feature for OpenTelemetry. Roughly organized by priority. + +### Completed Projects + +Projects which have been successfully implemented, and no longer need any attention beyond responding to user feedback. From dbea54d2a416fe9e487d78dd33f8ab9251596bc2 Mon Sep 17 00:00:00 2001 From: Joao Grassi Date: Tue, 2 Aug 2022 18:08:08 +0200 Subject: [PATCH 7/7] Add support for partial success in an OTLP export response [2] (#2696) --- CHANGELOG.md | 3 + specification/protocol/otlp.md | 127 ++++++++++++++++++++++++++------- 2 files changed, 104 insertions(+), 26 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c36531d0c22..e586aa94f0f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -66,6 +66,9 @@ release. ### OpenTelemetry Protocol +- Add support for partial success in an OTLP export response + ([#2696](https://github.com/open-telemetry/opentelemetry-specification/pull/2696)). + ### SDK Configuration - Mark `OTEL_METRIC_EXPORT_INTERVAL`, `OTEL_METRIC_EXPORT_TIMEOUT` diff --git a/specification/protocol/otlp.md b/specification/protocol/otlp.md index 42766e721c6..e8d548a4994 100644 --- a/specification/protocol/otlp.md +++ b/specification/protocol/otlp.md @@ -16,14 +16,18 @@ nodes such as collectors and telemetry backends. * [OTLP/gRPC](#otlpgrpc) + [OTLP/gRPC Concurrent Requests](#otlpgrpc-concurrent-requests) + [OTLP/gRPC Response](#otlpgrpc-response) + - [Full Success](#full-success) + - [Partial Success](#partial-success) + - [Failures](#failures) + [OTLP/gRPC Throttling](#otlpgrpc-throttling) + [OTLP/gRPC Service and Protobuf Definitions](#otlpgrpc-service-and-protobuf-definitions) + [OTLP/gRPC Default Port](#otlpgrpc-default-port) * [OTLP/HTTP](#otlphttp) + [OTLP/HTTP Request](#otlphttp-request) + [OTLP/HTTP Response](#otlphttp-response) - - [Success](#success) - - [Failures](#failures) + - [Full Success](#full-success-1) + - [Partial Success](#partial-success-1) + - [Failures](#failures-1) - [Bad Data](#bad-data) - [OTLP/HTTP Throttling](#otlphttp-throttling) - [All Other Responses](#all-other-responses) @@ -35,7 +39,6 @@ nodes such as collectors and telemetry backends. - [Known Limitations](#known-limitations) * [Request Acknowledgements](#request-acknowledgements) + [Duplicate Data](#duplicate-data) - * [Partial Success](#partial-success) - [Future Versions and Interoperability](#future-versions-and-interoperability) - [Glossary](#glossary) - [References](#references) @@ -145,16 +148,57 @@ was not delivered. #### OTLP/gRPC Response -The server may respond with either a success or an error to the requests. +The response MUST be the appropriate message (see below for +the specific message to use in the [Full Success](#full-success), +[Partial Success](#partial-success) and [Failure](#failures) cases). + +##### Full Success -The success response indicates telemetry data is successfully processed by the -server. If the server receives an empty request (a request that does not carry +The success response indicates telemetry data is successfully accepted by the +server. + +If the server receives an empty request (a request that does not carry any telemetry data) the server SHOULD respond with success. -Success response is returned via -[Export*ServiceResponse](https://github.com/open-telemetry/opentelemetry-proto) -message (`ExportTraceServiceResponse` for traces, `ExportMetricsServiceResponse` -for metrics, `ExportLogsServiceResponse` for logs). +On success, the server response MUST be a +[ExportServiceResponse](https://github.com/open-telemetry/opentelemetry-proto/tree/main/opentelemetry/proto/collector) +message (`ExportTraceServiceResponse` for traces, +`ExportMetricsServiceResponse` for metrics and +`ExportLogsServiceResponse` for logs). + +The server MUST leave the `partial_success` field unset +in case of a successful response. + +##### Partial Success + +If the request is only partially accepted +(i.e. when the server accepts only parts of the data and rejects the rest), the +server response MUST be the same +[ExportServiceResponse](https://github.com/open-telemetry/opentelemetry-proto/tree/main/opentelemetry/proto/collector) +message as in the [Full Success](#full-success) case. + +Additionally, the server MUST initialize the `partial_success` field +(`ExportTracePartialSuccess` message for traces, +`ExportMetricsPartialSuccess` message for metrics and +`ExportLogsPartialSuccess` message for logs), and it MUST set the respective +`rejected_spans`, `rejected_data_points` or `rejected_log_records` field with +the number of spans/data points/log records it rejected. + +The server SHOULD populate the `error_message` field with a human-readable +error message in English. The message should explain why the +server rejected parts of the data, and might offer guidance on how users +can address the issues. +The protocol does not attempt to define the structure of the error message. + +Servers MAY also make use of the `partial_success` field to convey +warnings/suggestions to clients even when the request was fully accepted. +In such cases, the `rejected_` field MUST have a value of `0` and +the `error_message` field MUST be non-empty. + +The client MUST NOT retry the request when it receives a partial success +response where the `partial_success` is populated. + +##### Failures When an error is returned by the server it falls into 2 broad categories: retryable and not-retryable: @@ -382,8 +426,9 @@ numbers or strings are accepted when decoding. #### OTLP/HTTP Response -Response body MUST be the appropriate serialized Protobuf message (see below for -the specific message to use in the Success and Failure cases). +The response body MUST be the appropriate serialized Protobuf message (see below for +the specific message to use in the [Full Success](#full-success-1), +[Partial Success](#partial-success-1) and [Failure](#failures-1) cases). The server MUST set "Content-Type: application/x-protobuf" header if the response body is binary-encoded Protobuf payload. The server MUST set @@ -395,15 +440,52 @@ If the request header "Accept-Encoding: gzip" is present in the request the server MAY gzip-encode the response and set "Content-Encoding: gzip" response header. -##### Success +##### Full Success -On success the server MUST respond with `HTTP 200 OK`. Response body MUST be -Protobuf-encoded `ExportTraceServiceResponse` message for traces, -`ExportMetricsServiceResponse` message for metrics and -`ExportLogsServiceResponse` message for logs. +The success response indicates telemetry data is successfully accepted by the +server. -The server SHOULD respond with success no sooner than after successfully -decoding and validating the request. +If the server receives an empty request (a request that does not carry +any telemetry data) the server SHOULD respond with success. + +On success, the server MUST respond with `HTTP 200 OK`. The response body MUST be +a Protobuf-encoded +[ExportServiceResponse](https://github.com/open-telemetry/opentelemetry-proto/tree/main/opentelemetry/proto/collector) +message (`ExportTraceServiceResponse` for traces, +`ExportMetricsServiceResponse` for metrics and +`ExportLogsServiceResponse` for logs). + +The server MUST leave the `partial_success` field unset +in case of a successful response. + +##### Partial Success + +If the request is only partially accepted +(i.e. when the server accepts only parts of the data and rejects the rest), the +server MUST respond with `HTTP 200 OK`. The response body MUST be the same +[ExportServiceResponse](https://github.com/open-telemetry/opentelemetry-proto/tree/main/opentelemetry/proto/collector) +message as in the [Full Success](#full-success-1) case. + +Additionally, the server MUST initialize the `partial_success` field +(`ExportTracePartialSuccess` message for traces, +`ExportMetricsPartialSuccess` message for metrics and +`ExportLogsPartialSuccess` message for logs), and it MUST set the respective +`rejected_spans`, `rejected_data_points` or `rejected_log_records` field with +the number of spans/data points/log records it rejected. + +The server SHOULD populate the `error_message` field with a human-readable +error message in English. The message should explain why the +server rejected parts of the data, and might offer guidance on how users +can address the issues. +The protocol does not attempt to define the structure of the error message. + +Servers MAY also make use of the `partial_success` field to convey +warnings/suggestions to clients even when the request was fully accepted. +In such cases, the `rejected_` field MUST have a value of `0` and +the `error_message` field MUST be non-empty. + +The client MUST NOT retry the request when it receives a partial success +response where the `partial_success` is populated. ##### Failures @@ -520,13 +602,6 @@ received yet. The client will typically choose to re-send such data to guarantee delivery, which may result in duplicate data on the server side. This is a deliberate choice and is considered to be the right tradeoff for telemetry data. -### Partial Success - -The protocol does not attempt to communicate partial reception success from the -server to the client (i.e. when part of the data can be received by the server -and part of it cannot). Attempting to do so would complicate the protocol and -implementations significantly and is left out as a possible future area of work. - ## Future Versions and Interoperability OTLP will evolve and change over time. Future versions of OTLP must be designed