Skip to content
This repository has been archived by the owner on Jul 25, 2024. It is now read-only.

Rename OneAgent metadata to Dynatrace Metadata #19

Merged
merged 6 commits into from
Jul 29, 2021
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Upload Python Package to PyPI

on:
release:
types: [published]
types: [ published ]

jobs:
publish:
Expand Down
44 changes: 22 additions & 22 deletions .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,33 +11,33 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.8
- name: Install Dependencies
run: |
pip install -U pip
pip install -U wheel setuptools tox
- run: tox -e lint
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.8
- name: Install Dependencies
run: |
pip install -U pip
pip install -U wheel setuptools tox
- run: tox -e lint

build-and-test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: [ '3.6', '3.7', '3.8', '3.9', 'pypy3']
python-version: [ '3.6', '3.7', '3.8', '3.9', 'pypy3' ]

steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install Dependencies
run: |
pip install -U pip
pip install -U wheel setuptools tox
- name: Test
run: tox -e py
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install Dependencies
run: |
pip install -U pip
pip install -U wheel setuptools tox
- name: Test
run: tox -e py
36 changes: 22 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,14 @@

## Getting started

### Installation

To install the [latest version from PyPI](https://pypi.org/project/opentelemetry-exporter-dynatrace-metrics/) run:

```shell
pip install opentelemetry-exporter-dynatrace-metrics
```

### Usage

The general setup of OpenTelemetry Python is explained in the official [Getting Started Guide](https://opentelemetry-python.readthedocs.io/en/stable/getting-started.html).
Expand Down Expand Up @@ -35,8 +43,8 @@ counter.add(25, {"dimension-1", "value-1"})
To run the [example](example/basic_example.py), clone this repository and change to the `opentelemetry-metric-python` folder, then run:

```shell
pip install . # install the Dynatrace exporter
export LOGLEVEL=DEBUG # (optional) Set the log level to debug to see more output (default is INFO)
pip install . # install the Dynatrace exporter
export LOGLEVEL=DEBUG # (optional) Set the log level to debug to see more output (default is INFO)
python example/basic_example.py
```

Expand Down Expand Up @@ -80,29 +88,29 @@ The `prefix` parameter specifies an optional prefix, which is prepended to each
The `default_dimensions` parameter can be used to optionally specify a list of key/value pairs, which will be added as additional dimensions to all data points.
Dimension keys are unique, and labels on instruments will overwrite the default dimensions if key collisions appear.

#### Export OneAgent Metadata
#### Export Dynatrace Metadata

If running on a host with a running OneAgent, setting the `export_oneagent_metadata` option to `True` will export metadata collected by the OneAgent to the Dynatrace endpoint.
If running on a host with a running OneAgent or a Dynatrace Operator, setting the `export_dynatrace_metadata` option to `True` will export metadata collected by the OneAgent to the Dynatrace endpoint.
pirgeo marked this conversation as resolved.
Show resolved Hide resolved
If no Dynatrace API endpoint is set, the default exporter endpoint will be the OneAgent endpoint, and this option will be set automatically.
Therefore, if no endpoint is specified, we assume a OneAgent is running and export to it, including metadata.
More information on the underlying OneAgent feature that is used by the exporter can be found in the
Therefore, if no endpoint is specified, a OneAgent is assumed to be running and used as the export endpoint for all metric lines, including metadata.
More information on the underlying Dynatrace metadata feature that is used by the exporter can be found in the
[Dynatrace documentation](https://www.dynatrace.com/support/help/how-to-use-dynatrace/metrics/metric-ingestion/ingestion-methods/enrich-metrics/).

##### Dimensions precedence

When specifying default dimensions, labels and OneAgent metadata enrichment, the precedence of dimensions with the same key is as follows:
Default dimensions are overwritten by labels passed to instruments, which in turn are overwritten by the OneAgent dimensions (even though the likeliness of a collision here is very low, since the OneAgent metadata only contains [Dynatrace reserved dimensions](https://www.dynatrace.com/support/help/how-to-use-dynatrace/metrics/metric-ingestion/metric-ingestion-protocol/#syntax) starting with `dt.*`).
When specifying default dimensions, labels and Dynatrace metadata enrichment, the precedence of dimensions with the same key is as follows:
Default dimensions are overwritten by labels passed to instruments, which in turn are overwritten by the Dynatrace metadata dimensions (even though the likeliness of a collision here is very low, since the Dynatrace metadata only contains [Dynatrace reserved dimensions](https://www.dynatrace.com/support/help/how-to-use-dynatrace/metrics/metric-ingestion/metric-ingestion-protocol/#syntax) starting with `dt.*`).

## Development

### Requirements

Just [`tox`](https://pypi.org/project/tox/)
Just [`tox`](https://pypi.org/project/tox/).

### Running tests and lint

*Test all supported python versions:* `tox`
*Test all supported python versions in parallel:* `tox -p`
*A particular python version:* `tox -e 38`
*Current python version*: `tox -e py`
*Lint*: `tox -e lint`
* Test all supported python versions: `tox`
* Test all supported python versions in parallel: `tox -p`
* A particular python version: `tox -e 38`
* Current python version: `tox -e py`
* Lint: `tox -e lint`
19 changes: 10 additions & 9 deletions example/basic_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,18 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import random
import time

from dynatrace.opentelemetry.metrics.export import DynatraceMetricsExporter
from opentelemetry import metrics
from opentelemetry.sdk.metrics import MeterProvider

from os.path import splitext, basename
import argparse
import logging
import os
import psutil
import random
import time


# Callback to gather cpu usage
Expand Down Expand Up @@ -59,10 +60,11 @@ def parse_arguments():

parser.add_argument("-nm", "--no-metadata", dest="metadata_enrichment",
action="store_false",
help="Turn off OneAgent Metadata enrichment. If no "
"OneAgent is running on the machine, this is "
"ignored. Otherwise, OneAgent metadata will be "
"added to each of the exported metric lines.")
help="Turn off Dynatrace Metadata enrichment. If no "
"OneAgent or Dynatrace operator is running on "
pirgeo marked this conversation as resolved.
Show resolved Hide resolved
"the host, this is ignored. Otherwise, Dynatrace "
"metadata will be added to each of the exported "
"metric lines.")

parser.add_argument("-i", "--interval", default=10., type=float,
dest="interval",
Expand Down Expand Up @@ -101,8 +103,8 @@ def parse_arguments():
logger.info("setting up Dynatrace metrics exporting interface.")
exporter = DynatraceMetricsExporter(args.endpoint, args.token,
prefix="otel.python",
export_oneagent_metadata=args
.metadata_enrichment)
export_dynatrace_metadata=
args.metadata_enrichment)

logger.info("registering Dynatrace exporter with the global OpenTelemetry"
" instance...")
Expand Down Expand Up @@ -163,6 +165,5 @@ def parse_arguments():
requests_size.record(random.randint(0, 100), testing_labels)
time.sleep(5)


except KeyboardInterrupt:
logger.info("shutting down...")
12 changes: 6 additions & 6 deletions example/install_and_run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ else
fi

# change into the opentelemetry-metric-python folder if you haven't already
python3 -m venv .venv `# create a new virtual environment in the current folder`
python3 -m venv .venv `# create a new virtual environment in the current folder`

source .venv/bin/activate
pip3 install --upgrade setuptools `# make sure setuptools and wheel are on the latest version`
pip3 install psutil `# for observing cpu and ram`
pip3 install . `# install the library itself`
pip3 install --upgrade setuptools `# make sure setuptools and wheel are on the latest version`
pip3 install psutil `# for observing cpu and ram`
pip3 install . `# install the library itself`
# Valid log levels are: DEBUG, INFO, WARN/WARNING, ERROR, CRITICAL/FATAL
export LOGLEVEL=DEBUG `# set the log level`
python3 example/basic_example.py `# run the example in a venv`
export LOGLEVEL=DEBUG `# set the log level`
python3 example/basic_example.py `# run the example in a venv`
20 changes: 11 additions & 9 deletions src/dynatrace/opentelemetry/metrics/export/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@
)

from .serializer import DynatraceMetricsSerializer
from .oneagentmetadataenricher import OneAgentMetadataEnricher
from .dynatracemetadataenricher import DynatraceMetadataEnricher

VERSION = "0.1.0b0"
VERSION = "0.1.0b1"


class DynatraceMetricsExporter(MetricsExporter):
Expand All @@ -43,7 +43,7 @@ def __init__(
api_token: Optional[str] = None,
prefix: Optional[str] = None,
default_dimensions: Optional[Mapping[str, str]] = None,
export_oneagent_metadata: Optional[bool] = False,
export_dynatrace_metadata: Optional[bool] = False,
):
self.__logger = logging.getLogger(__name__)

Expand All @@ -54,15 +54,16 @@ def __init__(
"to default local OneAgent ingest endpoint.")
self._endpoint_url = "http://localhost:14499/metrics/ingest"

oneagent_dims = {}
dynatrace_metadata_dims = {}

if export_oneagent_metadata:
enricher = OneAgentMetadataEnricher()
enricher.add_oneagent_metadata_to_dimensions(oneagent_dims)
if export_dynatrace_metadata:
enricher = DynatraceMetadataEnricher()
enricher.add_dynatrace_metadata_to_dimensions(
dynatrace_metadata_dims)

self._serializer = DynatraceMetricsSerializer(prefix,
default_dimensions,
oneagent_dims)
dynatrace_metadata_dims)
self._session = requests.Session()
self._headers = {
"Accept": "*/*; q=0",
Expand Down Expand Up @@ -111,7 +112,8 @@ def export(
headers=self._headers,
) as resp:
resp.raise_for_status()
self.__logger.debug("got response: " + resp.content)
self.__logger.debug("got response: " +
resp.content.decode("utf-8"))
except Exception as ex:
self.__logger.warning("Failed to export metrics: %s", ex)
return MetricsExportResult.FAILURE
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,13 @@
from typing import List, Mapping


class OneAgentMetadataEnricher:
class DynatraceMetadataEnricher:
def __init__(self) -> None:
self.__logger = logging.getLogger(__name__)

def add_oneagent_metadata_to_dimensions(self, tags: Mapping[str, str]):
def add_dynatrace_metadata_to_dimensions(self, tags: Mapping[str, str]):
metadata_file_content = self._get_metadata_file_content()
parsed_metadata = self._parse_oneagent_metadata(metadata_file_content)
parsed_metadata = self._parse_dynatrace_metadata(metadata_file_content)
for key, value in parsed_metadata.items():
tags[key] = value

Expand All @@ -36,11 +36,11 @@ def _get_metadata_file_name(self, indirection_fname: str) -> str:
file_name = metadata_indirection_file.read()

if not file_name:
self.__logger.warning("OneAgent metadata file not specified "
self.__logger.warning("Dynatrace metadata file not specified "
"in indirection file.")

except OSError:
self.__logger.warning("Could not read local OneAgent metadata "
self.__logger.warning("Could not read local Dynatrace metadata "
"enrichment file. This is normal if no "
"OneAgent is installed.")

Expand All @@ -59,11 +59,11 @@ def _get_metadata_file_content(self) -> List[str]:
return attributes_file.readlines()
except OSError:
self.__logger.info(
"Could not read OneAgent metadata file ({}).".format(
"Could not read Dynatrace metadata file ({}).".format(
metadata_file_name))
return []

def _parse_oneagent_metadata(self, lines) -> Mapping[str, str]:
def _parse_dynatrace_metadata(self, lines) -> Mapping[str, str]:
key_value_pairs = {}
for line in lines:
self.__logger.debug("Parsing line {}".format(line.rstrip("\n")))
Expand Down
18 changes: 10 additions & 8 deletions src/dynatrace/opentelemetry/metrics/export/serializer.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ def __init__(
self,
prefix: Optional[str] = "",
default_dimensions: Optional[Mapping] = None,
oneagent_dimensions: Optional[Mapping] = None,
dynatrace_metadata_dimensions: Optional[Mapping] = None,
):
self._prefix = prefix
self._is_delta_export = None
Expand All @@ -81,7 +81,7 @@ def __init__(
default_dimensions)

self._static_dimensions = self._normalize_dimensions(
oneagent_dimensions)
dynatrace_metadata_dimensions)
self._static_dimensions["dt.metrics.source"] = "opentelemetry"

@classmethod
Expand Down Expand Up @@ -123,6 +123,7 @@ def _write_record(
metric_key = self._get_metric_key(record)
if metric_key == "":
return

string_buffer.append(metric_key)

# merge dimensions to make them unique
Expand All @@ -133,8 +134,8 @@ def _write_record(

# add the merged dimension to the string builder.
self._write_dimensions(string_buffer, unique_dimensions)

serialize_func(string_buffer, aggregator)

self._write_timestamp(string_buffer, aggregator)
string_buffer.append("\n")

Expand Down Expand Up @@ -305,10 +306,11 @@ def _make_unique_dimensions(cls,
default_dimensions: typing.Dict[str, str],
labels: Iterable[Tuple[str, str]],
one_agent_dimensions: typing.Dict[str, str]):
"""Merge default dimensions, user specified dimensions and OneAgent
dimensions. default dimensions will be overwritten by user-specified
dimensions, which will be overwritten by OneAgent dimensions.
Default and OneAgent dimensions are assumed to be normalized when
"""Merge default dimensions, user specified dimensions and Dynatrace
metadata dimensions. Default dimensions will be overwritten by
user-specified dimensions, which will be overwritten by Dynatrace
metadata dimensions.
Default and metadata dimensions are expected to be normalized when
they are passed to this function."""
dims_map = {}

Expand All @@ -323,7 +325,7 @@ def _make_unique_dimensions(cls,
dims_map[key] = cls._normalize_dimension_value(v)

# overwrite dimensions that the user set with the default dimensions
# and OneAgent metadata. Tags are normalized in __init__ so they
# and Dynatrace metadata. Tags are normalized in __init__ so they
# don't have to be re-normalized here.
if one_agent_dimensions:
for k, v in one_agent_dimensions.items():
Expand Down
Loading