From df8a761d491bace1158329c2c17fa9403c2bfc4c Mon Sep 17 00:00:00 2001 From: Povilas Versockas Date: Thu, 28 Sep 2023 08:33:21 +0300 Subject: [PATCH] merge upstream changes to coralogix-python-dev (#6) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Add otelTraceSampled to instrumetation-logging (#1773) * Add otelTraceSampled to instrumetation-logging * Updated code with black * Added to CHANGELOG.md --------- Co-authored-by: Srikanth Chekuri * Expand sqlalchemy pool.name to follow the semantic conventions (#1778) * Skip requests tests for pypy3 (#1806) * Update version to 1.19.0.dev/0.40b0.dev (#1797) Co-authored-by: Diego Hurtado Co-authored-by: Srikanth Chekuri * Resource detector for container properties (#1584) Co-authored-by: Diego Hurtado Co-authored-by: Srikanth Chekuri Co-authored-by: Leighton Chen * botocore: always use x-ray for http header injection (#1741) Co-authored-by: Diego Hurtado * Refactor CODEOWNERS file (#1804) * Refactor CODEOWNERS file Fixes #1803 * Remove CODEOWNERS * Refactor component owners configuration * Refactor CODEOWNERS to select any file but the ones in instrumentation --------- Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * Update maintainers list (#1817) * skip urllib3 test on pypy3 (#1826) * Fix celery docker tests (#1841) * Use HTTP mock server for aiohttp tests (#1849) Fixes #1842 * Use HTTP mock server for tornado tests (#1855) * Use HTTP mock server for tornado tests Fixes #1681 * Fix lint * Remove use of httpbin (#1854) * Use `request_ctx` to determine whether or not `_teardown_request` should end flask span (#1692) Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> Co-authored-by: Diego Hurtado * remove srikanthccv from maintainers (#1792) Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> Co-authored-by: Diego Hurtado * Sanitize redis db_statement by default (#1776) Co-authored-by: Srikanth Chekuri Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * Relax httpx version to allow >= 0.18.0 (#1748) * Issue #1757 - Update HTTP server/client instrumentation span names (#1759) Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> Co-authored-by: Srikanth Chekuri * Fix falcon usage of Span Status (#1840) * Fix falcon usage of Span Status to only set the description if the status code is ERROR * Update changelog * Update CHANGELOG.md Co-authored-by: Srikanth Chekuri * fix lint * Use fewer variables to satisfy R0914 lint rule --------- Co-authored-by: Srikanth Chekuri * Request Flask attributes passed to Sampler (#1784) * Request Flask attributes passed to Sampler * Update changelog * Lint * Fix botocore test keyerror * Revert "Fix botocore test keyerror" This reverts commit fd03c55a3902b3456afd6a2ecf429afba11b0691. * botocore test does get_queue_url * Revert "botocore test does get_queue_url" This reverts commit 9530cd250dd836b3181a9361decb130e2aae1202. * Update changelog --------- Co-authored-by: Srikanth Chekuri Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * Fixed urllib3 instrumentation example in instrumentation documentation (#1793) * corrected instrumentation example in urllib3 * Remove changelog entry --------- Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> Co-authored-by: Diego Hurtado * Fix Invalid type NoneType for attribute X error | AWS-Lambda instrumentation (#1785) * Add None checking to the aws-lambda logic * Update changelog. * Change .get() check to 'key' in dict check. * Fix consistency issues. * Update changelog. --------- Co-authored-by: Srikanth Chekuri Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * Add metrics instrumentation celery (#1679) Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * Add support for confluent_kafka until 2.1.1 version (#1815) * Add support for confulent_kafka until 2.1.1 version * Include 2.1.1 version * update CHANGELOG.md * run: 'tox -e generate' * resolve comments * update top version to 2.2.0 --------- Co-authored-by: Ran Nozik * fix redis doc (#1808) doc string rendered at https://opentelemetry-python-contrib.readthedocs.io/en/latest/instrumentation/redis/redis.html refers to `opentelemetry-instrumentation` executable which appears to be a typo Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * Add http.server.response.size metric to ASGI implementation. (#1789) * Add http.server.response.size metric to ASGI implementation. Add new unit tests. * Update changelog. * Fix linting by disabling too-many-nested-blocks * Put new logic in a new method * Refactor the placement of new logic. * Fixed the unit tests in FastAPI and Starlette * Update changelog. * FIx lint errors. * Refactor getting content-length header * Refactor getting content-length header --------- Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> Co-authored-by: Diego Hurtado * fix: Update falcon instrumentation to follow semantic conventions (#1824) * fix: Update falcon instrumentation to follow semantic conventions * docs: Update changelog * fix linter errors * Disable falcon.HTTP_200 pylint checck --------- Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> Co-authored-by: Srikanth Chekuri * Bump requests from 2.28.1 to 2.31.0 (#1818) Bumps [requests](https://github.com/psf/requests) from 2.28.1 to 2.31.0. - [Release notes](https://github.com/psf/requests/releases) - [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md) - [Commits](https://github.com/psf/requests/compare/v2.28.1...v2.31.0) --- updated-dependencies: - dependency-name: requests dependency-type: direct:production ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * fix(grpc): Allow gRPC connections via Unix socket (#1833) * fix(grpc): Allow gRPC connections via Unix socket This commit addresses issue #1832. The way `NET_PEER_IP` and `NET_PEER_PORT` are retrieved raises a `ValueError` when gRPC connections are handled via Unix sockets. ```py ip, port = ( context.peer().split(",")[0].split(":", 1)[1].rsplit(":", 1) ) ``` When using an address like `unix:///tmp/grpc.sock` the value of `context.peer()` is `"unix:"`. Substituting that in the function above... ```py ip, port = "unix:".split(",")[0].split(":", 1)[1].rsplit(":", 1) ip, port = ["unix:"][0].split(":", 1)[1].rsplit(":", 1) ip, port = "unix:".split(":", 1)[1].rsplit(":", 1) ip, port = ["unix", ""][1].rsplit(":", 1) ip, port = "".rsplit(":", 1) ip, port = [""] # ValueError ``` I "addressed" the issue by guarding the retrieval of `net.peer.*` values under an `if` statement that checks if we are using a Unix socket. I extended the `server_interceptor` tests to run against TCP and Unix socket configurations. --- **Open Questions** - [ ] The socket tests will fail on Windows. Is there a way to annotate that? - [ ] Are there other span values we should be setting for the unix socket? * Update CHANGELOG * Add placeholder attributes for linter * fix lint --------- Co-authored-by: Matt Oberle Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * Add http.server.request.size for ASGI metric implementation (#1867) * Update changelog file. * Update changelog file. * Add new request.size metric for ASGI middleware. * Clean-up. * Refactor try except section. --------- Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * Fix elastic-search sanitization for bulk queries (#1870) * support sanitization for str body response * add CHANGELOG entry --------- Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * Add conditional elastic_transport import (#1810) * Add conditional elastic_transport import * Update changelog * Add future es8 tests * Update CHANGELOG.md Co-authored-by: Diego Hurtado * Add license, rm pylint disable * Consistent elastic version check * lint import * Update CHANGELOG.md --------- Co-authored-by: Diego Hurtado Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * Fix async redis clients tracing (#1830) * Fix async redis clients tracing * Update changelog * Add functional integration tests and fix linting issues --------- Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * Update maintainers list (#1874) * Use a weak reference to sqlalchemy Engine to avoid memory leak (#1771) * Use a weak reference to sqlalchemy Engine to avoid memory leak Closes #1761 By using a weak reference to the `Engine` object, we can avoid the memory leak as disposed `Engines` get properly deallocated. Whenever `SQLAlchemy` is uninstrumented, we only trigger a removal for those event listeners which are listening for objects that haven't been garbage-collected yet. * Made a mistake in resolving the weak reference * Fixed formatting issues * Updated changelog * Added unit test to check that engine was garbage collected * Do not save engine in EngineTracer to avoid memory leak * Add an empty line to satisfy black formatter * Fix isort complaints * Fixed the issue when pool name is not set and =None * Fix formatting issue * Rebased after changes in a recent commit * Updated PR number in changelog --------- Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * sqlalchemy wrap_create_engine now accepts sqlcommenter options (#1873) * sqlalchemy wrap_create_engine accepts sqlcommenter options * Changelog * Lint * Fix default val * Add sqlalchemy tests * Change a default in _instrument get * Lint * More lint * Update default Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * Update args doc * lintttt --------- Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * Add statement of maintainership (#1859) Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * urllib3: Add instrumentation support for version 2 (#1879) * urllib3: Add instrumentation support for version 2 * changelog * Distro selection (#1823) * fix(django): avoid empty span name on empty path (#1788) Co-authored-by: Srikanth Chekuri Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> Co-authored-by: Diego Hurtado * Instrument mysqlclient library (#1744) Co-authored-by: Diego Hurtado * Update version to 1.20.0.dev/0.41b0.dev (#1885) Co-authored-by: Diego Hurtado * Fix pyyaml version (#1892) * update awslambda to use _X_AMZN_TRACE_ID as a Span Link (#1657) Co-authored-by: Ron Yishai Co-authored-by: Srikanth Chekuri Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> Co-authored-by: Diego Hurtado * Fix UnboundLocalError local variable 'start' referenced before assignment (#1889) Co-authored-by: Pablo Collins * Merge pull request from GHSA-5rv5-6h4r-h22v * Fix unbound cardinality for label http_method in wsgi based middlewares * cr: rename file * cr: change label UNKNOWN to NONSTANDARD * Update instrumentation/opentelemetry-instrumentation-wsgi/src/opentelemetry/instrumentation/wsgi/__init__.py --------- Co-authored-by: Diego Hurtado * Fix errors introduced in regression (#1913) * Azure resource detectors (#1901) * Revert "update awslambda to use _X_AMZN_TRACE_ID as a Span Link" (#1911) Co-authored-by: Diego Hurtado * feat(confluent-kafka): Add instrumentation to consume method (#1786) Co-authored-by: Diego Hurtado * Add Cassandra instrumentation (#1902) * Unwrap Celery's `ExceptionInfo` (#1863) * Unwrap `ExceptionInfo` and `ExceptionWithTraceback` Instead of reporting the `ExceptionInfo` and `ExceptionWithTraceback` wrappers raised by Celery, report the exceptions that they wrap. This ensures that the exception in the OpenTelemetry span has a type and traceback that are meaningful and relevant to the developer. * Fix typo The exception is expected, not excepted. Well, I guess it is also excepted, because it's an exception, but you get what I mean. * Reformat file with `black` Reformat the `__init__.py` file in the Celery instrumentation using `black`, fixing a CI linter error. * Address review feedback Use the VERSION attribute exposed by Billiard to decide whether to import ExceptionWithTraceback. Add a test for a failing task and check that the exceptions' type and message are preserved. * Amend ExceptionWithTraceback import * doc: fix `commenter_options` type (#1926) Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> * Update version to 1.21.0.dev/0.42b0.dev (#1930) Co-authored-by: Diego Hurtado * Modify eachdist and build script for container packages (#1949) * Update pyproject.toml (#1950) --------- Signed-off-by: dependabot[bot] Co-authored-by: Nahian-Al Hasan Co-authored-by: Srikanth Chekuri Co-authored-by: Shalev Roda <65566801+shalevr@users.noreply.github.com> Co-authored-by: OpenTelemetry Bot <107717825+opentelemetrybot@users.noreply.github.com> Co-authored-by: Diego Hurtado Co-authored-by: Sanket Mehta Co-authored-by: Leighton Chen Co-authored-by: Tristan Sloughter Co-authored-by: Matthew Grossman Co-authored-by: Nimrod Shlagman Co-authored-by: Phillip Verheyden Co-authored-by: Maciej Nachtygal <82878433+macieyng@users.noreply.github.com> Co-authored-by: Filip Nikolovski Co-authored-by: Tammy Baylis <96076570+tammy-baylis-swi@users.noreply.github.com> Co-authored-by: Yashaswi Makula Co-authored-by: Iman Shafiei Co-authored-by: Akochavi <121871419+Akochavi@users.noreply.github.com> Co-authored-by: David Gonoradsky Co-authored-by: Ran Nozik Co-authored-by: Pablo Collins Co-authored-by: Iman Shafiei Co-authored-by: Michael Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Matt Oberle Co-authored-by: Matt Oberle Co-authored-by: Vivanov98 <66319645+Vivanov98@users.noreply.github.com> Co-authored-by: Rytis Bagdziunas Co-authored-by: Mario Jonke Co-authored-by: Jeremy Voss Co-authored-by: Marc Dougherty Co-authored-by: Alexandre Papin Co-authored-by: Ron Yishai Co-authored-by: Thomas LÉVEIL Co-authored-by: Jakub Warczarek Co-authored-by: Raphael Philipe Mendes da Silva Co-authored-by: Javier Fernández Rodríguez Co-authored-by: mattcontinisio Co-authored-by: Noemi <45180344+unflxw@users.noreply.github.com> Co-authored-by: João Thallis --- .github/CODEOWNERS | 28 +- .github/component_owners.yml | 18 + .github/workflows/test.yml | 4 +- CHANGELOG.md | 96 ++++- CONTRIBUTING.md | 11 + README.md | 9 +- _template/version.py | 2 +- dev-requirements.txt | 4 +- docs-requirements.txt | 4 +- docs/conf.py | 8 +- docs/index.rst | 11 +- docs/instrumentation/cassandra/cassandra.rst | 7 + .../mysqlclient/mysqlclient.rst | 7 + docs/resource/container/container.rst | 7 + eachdist.ini | 6 +- .../prometheus_remote_write/version.py | 2 +- .../pyproject.toml | 2 +- .../exporter/richconsole/version.py | 2 +- instrumentation/README.md | 6 +- .../pyproject.toml | 2 +- .../instrumentation/aio_pika/version.py | 2 +- .../pyproject.toml | 7 +- .../aiohttp_client/__init__.py | 2 +- .../instrumentation/aiohttp_client/version.py | 2 +- .../tests/test_aiohttp_client_integration.py | 46 ++- .../pyproject.toml | 8 +- .../instrumentation/aiopg/version.py | 2 +- .../pyproject.toml | 8 +- .../instrumentation/asgi/__init__.py | 53 ++- .../instrumentation/asgi/version.py | 2 +- .../tests/test_asgi_middleware.py | 89 +++- .../pyproject.toml | 6 +- .../instrumentation/asyncpg/version.py | 2 +- .../pyproject.toml | 6 +- .../instrumentation/aws_lambda/__init__.py | 93 +++-- .../instrumentation/aws_lambda/version.py | 2 +- .../pyproject.toml | 6 +- .../instrumentation/boto/version.py | 2 +- .../pyproject.toml | 6 +- .../instrumentation/boto3sqs/version.py | 2 +- .../pyproject.toml | 7 +- .../instrumentation/botocore/__init__.py | 3 +- .../instrumentation/botocore/version.py | 2 +- .../tests/test_botocore_instrumentation.py | 66 +-- .../LICENSE | 201 +++++++++ .../README.rst | 25 ++ .../pyproject.toml | 60 +++ .../instrumentation/cassandra/__init__.py | 91 +++++ .../instrumentation/cassandra/package.py} | 5 +- .../instrumentation/cassandra/version.py | 15 + .../tests/__init__.py | 0 .../tests/test_cassandra_integration.py | 121 ++++++ .../pyproject.toml | 6 +- .../instrumentation/celery/__init__.py | 66 ++- .../instrumentation/celery/utils.py | 1 + .../instrumentation/celery/version.py | 2 +- .../tests/celery_test_tasks.py | 9 + .../tests/test_metrics.py | 76 ++++ .../tests/test_tasks.py | 72 +++- .../pyproject.toml | 2 +- .../confluent_kafka/__init__.py | 73 +++- .../confluent_kafka/package.py | 2 +- .../instrumentation/confluent_kafka/utils.py | 34 +- .../confluent_kafka/version.py | 2 +- .../tests/test_instrumentation.py | 146 ++++++- .../tests/utils.py | 39 ++ .../pyproject.toml | 6 +- .../instrumentation/dbapi/version.py | 2 +- .../pyproject.toml | 12 +- .../django/middleware/otel_middleware.py | 15 +- .../instrumentation/django/version.py | 2 +- .../tests/test_middleware.py | 43 +- .../tests/test_middleware_asgi.py | 14 +- .../pyproject.toml | 6 +- .../instrumentation/elasticsearch/__init__.py | 38 +- .../instrumentation/elasticsearch/utils.py | 4 + .../instrumentation/elasticsearch/version.py | 2 +- .../tests/helpers_es8.py | 41 ++ .../tests/test_elasticsearch.py | 8 +- .../pyproject.toml | 10 +- .../instrumentation/falcon/__init__.py | 21 +- .../instrumentation/falcon/version.py | 2 +- .../tests/app.py | 9 + .../tests/test_falcon.py | 33 +- .../pyproject.toml | 10 +- .../instrumentation/fastapi/__init__.py | 37 +- .../instrumentation/fastapi/version.py | 2 +- .../tests/test_fastapi_instrumentation.py | 32 +- .../pyproject.toml | 13 +- .../instrumentation/flask/__init__.py | 53 ++- .../instrumentation/flask/version.py | 2 +- .../tests/base_test.py | 18 +- .../tests/test_copy_context.py | 48 +++ .../tests/test_programmatic.py | 96 ++++- .../pyproject.toml | 6 +- .../instrumentation/grpc/_server.py | 38 +- .../instrumentation/grpc/version.py | 2 +- .../tests/test_server_interceptor.py | 188 ++++----- .../README.rst | 6 +- .../pyproject.toml | 8 +- .../instrumentation/httpx/__init__.py | 8 +- .../instrumentation/httpx/version.py | 2 +- .../tests/test_httpx_integration.py | 16 +- .../pyproject.toml | 4 +- .../instrumentation/jinja2/version.py | 2 +- .../pyproject.toml | 6 +- .../instrumentation/kafka/version.py | 2 +- .../pyproject.toml | 4 +- .../instrumentation/logging/__init__.py | 2 + .../instrumentation/logging/constants.py | 5 +- .../instrumentation/logging/version.py | 2 +- .../tests/test_logging.py | 20 +- .../pyproject.toml | 6 +- .../instrumentation/mysql/version.py | 2 +- .../LICENSE | 201 +++++++++ .../README.rst | 21 + .../pyproject.toml | 58 +++ .../instrumentation/mysqlclient/__init__.py | 117 ++++++ .../instrumentation/mysqlclient/package.py | 16 + .../instrumentation/mysqlclient/version.py | 15 + .../tests/__init__.py | 0 .../tests/test_mysqlclient_integration.py | 118 ++++++ .../pyproject.toml | 2 +- .../instrumentation/pika/version.py | 2 +- .../pyproject.toml | 6 +- .../instrumentation/psycopg2/version.py | 2 +- .../pyproject.toml | 6 +- .../instrumentation/pymemcache/version.py | 2 +- .../pyproject.toml | 6 +- .../instrumentation/pymongo/version.py | 2 +- .../pyproject.toml | 6 +- .../instrumentation/pymysql/version.py | 2 +- .../pyproject.toml | 10 +- .../instrumentation/pyramid/version.py | 2 +- .../tests/pyramid_base_test.py | 6 +- .../tests/test_programmatic.py | 2 +- .../pyproject.toml | 6 +- .../instrumentation/redis/__init__.py | 157 ++++--- .../instrumentation/redis/util.py | 46 +-- .../instrumentation/redis/version.py | 2 +- .../tests/test_redis.py | 77 ++-- .../pyproject.toml | 6 +- .../instrumentation/remoulade/version.py | 2 +- .../pyproject.toml | 8 +- .../instrumentation/requests/__init__.py | 12 +- .../instrumentation/requests/version.py | 2 +- .../tests/test_requests_integration.py | 12 +- .../tests/test_requests_ip_support.py | 2 +- .../pyproject.toml | 4 +- .../instrumentation/sklearn/version.py | 2 +- .../pyproject.toml | 4 +- .../instrumentation/sqlalchemy/__init__.py | 17 +- .../instrumentation/sqlalchemy/engine.py | 64 ++- .../instrumentation/sqlalchemy/version.py | 2 +- .../tests/test_sqlalchemy.py | 120 ++++++ .../tests/test_sqlalchemy_metrics.py | 3 +- .../tests/test_sqlcommenter.py | 18 + .../pyproject.toml | 6 +- .../instrumentation/sqlite3/version.py | 2 +- .../pyproject.toml | 10 +- .../instrumentation/starlette/__init__.py | 37 +- .../instrumentation/starlette/version.py | 2 +- .../tests/test_starlette_instrumentation.py | 24 +- .../pyproject.toml | 2 +- .../instrumentation/system_metrics/version.py | 2 +- .../pyproject.toml | 9 +- .../instrumentation/tornado/__init__.py | 23 +- .../instrumentation/tornado/version.py | 2 +- .../tests/test_instrumentation.py | 79 ++-- .../pyproject.toml | 6 +- .../instrumentation/tortoiseorm/version.py | 2 +- .../pyproject.toml | 8 +- .../instrumentation/urllib/__init__.py | 2 +- .../instrumentation/urllib/version.py | 2 +- .../tests/test_metrics_instrumentation.py | 4 +- .../tests/test_urllib_integration.py | 22 +- .../README.rst | 4 +- .../pyproject.toml | 10 +- .../instrumentation/urllib3/__init__.py | 6 +- .../instrumentation/urllib3/package.py | 2 +- .../instrumentation/urllib3/version.py | 2 +- .../tests/test_urllib3_integration.py | 34 +- .../tests/test_urllib3_ip_support.py | 2 +- .../tests/test_urllib3_metrics.py | 32 +- .../pyproject.toml | 8 +- .../instrumentation/wsgi/__init__.py | 26 +- .../instrumentation/wsgi/version.py | 2 +- .../tests/test_wsgi_middleware.py | 36 +- .../pyproject.toml | 86 ++-- .../contrib-instrumentations/version.py | 2 +- opentelemetry-distro/pyproject.toml | 4 +- .../src/opentelemetry/distro/version.py | 2 +- opentelemetry-instrumentation/README.rst | 7 +- .../auto_instrumentation/_load.py | 124 ++++++ .../auto_instrumentation/sitecustomize.py | 93 +---- .../instrumentation/bootstrap_gen.py | 104 ++--- .../instrumentation/environment_variables.py | 10 + .../opentelemetry/instrumentation/utils.py | 2 +- .../opentelemetry/instrumentation/version.py | 2 +- .../tests/auto_instrumentation/test_load.py | 312 ++++++++++++++ .../{ => auto_instrumentation}/test_run.py | 0 .../propagators/ot_trace/version.py | 2 +- .../LICENSE | 201 +++++++++ .../MANIFEST.rst | 9 + .../README.rst | 82 ++++ .../pyproject.toml | 51 +++ .../resource/detector/azure/app_service.py | 75 ++++ .../resource/detector/azure/version.py | 15 + .../resource/detector/azure/vm.py | 97 +++++ .../tests/__init__.py | 13 + .../tests/test_app_service.py | 116 ++++++ .../tests/test_vm.py | 382 ++++++++++++++++++ .../LICENSE | 201 +++++++++ .../MANIFEST.rst | 9 + .../README.rst | 46 +++ .../pyproject.toml | 50 +++ .../resource/detector/container/__init__.py | 95 +++++ .../resource/detector/container/version.py | 15 + .../tests/__init__.py | 0 .../tests/test_container.py | 146 +++++++ scripts/build.sh | 2 +- .../tests/celery/test_celery_functional.py | 12 +- .../tests/redis/test_redis_functional.py | 174 ++++++-- .../tests/sqlalchemy_tests/test_postgres.py | 37 ++ tox.ini | 61 ++- .../src/opentelemetry/util/http/__init__.py | 15 +- .../src/opentelemetry/util/http/version.py | 2 +- .../tests/test_sanitize_method.py | 44 ++ 228 files changed, 5777 insertions(+), 1115 deletions(-) create mode 100644 docs/instrumentation/cassandra/cassandra.rst create mode 100644 docs/instrumentation/mysqlclient/mysqlclient.rst create mode 100644 docs/resource/container/container.rst create mode 100644 instrumentation/opentelemetry-instrumentation-cassandra/LICENSE create mode 100644 instrumentation/opentelemetry-instrumentation-cassandra/README.rst create mode 100644 instrumentation/opentelemetry-instrumentation-cassandra/pyproject.toml create mode 100644 instrumentation/opentelemetry-instrumentation-cassandra/src/opentelemetry/instrumentation/cassandra/__init__.py rename instrumentation/{opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/environment_variables.py => opentelemetry-instrumentation-cassandra/src/opentelemetry/instrumentation/cassandra/package.py} (85%) create mode 100644 instrumentation/opentelemetry-instrumentation-cassandra/src/opentelemetry/instrumentation/cassandra/version.py create mode 100644 instrumentation/opentelemetry-instrumentation-cassandra/tests/__init__.py create mode 100644 instrumentation/opentelemetry-instrumentation-cassandra/tests/test_cassandra_integration.py create mode 100644 instrumentation/opentelemetry-instrumentation-celery/tests/test_metrics.py create mode 100644 instrumentation/opentelemetry-instrumentation-confluent-kafka/tests/utils.py create mode 100644 instrumentation/opentelemetry-instrumentation-elasticsearch/tests/helpers_es8.py create mode 100644 instrumentation/opentelemetry-instrumentation-flask/tests/test_copy_context.py create mode 100644 instrumentation/opentelemetry-instrumentation-mysqlclient/LICENSE create mode 100644 instrumentation/opentelemetry-instrumentation-mysqlclient/README.rst create mode 100644 instrumentation/opentelemetry-instrumentation-mysqlclient/pyproject.toml create mode 100644 instrumentation/opentelemetry-instrumentation-mysqlclient/src/opentelemetry/instrumentation/mysqlclient/__init__.py create mode 100644 instrumentation/opentelemetry-instrumentation-mysqlclient/src/opentelemetry/instrumentation/mysqlclient/package.py create mode 100644 instrumentation/opentelemetry-instrumentation-mysqlclient/src/opentelemetry/instrumentation/mysqlclient/version.py create mode 100644 instrumentation/opentelemetry-instrumentation-mysqlclient/tests/__init__.py create mode 100644 instrumentation/opentelemetry-instrumentation-mysqlclient/tests/test_mysqlclient_integration.py create mode 100644 opentelemetry-instrumentation/src/opentelemetry/instrumentation/auto_instrumentation/_load.py create mode 100644 opentelemetry-instrumentation/tests/auto_instrumentation/test_load.py rename opentelemetry-instrumentation/tests/{ => auto_instrumentation}/test_run.py (100%) create mode 100644 resource/opentelemetry-resource-detector-azure/LICENSE create mode 100644 resource/opentelemetry-resource-detector-azure/MANIFEST.rst create mode 100644 resource/opentelemetry-resource-detector-azure/README.rst create mode 100644 resource/opentelemetry-resource-detector-azure/pyproject.toml create mode 100644 resource/opentelemetry-resource-detector-azure/src/opentelemetry/resource/detector/azure/app_service.py create mode 100644 resource/opentelemetry-resource-detector-azure/src/opentelemetry/resource/detector/azure/version.py create mode 100644 resource/opentelemetry-resource-detector-azure/src/opentelemetry/resource/detector/azure/vm.py create mode 100644 resource/opentelemetry-resource-detector-azure/tests/__init__.py create mode 100644 resource/opentelemetry-resource-detector-azure/tests/test_app_service.py create mode 100644 resource/opentelemetry-resource-detector-azure/tests/test_vm.py create mode 100644 resource/opentelemetry-resource-detector-container/LICENSE create mode 100644 resource/opentelemetry-resource-detector-container/MANIFEST.rst create mode 100644 resource/opentelemetry-resource-detector-container/README.rst create mode 100644 resource/opentelemetry-resource-detector-container/pyproject.toml create mode 100644 resource/opentelemetry-resource-detector-container/src/opentelemetry/resource/detector/container/__init__.py create mode 100644 resource/opentelemetry-resource-detector-container/src/opentelemetry/resource/detector/container/version.py create mode 100644 resource/opentelemetry-resource-detector-container/tests/__init__.py create mode 100644 resource/opentelemetry-resource-detector-container/tests/test_container.py create mode 100644 util/opentelemetry-util-http/tests/test_sanitize_method.py diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 868fbf8f9c..0cb85929af 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -1,26 +1,12 @@ -# Code owners file. -# This file controls who is tagged for review for any given pull request. -# -# What is a "CODEOWNER"? -# -# A CODEOWNER lends their expertise to a specific package hosted by an OpenTelemetry repository. -# -# A CODEOWNER MUST: -# - introduce themselves on the CNCF OTel Python channel: https://cloud-native.slack.com/archives/C01PD4HUVBL -# - have enough knowledge of the corresponding instrumented library -# - respond to issues -# - fix failing unit tests or any other blockers to the CI/CD workflow -# - update usage of `opentelemetry-python-core` APIs upon the introduction of breaking changes -# - be a member of the OpenTelemetry community so that the `component-owners.yml` action to automatically assign CODEOWNERS to PRs works correctly. -# +# This file is only used as a way to assign any change to the approvers team +# except for a change in any of the instrumentations. The actual codeowners +# of the instrumentations are in .github/component_owners.yml. - -# For anything not explicitly taken by someone else: +# Assigns any change to the approvers team... * @open-telemetry/opentelemetry-python-contrib-approvers +# ...except for changes in any instrumentation. +/instrumentation + # Learn about CODEOWNERS file format: # https://help.github.com/en/articles/about-code-owners -# -# Learn about membership in OpenTelemetry community: -# https://github.com/open-telemetry/community/blob/main/community-membership.md -# diff --git a/.github/component_owners.yml b/.github/component_owners.yml index 933dc7da13..e31fde9b48 100644 --- a/.github/component_owners.yml +++ b/.github/component_owners.yml @@ -3,6 +3,7 @@ components: docs/instrumentation: - nemoshlag + instrumentation/opentelemetry-instrumentation-aio-pika: - ofek1weiss @@ -43,9 +44,26 @@ components: instrumentation/opentelemetry-instrumentation-urllib: - shalevr + - ocelotl instrumentation/opentelemetry-instrumentation-urllib3: - shalevr + - ocelotl instrumentation/opentelemetry-instrumentation-sqlalchemy: - shalevr + + instrumentation/opentelemetry-instrumentation-flask: + - ocelotl + + instrumentation/opentelemetry-instrumentation-jinja2: + - ocelotl + + instrumentation/opentelemetry-instrumentation-logging: + - ocelotl + + instrumentation/opentelemetry-instrumentation-requests: + - ocelotl + + instrumentation/opentelemetry-instrumentation-cassandra: + - mattcontinisio diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index f8ecec6532..8decdb1a42 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -6,7 +6,7 @@ on: - 'release/*' pull_request: env: - CORE_REPO_SHA: 2387b4465d930b020df79692a8097e1d54b66ec1 + CORE_REPO_SHA: 0ef76a5cc39626f783416ca75e43556e2bb2739d jobs: build: @@ -24,7 +24,7 @@ jobs: fail-fast: false # ensures the entire test matrix is run, even if one permutation fails matrix: python-version: [ py37, py38, py39, py310, py311, pypy3 ] - package: ["instrumentation", "distro", "exporter", "sdkextension", "propagator"] + package: ["instrumentation", "distro", "exporter", "sdkextension", "propagator", "resource"] os: [ ubuntu-20.04 ] steps: - name: Checkout Contrib Repo @ SHA - ${{ github.sha }} diff --git a/CHANGELOG.md b/CHANGELOG.md index 73fc2b9aa1..1206844e55 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,32 +7,118 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## Unreleased -- `opentelemetry-instrumentation-system-metrics` Add `process.` prefix to `runtime.memory`, `runtime.cpu.time`, and `runtime.gc_count`. Change `runtime.memory` from count to UpDownCounter. ([#1735](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1735)) +## Version 1.20.0/0.41b0 (2023-09-01) + +### Fixed + +- `opentelemetry-instrumentation-asgi` Fix UnboundLocalError local variable 'start' referenced before assignment + ([#1889](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1889)) +- Fixed union typing error not compatible with Python 3.7 introduced in `opentelemetry-util-http`, fix tests introduced by patch related to sanitize method for wsgi + ([#1913](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1913)) +- `opentelemetry-instrumentation-celery` Unwrap Celery's `ExceptionInfo` errors and report the actual exception that was raised. ([#1863](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1863)) + +### Added + +- `opentelemetry-resource-detector-azure` Add resource detectors for Azure App Service and VM + ([#1901](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1901)) + +## Version 1.19.0/0.40b0 (2023-07-13) +- `opentelemetry-instrumentation-asgi` Add `http.server.request.size` metric + ([#1867](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1867)) + +### Fixed + +- `opentelemetry-instrumentation-django` Fix empty span name when using + `path("", ...)` ([#1788](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1788) +- Fix elastic-search instrumentation sanitization to support bulk queries + ([#1870](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1870)) +- Update falcon instrumentation to follow semantic conventions + ([#1824](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1824)) +- Fix sqlalchemy instrumentation wrap methods to accept sqlcommenter options + ([#1873](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1873)) + +### Added + +- Add instrumentor support for cassandra and scylla + ([#1902](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1902)) +- Add instrumentor support for mysqlclient + ([#1744](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1744)) +- Fix async redis clients not being traced correctly + ([#1830](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1830)) +- Make Flask request span attributes available for `start_span`. + ([#1784](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1784)) +- Fix falcon instrumentation's usage of Span Status to only set the description if the status code is ERROR. + ([#1840](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1840)) +- Instrument all httpx versions >= 0.18. + ([#1748](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1748)) +- Fix `Invalid type NoneType for attribute X (opentelemetry-instrumentation-aws-lambda)` error when some attributes do not exist + ([#1780](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1780)) +- Add metric instrumentation for celery + ([#1679](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1679)) +- `opentelemetry-instrumentation-asgi` Add `http.server.response.size` metric + ([#1789](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1789)) +- `opentelemetry-instrumentation-grpc` Allow gRPC connections via Unix socket + ([#1833](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1833)) +- Fix elasticsearch `Transport.perform_request` instrument wrap for elasticsearch >= 8 + ([#1810](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1810)) +- `opentelemetry-instrumentation-urllib3` Add support for urllib3 version 2 + ([#1879](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1879)) +- Add optional distro and configurator selection for auto-instrumentation + ([#1823](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1823)) + +### Added +- `opentelemetry-instrumentation-kafka-python` Add instrumentation to `consume` method + ([#1786](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1786)) + +## Version 1.18.0/0.39b0 (2023-05-10) + +- Update runtime metrics to follow semantic conventions + ([#1735](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1735)) - Add request and response hooks for GRPC instrumentation (client only) ([#1706](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1706)) +- Fix memory leak in SQLAlchemy instrumentation where disposed `Engine` does not get garbage collected + ([#1771](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1771)) - `opentelemetry-instrumentation-pymemcache` Update instrumentation to support pymemcache >4 ([#1764](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1764)) +- `opentelemetry-instrumentation-confluent-kafka` Add support for higher versions of confluent_kafka + ([#1815](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1815)) ### Added +- Expand sqlalchemy pool.name to follow the semantic conventions + ([#1778](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1778)) - Add `excluded_urls` functionality to `urllib` and `urllib3` instrumentations ([#1733](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1733)) -- Make Django request span attributes available for `start_span`. +- Make Django request span attributes available for `start_span`. ([#1730](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1730)) -- Make ASGI request span attributes available for `start_span`. +- Make ASGI request span attributes available for `start_span`. ([#1762](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1762)) - `opentelemetry-instrumentation-celery` Add support for anonymous tasks. - ([#1407](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1407) + ([#1407](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1407)) +- `opentelemetry-instrumentation-logging` Add `otelTraceSampled` to instrumetation-logging + ([#1773](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1773)) + +### Changed +- `opentelemetry-instrumentation-botocore` now uses the AWS X-Ray propagator by default + ([#1741](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1741)) ### Fixed +- Fix redis db.statements to be sanitized by default + ([#1778](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1778)) - Fix elasticsearch db.statement attribute to be sanitized by default ([#1758](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1758)) - Fix `AttributeError` when AWS Lambda handler receives a list event ([#1738](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1738)) - Fix `None does not implement middleware` error when there are no middlewares registered ([#1766](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1766)) +- Fix Flask instrumentation to only close the span if it was created by the same request context. + ([#1692](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1692)) + +### Changed +- Update HTTP server/client instrumentation span names to comply with spec + ([#1759](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1759)) ## Version 1.17.0/0.38b0 (2023-03-22) @@ -121,6 +207,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- `opentelemetry-resource-detector-container` Add support resource detection of container properties. + ([#1584](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1584)) - `opentelemetry-instrumentation-pymysql` Add tests for commit() and rollback(). ([#1424](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1424)) - `opentelemetry-instrumentation-fastapi` Add support for regular expression matching and sanitization of HTTP headers. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a159bd1f03..41f2aa08cb 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -124,6 +124,17 @@ Open a pull request against the main `opentelemetry-python-contrib` repo. as `work-in-progress`, or mark it as [`draft`](https://github.blog/2019-02-14-introducing-draft-pull-requests/). * Make sure CLA is signed and CI is clear. +### How to Get PRs Reviewed + +The maintainers and approvers of this repo are not experts in every instrumentation there is here. +In fact each one of us knows enough about them to only review a few. Unfortunately it can be hard +to find enough experts in every instrumentation to quickly review every instrumentation PR. The +instrumentation experts are listed in `.github/component_owners.yml` with their corresponding files +or directories that they own. The owners listed there will be notified when PRs that modify their +files are opened. + +If you are not getting reviews, please contact the respective owners directly. + ### How to Get PRs Merged A PR is considered to be **ready to merge** when: diff --git a/README.md b/README.md index 4b7cc036e5..ce1f8f3df4 100644 --- a/README.md +++ b/README.md @@ -49,6 +49,8 @@ depend on `opentelemetry-sdk` or another package that implements the API. **Please note** that these libraries are currently in _beta_, and shouldn't generally be used in production environments. +Unless explicitly stated otherwise, any instrumentation here for a particular library is not developed or maintained by the authors of such library. + The [`instrumentation/`](https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation) directory includes OpenTelemetry instrumentation packages, which can be installed @@ -95,12 +97,12 @@ Meeting notes are available as a public [Google doc](https://docs.google.com/doc Approvers ([@open-telemetry/python-approvers](https://github.com/orgs/open-telemetry/teams/python-approvers)): - [Aaron Abbott](https://github.com/aabmass), Google +- [Jeremy Voss](https://github.com/jeremydvoss), Microsoft - [Sanket Mehta](https://github.com/sanketmehta28), Cisco -- [Shalev Roda](https://github.com/shalevr), Cisco Emeritus Approvers: -- [Hector Hernandez](https://github.com/hectorhdzg), Microsoft +- [Héctor Hernández](https://github.com/hectorhdzg), Microsoft - [Yusuke Tsutsumi](https://github.com/toumorokoshi), Google - [Nathaniel Ruiz Nowell](https://github.com/NathanielRN), AWS - [Ashutosh Goel](https://github.com/ashu658), Cisco @@ -111,12 +113,13 @@ Maintainers ([@open-telemetry/python-maintainers](https://github.com/orgs/open-t - [Diego Hurtado](https://github.com/ocelotl), Lightstep - [Leighton Chen](https://github.com/lzchen), Microsoft -- [Srikanth Chekuri](https://github.com/srikanthccv), signoz.io +- [Shalev Roda](https://github.com/shalevr), Cisco Emeritus Maintainers: - [Alex Boten](https://github.com/codeboten), Lightstep - [Owais Lone](https://github.com/owais), Splunk +- [Srikanth Chekuri](https://github.com/srikanthccv), signoz.io *Find more about the maintainer role in [community repository](https://github.com/open-telemetry/community/blob/main/community-membership.md#maintainer).* diff --git a/_template/version.py b/_template/version.py index eb62a67e28..c2996671d6 100644 --- a/_template/version.py +++ b/_template/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/dev-requirements.txt b/dev-requirements.txt index a8efb950dd..feab6b4b02 100644 --- a/dev-requirements.txt +++ b/dev-requirements.txt @@ -14,7 +14,7 @@ bleach==4.1.0 # transient dependency for readme-renderer grpcio-tools==1.29.0 mypy-protobuf>=1.23 protobuf~=3.13 -markupsafe==2.0.1 +markupsafe>=2.0.1 codespell==2.1.0 -requests==2.28.1 +requests==2.31.0 ruamel.yaml==0.17.21 diff --git a/docs-requirements.txt b/docs-requirements.txt index 6d11d20198..32f4a406aa 100644 --- a/docs-requirements.txt +++ b/docs-requirements.txt @@ -25,14 +25,16 @@ asyncpg>=0.12.0 boto~=2.0 botocore~=1.0 boto3~=1.0 +cassandra-driver~=3.25 celery>=4.0 -confluent-kafka>= 1.8.2,< 2.0.0 +confluent-kafka>= 1.8.2,<= 2.2.0 elasticsearch>=2.0,<9.0 flask~=2.0 falcon~=2.0 grpcio~=1.27 kafka-python>=2.0,<3.0 mysql-connector-python~=8.0 +mysqlclient~=2.1.1 psutil>=5 pika>=0.12.0 pymongo~=3.1 diff --git a/docs/conf.py b/docs/conf.py index 23918eb331..4b2bda04a8 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -54,7 +54,13 @@ if isdir(join(sdk_ext, f)) ] -sys.path[:0] = exp_dirs + instr_dirs + sdk_ext_dirs + prop_dirs +resource = "../resource" +resource_dirs = [ + os.path.abspath("/".join(["../resource", f, "src"])) + for f in listdir(resource) + if isdir(join(resource, f)) +] +sys.path[:0] = exp_dirs + instr_dirs + sdk_ext_dirs + prop_dirs + resource_dirs # -- Project information ----------------------------------------------------- diff --git a/docs/index.rst b/docs/index.rst index 44fbfc1188..5203c377e4 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -33,7 +33,7 @@ Extensions Visit `OpenTelemetry Registry `_ to find a lot of related projects like exporters, instrumentation libraries, tracer -implementations, etc. +implementations, resource, etc. Installing Cutting Edge Packages ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -51,6 +51,7 @@ install pip install -e ./instrumentation/opentelemetry-instrumentation-flask pip install -e ./instrumentation/opentelemetry-instrumentation-botocore pip install -e ./sdk-extension/opentelemetry-sdk-extension-aws + pip install -e ./resource/opentelemetry-resource-detector-container .. toctree:: @@ -85,6 +86,14 @@ install sdk-extension/** +.. toctree:: + :maxdepth: 2 + :caption: OpenTelemetry Resource Detectors + :name: Resource Detectors + :glob: + + resource/** + Indices and tables ------------------ diff --git a/docs/instrumentation/cassandra/cassandra.rst b/docs/instrumentation/cassandra/cassandra.rst new file mode 100644 index 0000000000..2f2ab8b9f8 --- /dev/null +++ b/docs/instrumentation/cassandra/cassandra.rst @@ -0,0 +1,7 @@ +OpenTelemetry Cassandra Instrumentation +======================================= + +.. automodule:: opentelemetry.instrumentation.cassandra + :members: + :undoc-members: + :show-inheritance: diff --git a/docs/instrumentation/mysqlclient/mysqlclient.rst b/docs/instrumentation/mysqlclient/mysqlclient.rst new file mode 100644 index 0000000000..d9c9811c31 --- /dev/null +++ b/docs/instrumentation/mysqlclient/mysqlclient.rst @@ -0,0 +1,7 @@ +OpenTelemetry mysqlclient Instrumentation +========================================= + +.. automodule:: opentelemetry.instrumentation.mysqlclient + :members: + :undoc-members: + :show-inheritance: diff --git a/docs/resource/container/container.rst b/docs/resource/container/container.rst new file mode 100644 index 0000000000..e69a2f6a0f --- /dev/null +++ b/docs/resource/container/container.rst @@ -0,0 +1,7 @@ +OpenTelemetry Python - Resource Detector for Containers +======================================================= + +.. automodule:: opentelemetry.resource.detector.container + :members: + :undoc-members: + :show-inheritance: diff --git a/eachdist.ini b/eachdist.ini index eb34aed0f0..dacd4fc42f 100644 --- a/eachdist.ini +++ b/eachdist.ini @@ -16,7 +16,7 @@ sortfirst= ext/* [stable] -version=1.18.0.dev +version=1.21.0.dev packages= opentelemetry-sdk @@ -34,7 +34,7 @@ packages= opentelemetry-api [prerelease] -version=0.39b0.dev +version=0.42b0.dev packages= all @@ -43,9 +43,11 @@ packages= opentelemetry-instrumentation opentelemetry-contrib-instrumentations opentelemetry-distro + opentelemetry-resource-detector-container [exclude_release] packages= + opentelemetry-resource-detector-azure opentelemetry-sdk-extension-aws opentelemetry-propagator-aws-xray diff --git a/exporter/opentelemetry-exporter-prometheus-remote-write/src/opentelemetry/exporter/prometheus_remote_write/version.py b/exporter/opentelemetry-exporter-prometheus-remote-write/src/opentelemetry/exporter/prometheus_remote_write/version.py index eb62a67e28..c2996671d6 100644 --- a/exporter/opentelemetry-exporter-prometheus-remote-write/src/opentelemetry/exporter/prometheus_remote_write/version.py +++ b/exporter/opentelemetry-exporter-prometheus-remote-write/src/opentelemetry/exporter/prometheus_remote_write/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/exporter/opentelemetry-exporter-richconsole/pyproject.toml b/exporter/opentelemetry-exporter-richconsole/pyproject.toml index d6bbc580d8..202dda7a51 100644 --- a/exporter/opentelemetry-exporter-richconsole/pyproject.toml +++ b/exporter/opentelemetry-exporter-richconsole/pyproject.toml @@ -27,7 +27,7 @@ classifiers = [ dependencies = [ "opentelemetry-api ~= 1.12", "opentelemetry-sdk ~= 1.12", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", "rich>=10.0.0", ] diff --git a/exporter/opentelemetry-exporter-richconsole/src/opentelemetry/exporter/richconsole/version.py b/exporter/opentelemetry-exporter-richconsole/src/opentelemetry/exporter/richconsole/version.py index eb62a67e28..c2996671d6 100644 --- a/exporter/opentelemetry-exporter-richconsole/src/opentelemetry/exporter/richconsole/version.py +++ b/exporter/opentelemetry-exporter-richconsole/src/opentelemetry/exporter/richconsole/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/README.md b/instrumentation/README.md index e69dd6adbd..12b8ada105 100644 --- a/instrumentation/README.md +++ b/instrumentation/README.md @@ -10,8 +10,9 @@ | [opentelemetry-instrumentation-boto](./opentelemetry-instrumentation-boto) | boto~=2.0 | No | [opentelemetry-instrumentation-boto3sqs](./opentelemetry-instrumentation-boto3sqs) | boto3 ~= 1.0 | No | [opentelemetry-instrumentation-botocore](./opentelemetry-instrumentation-botocore) | botocore ~= 1.0 | No +| [opentelemetry-instrumentation-cassandra](./opentelemetry-instrumentation-cassandra) | cassandra-driver ~= 3.25,scylla-driver ~= 3.25 | No | [opentelemetry-instrumentation-celery](./opentelemetry-instrumentation-celery) | celery >= 4.0, < 6.0 | No -| [opentelemetry-instrumentation-confluent-kafka](./opentelemetry-instrumentation-confluent-kafka) | confluent-kafka >= 1.8.2, < 2.0.0 | No +| [opentelemetry-instrumentation-confluent-kafka](./opentelemetry-instrumentation-confluent-kafka) | confluent-kafka >= 1.8.2, <= 2.2.0 | No | [opentelemetry-instrumentation-dbapi](./opentelemetry-instrumentation-dbapi) | dbapi | No | [opentelemetry-instrumentation-django](./opentelemetry-instrumentation-django) | django >= 1.10 | Yes | [opentelemetry-instrumentation-elasticsearch](./opentelemetry-instrumentation-elasticsearch) | elasticsearch >= 2.0 | No @@ -24,6 +25,7 @@ | [opentelemetry-instrumentation-kafka-python](./opentelemetry-instrumentation-kafka-python) | kafka-python >= 2.0 | No | [opentelemetry-instrumentation-logging](./opentelemetry-instrumentation-logging) | logging | No | [opentelemetry-instrumentation-mysql](./opentelemetry-instrumentation-mysql) | mysql-connector-python ~= 8.0 | No +| [opentelemetry-instrumentation-mysqlclient](./opentelemetry-instrumentation-mysqlclient) | mysqlclient < 3 | No | [opentelemetry-instrumentation-pika](./opentelemetry-instrumentation-pika) | pika >= 0.12.0 | No | [opentelemetry-instrumentation-psycopg2](./opentelemetry-instrumentation-psycopg2) | psycopg2 >= 2.7.3.1 | No | [opentelemetry-instrumentation-pymemcache](./opentelemetry-instrumentation-pymemcache) | pymemcache >= 1.3.5, < 5 | No @@ -41,5 +43,5 @@ | [opentelemetry-instrumentation-tornado](./opentelemetry-instrumentation-tornado) | tornado >= 5.1.1 | Yes | [opentelemetry-instrumentation-tortoiseorm](./opentelemetry-instrumentation-tortoiseorm) | tortoise-orm >= 0.17.0 | No | [opentelemetry-instrumentation-urllib](./opentelemetry-instrumentation-urllib) | urllib | Yes -| [opentelemetry-instrumentation-urllib3](./opentelemetry-instrumentation-urllib3) | urllib3 >= 1.0.0, < 2.0.0 | Yes +| [opentelemetry-instrumentation-urllib3](./opentelemetry-instrumentation-urllib3) | urllib3 >= 1.0.0, < 3.0.0 | Yes | [opentelemetry-instrumentation-wsgi](./opentelemetry-instrumentation-wsgi) | wsgi | Yes \ No newline at end of file diff --git a/instrumentation/opentelemetry-instrumentation-aio-pika/pyproject.toml b/instrumentation/opentelemetry-instrumentation-aio-pika/pyproject.toml index 53e1f1606e..af57ebe6e7 100644 --- a/instrumentation/opentelemetry-instrumentation-aio-pika/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-aio-pika/pyproject.toml @@ -35,7 +35,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-aio-pika[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", "pytest", "wrapt >= 1.0.0, < 2.0.0", ] diff --git a/instrumentation/opentelemetry-instrumentation-aio-pika/src/opentelemetry/instrumentation/aio_pika/version.py b/instrumentation/opentelemetry-instrumentation-aio-pika/src/opentelemetry/instrumentation/aio_pika/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-aio-pika/src/opentelemetry/instrumentation/aio_pika/version.py +++ b/instrumentation/opentelemetry-instrumentation-aio-pika/src/opentelemetry/instrumentation/aio_pika/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-aiohttp-client/pyproject.toml b/instrumentation/opentelemetry-instrumentation-aiohttp-client/pyproject.toml index 0a8b8a937d..c466977377 100644 --- a/instrumentation/opentelemetry-instrumentation-aiohttp-client/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-aiohttp-client/pyproject.toml @@ -26,9 +26,9 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", - "opentelemetry-util-http == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", + "opentelemetry-util-http == 0.42b0.dev", "wrapt >= 1.0.0, < 2.0.0", ] @@ -38,6 +38,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-aiohttp-client[instruments]", + "http-server-mock" ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-aiohttp-client/src/opentelemetry/instrumentation/aiohttp_client/__init__.py b/instrumentation/opentelemetry-instrumentation-aiohttp-client/src/opentelemetry/instrumentation/aiohttp_client/__init__.py index 101e67f2ad..65e1601f34 100644 --- a/instrumentation/opentelemetry-instrumentation-aiohttp-client/src/opentelemetry/instrumentation/aiohttp_client/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-aiohttp-client/src/opentelemetry/instrumentation/aiohttp_client/__init__.py @@ -179,7 +179,7 @@ async def on_request_start( return http_method = params.method.upper() - request_span_name = f"HTTP {http_method}" + request_span_name = f"{http_method}" request_url = ( remove_url_credentials(trace_config_ctx.url_filter(params.url)) if callable(trace_config_ctx.url_filter) diff --git a/instrumentation/opentelemetry-instrumentation-aiohttp-client/src/opentelemetry/instrumentation/aiohttp_client/version.py b/instrumentation/opentelemetry-instrumentation-aiohttp-client/src/opentelemetry/instrumentation/aiohttp_client/version.py index 88ab55ca31..e9ca2a1777 100644 --- a/instrumentation/opentelemetry-instrumentation-aiohttp-client/src/opentelemetry/instrumentation/aiohttp_client/version.py +++ b/instrumentation/opentelemetry-instrumentation-aiohttp-client/src/opentelemetry/instrumentation/aiohttp_client/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-aiohttp-client/tests/test_aiohttp_client_integration.py b/instrumentation/opentelemetry-instrumentation-aiohttp-client/tests/test_aiohttp_client_integration.py index d9f76f0239..6af9d41900 100644 --- a/instrumentation/opentelemetry-instrumentation-aiohttp-client/tests/test_aiohttp_client_integration.py +++ b/instrumentation/opentelemetry-instrumentation-aiohttp-client/tests/test_aiohttp_client_integration.py @@ -23,6 +23,7 @@ import aiohttp import aiohttp.test_utils import yarl +from http_server_mock import HttpServerMock from pkg_resources import iter_entry_points from opentelemetry import context @@ -118,7 +119,7 @@ def test_status_codes(self): self.assert_spans( [ ( - "HTTP GET", + "GET", (span_status, None), { SpanAttributes.HTTP_METHOD: "GET", @@ -212,7 +213,7 @@ def strip_query_params(url: yarl.URL) -> str: self.assert_spans( [ ( - "HTTP GET", + "GET", (StatusCode.UNSET, None), { SpanAttributes.HTTP_METHOD: "GET", @@ -246,7 +247,7 @@ async def do_request(url): self.assert_spans( [ ( - "HTTP GET", + "GET", (expected_status, None), { SpanAttributes.HTTP_METHOD: "GET", @@ -273,7 +274,7 @@ async def request_handler(request): self.assert_spans( [ ( - "HTTP GET", + "GET", (StatusCode.ERROR, None), { SpanAttributes.HTTP_METHOD: "GET", @@ -300,7 +301,7 @@ async def request_handler(request): self.assert_spans( [ ( - "HTTP GET", + "GET", (StatusCode.ERROR, None), { SpanAttributes.HTTP_METHOD: "GET", @@ -313,27 +314,37 @@ async def request_handler(request): def test_credential_removal(self): trace_configs = [aiohttp_client.create_trace_config()] - url = "http://username:password@httpbin.org/status/200" - with self.subTest(url=url): + app = HttpServerMock("test_credential_removal") - async def do_request(url): - async with aiohttp.ClientSession( - trace_configs=trace_configs, - ) as session: - async with session.get(url): - pass + @app.route("/status/200") + def index(): + return "hello" - loop = asyncio.get_event_loop() - loop.run_until_complete(do_request(url)) + url = "http://username:password@localhost:5000/status/200" + + with app.run("localhost", 5000): + with self.subTest(url=url): + + async def do_request(url): + async with aiohttp.ClientSession( + trace_configs=trace_configs, + ) as session: + async with session.get(url): + pass + + loop = asyncio.get_event_loop() + loop.run_until_complete(do_request(url)) self.assert_spans( [ ( - "HTTP GET", + "GET", (StatusCode.UNSET, None), { SpanAttributes.HTTP_METHOD: "GET", - SpanAttributes.HTTP_URL: "http://httpbin.org/status/200", + SpanAttributes.HTTP_URL: ( + "http://localhost:5000/status/200" + ), SpanAttributes.HTTP_STATUS_CODE: int(HTTPStatus.OK), }, ) @@ -380,6 +391,7 @@ def test_instrument(self): self.get_default_request(), self.URL, self.default_handler ) span = self.assert_spans(1) + self.assertEqual("GET", span.name) self.assertEqual("GET", span.attributes[SpanAttributes.HTTP_METHOD]) self.assertEqual( f"http://{host}:{port}/test-path", diff --git a/instrumentation/opentelemetry-instrumentation-aiopg/pyproject.toml b/instrumentation/opentelemetry-instrumentation-aiopg/pyproject.toml index 2867ed8338..024f707d89 100644 --- a/instrumentation/opentelemetry-instrumentation-aiopg/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-aiopg/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-instrumentation-dbapi == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-instrumentation-dbapi == 0.42b0.dev", "wrapt >= 1.0.0, < 2.0.0", ] @@ -37,8 +37,8 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-aiopg[instruments]", - "opentelemetry-semantic-conventions == 0.39b0.dev", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-aiopg/src/opentelemetry/instrumentation/aiopg/version.py b/instrumentation/opentelemetry-instrumentation-aiopg/src/opentelemetry/instrumentation/aiopg/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-aiopg/src/opentelemetry/instrumentation/aiopg/version.py +++ b/instrumentation/opentelemetry-instrumentation-aiopg/src/opentelemetry/instrumentation/aiopg/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-asgi/pyproject.toml b/instrumentation/opentelemetry-instrumentation-asgi/pyproject.toml index f18ee611d8..60f15e9ba7 100644 --- a/instrumentation/opentelemetry-instrumentation-asgi/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-asgi/pyproject.toml @@ -27,9 +27,9 @@ classifiers = [ dependencies = [ "asgiref ~= 3.0", "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", - "opentelemetry-util-http == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", + "opentelemetry-util-http == 0.42b0.dev", ] [project.optional-dependencies] @@ -38,7 +38,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-asgi[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.urls] diff --git a/instrumentation/opentelemetry-instrumentation-asgi/src/opentelemetry/instrumentation/asgi/__init__.py b/instrumentation/opentelemetry-instrumentation-asgi/src/opentelemetry/instrumentation/asgi/__init__.py index 6fc88d3eeb..c0dcd39fd2 100644 --- a/instrumentation/opentelemetry-instrumentation-asgi/src/opentelemetry/instrumentation/asgi/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-asgi/src/opentelemetry/instrumentation/asgi/__init__.py @@ -415,18 +415,23 @@ def set_status_code(span, status_code): def get_default_span_details(scope: dict) -> Tuple[str, dict]: - """Default implementation for get_default_span_details + """ + Default span name is the HTTP method and URL path, or just the method. + https://github.com/open-telemetry/opentelemetry-specification/pull/3165 + https://opentelemetry.io/docs/reference/specification/trace/semantic_conventions/http/#name + Args: scope: the ASGI scope dictionary Returns: a tuple of the span name, and any attributes to attach to the span. """ - span_name = ( - scope.get("path", "").strip() - or f"HTTP {scope.get('method', '').strip()}" - ) - - return span_name, {} + path = scope.get("path", "").strip() + method = scope.get("method", "").strip() + if method and path: # http + return f"{method} {path}", {} + if path: # websocket + return path, {} + return method, {} # http with no path def _collect_target_attribute( @@ -501,6 +506,16 @@ def __init__( unit="ms", description="measures the duration of the inbound HTTP request", ) + self.server_response_size_histogram = self.meter.create_histogram( + name=MetricInstruments.HTTP_SERVER_RESPONSE_SIZE, + unit="By", + description="measures the size of HTTP response messages (compressed).", + ) + self.server_request_size_histogram = self.meter.create_histogram( + name=MetricInstruments.HTTP_SERVER_REQUEST_SIZE, + unit="By", + description="Measures the size of HTTP request messages (compressed).", + ) self.active_requests_counter = self.meter.create_up_down_counter( name=MetricInstruments.HTTP_SERVER_ACTIVE_REQUESTS, unit="requests", @@ -513,6 +528,7 @@ def __init__( self.server_request_hook = server_request_hook self.client_request_hook = client_request_hook self.client_response_hook = client_response_hook + self.content_length_header = None async def __call__(self, scope, receive, send): """The ASGI application @@ -522,6 +538,7 @@ async def __call__(self, scope, receive, send): receive: An awaitable callable yielding dictionaries send: An awaitable callable taking a single dictionary as argument. """ + start = default_timer() if scope["type"] not in ("http", "websocket"): return await self.app(scope, receive, send) @@ -575,7 +592,6 @@ async def __call__(self, scope, receive, send): send, duration_attrs, ) - start = default_timer() await self.app(scope, otel_receive, otel_send) finally: @@ -588,6 +604,20 @@ async def __call__(self, scope, receive, send): self.active_requests_counter.add( -1, active_requests_count_attrs ) + if self.content_length_header: + self.server_response_size_histogram.record( + self.content_length_header, duration_attrs + ) + request_size = asgi_getter.get(scope, "content-length") + if request_size: + try: + request_size_amount = int(request_size[0]) + except ValueError: + pass + else: + self.server_request_size_histogram.record( + request_size_amount, duration_attrs + ) if token: context.detach(token) @@ -655,6 +685,13 @@ async def otel_send(message): setter=asgi_setter, ) + content_length = asgi_getter.get(message, "content-length") + if content_length: + try: + self.content_length_header = int(content_length[0]) + except ValueError: + pass + await send(message) return otel_send diff --git a/instrumentation/opentelemetry-instrumentation-asgi/src/opentelemetry/instrumentation/asgi/version.py b/instrumentation/opentelemetry-instrumentation-asgi/src/opentelemetry/instrumentation/asgi/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-asgi/src/opentelemetry/instrumentation/asgi/version.py +++ b/instrumentation/opentelemetry-instrumentation-asgi/src/opentelemetry/instrumentation/asgi/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-asgi/tests/test_asgi_middleware.py b/instrumentation/opentelemetry-instrumentation-asgi/tests/test_asgi_middleware.py index bfa5720f99..cb22174482 100644 --- a/instrumentation/opentelemetry-instrumentation-asgi/tests/test_asgi_middleware.py +++ b/instrumentation/opentelemetry-instrumentation-asgi/tests/test_asgi_middleware.py @@ -14,6 +14,7 @@ # pylint: disable=too-many-lines +import asyncio import sys import unittest from timeit import default_timer @@ -46,22 +47,30 @@ _expected_metric_names = [ "http.server.active_requests", "http.server.duration", + "http.server.response.size", + "http.server.request.size", ] _recommended_attrs = { "http.server.active_requests": _active_requests_count_attrs, "http.server.duration": _duration_attrs, + "http.server.response.size": _duration_attrs, + "http.server.request.size": _duration_attrs, } async def http_app(scope, receive, send): message = await receive() + scope["headers"] = [(b"content-length", b"128")] assert scope["type"] == "http" if message.get("type") == "http.request": await send( { "type": "http.response.start", "status": 200, - "headers": [[b"Content-Type", b"text/plain"]], + "headers": [ + [b"Content-Type", b"text/plain"], + [b"content-length", b"1024"], + ], } ) await send({"type": "http.response.body", "body": b"*"}) @@ -94,6 +103,7 @@ async def error_asgi(scope, receive, send): assert isinstance(scope, dict) assert scope["type"] == "http" message = await receive() + scope["headers"] = [(b"content-length", b"128")] if message.get("type") == "http.request": try: raise ValueError @@ -103,7 +113,10 @@ async def error_asgi(scope, receive, send): { "type": "http.response.start", "status": 200, - "headers": [[b"Content-Type", b"text/plain"]], + "headers": [ + [b"Content-Type", b"text/plain"], + [b"content-length", b"1024"], + ], } ) await send({"type": "http.response.body", "body": b"*"}) @@ -126,7 +139,8 @@ def validate_outputs(self, outputs, error=None, modifiers=None): # Check http response start self.assertEqual(response_start["status"], 200) self.assertEqual( - response_start["headers"], [[b"Content-Type", b"text/plain"]] + response_start["headers"], + [[b"Content-Type", b"text/plain"], [b"content-length", b"1024"]], ) exc_info = self.scope.get("hack_exc_info") @@ -142,12 +156,12 @@ def validate_outputs(self, outputs, error=None, modifiers=None): self.assertEqual(len(span_list), 4) expected = [ { - "name": "/ http receive", + "name": "GET / http receive", "kind": trace_api.SpanKind.INTERNAL, "attributes": {"type": "http.request"}, }, { - "name": "/ http send", + "name": "GET / http send", "kind": trace_api.SpanKind.INTERNAL, "attributes": { SpanAttributes.HTTP_STATUS_CODE: 200, @@ -155,12 +169,12 @@ def validate_outputs(self, outputs, error=None, modifiers=None): }, }, { - "name": "/ http send", + "name": "GET / http send", "kind": trace_api.SpanKind.INTERNAL, "attributes": {"type": "http.response.body"}, }, { - "name": "/", + "name": "GET /", "kind": trace_api.SpanKind.SERVER, "attributes": { SpanAttributes.HTTP_METHOD: "GET", @@ -231,7 +245,7 @@ def update_expected_span_name(expected): entry["name"] = span_name else: entry["name"] = " ".join( - [span_name] + entry["name"].split(" ")[1:] + [span_name] + entry["name"].split(" ")[2:] ) return expected @@ -352,6 +366,7 @@ def test_traceresponse_header(self): response_start["headers"], [ [b"Content-Type", b"text/plain"], + [b"content-length", b"1024"], [b"traceresponse", f"{traceresponse}".encode()], [b"access-control-expose-headers", b"traceresponse"], ], @@ -493,9 +508,9 @@ def update_expected_hook_results(expected): for entry in expected: if entry["kind"] == trace_api.SpanKind.SERVER: entry["name"] = "name from server hook" - elif entry["name"] == "/ http receive": + elif entry["name"] == "GET / http receive": entry["name"] = "name from client request hook" - elif entry["name"] == "/ http send": + elif entry["name"] == "GET / http send": entry["attributes"].update({"attr-from-hook": "value"}) return expected @@ -565,6 +580,7 @@ def test_basic_metric_success(self): "http.flavor": "1.0", } metrics_list = self.memory_metrics_reader.get_metrics_data() + # pylint: disable=too-many-nested-blocks for resource_metric in metrics_list.resource_metrics: for scope_metrics in resource_metric.scope_metrics: for metric in scope_metrics.metrics: @@ -575,9 +591,14 @@ def test_basic_metric_success(self): dict(point.attributes), ) self.assertEqual(point.count, 1) - self.assertAlmostEqual( - duration, point.sum, delta=5 - ) + if metric.name == "http.server.duration": + self.assertAlmostEqual( + duration, point.sum, delta=5 + ) + elif metric.name == "http.server.response.size": + self.assertEqual(1024, point.sum) + elif metric.name == "http.server.request.size": + self.assertEqual(128, point.sum) elif isinstance(point, NumberDataPoint): self.assertDictEqual( expected_requests_count_attributes, @@ -602,13 +623,12 @@ async def target_asgi(scope, receive, send): app = otel_asgi.OpenTelemetryMiddleware(target_asgi) self.seed_app(app) self.send_default_request() - metrics_list = self.memory_metrics_reader.get_metrics_data() assertions = 0 for resource_metric in metrics_list.resource_metrics: for scope_metrics in resource_metric.scope_metrics: for metric in scope_metrics.metrics: - if metric.name != "http.server.duration": + if metric.name == "http.server.active_requests": continue for point in metric.data.data_points: if isinstance(point, HistogramDataPoint): @@ -617,7 +637,7 @@ async def target_asgi(scope, receive, send): expected_target, ) assertions += 1 - self.assertEqual(assertions, 1) + self.assertEqual(assertions, 3) def test_no_metric_for_websockets(self): self.scope = { @@ -705,11 +725,11 @@ def test_response_attributes_invalid_status_code(self): self.assertEqual(self.span.set_status.call_count, 1) def test_credential_removal(self): - self.scope["server"] = ("username:password@httpbin.org", 80) + self.scope["server"] = ("username:password@mock", 80) self.scope["path"] = "/status/200" attrs = otel_asgi.collect_request_attributes(self.scope) self.assertEqual( - attrs[SpanAttributes.HTTP_URL], "http://httpbin.org/status/200" + attrs[SpanAttributes.HTTP_URL], "http://mock/status/200" ) def test_collect_target_attribute_missing(self): @@ -777,5 +797,38 @@ async def wrapped_app(scope, receive, send): ) +class TestAsgiApplicationRaisingError(AsgiTestBase): + def tearDown(self): + pass + + @mock.patch( + "opentelemetry.instrumentation.asgi.collect_custom_request_headers_attributes", + side_effect=ValueError("whatever"), + ) + def test_asgi_issue_1883( + self, mock_collect_custom_request_headers_attributes + ): + """ + Test that exception UnboundLocalError local variable 'start' referenced before assignment is not raised + See https://github.com/open-telemetry/opentelemetry-python-contrib/issues/1883 + """ + app = otel_asgi.OpenTelemetryMiddleware(simple_asgi) + self.seed_app(app) + self.send_default_request() + try: + asyncio.get_event_loop().run_until_complete( + self.communicator.stop() + ) + except ValueError as exc_info: + self.assertEqual(exc_info.args[0], "whatever") + except Exception as exc_info: # pylint: disable=W0703 + self.fail( + "expecting ValueError('whatever'), received instead: " + + str(exc_info) + ) + else: + self.fail("expecting ValueError('whatever')") + + if __name__ == "__main__": unittest.main() diff --git a/instrumentation/opentelemetry-instrumentation-asyncpg/pyproject.toml b/instrumentation/opentelemetry-instrumentation-asyncpg/pyproject.toml index 672623ccaf..41a7e3ef3c 100644 --- a/instrumentation/opentelemetry-instrumentation-asyncpg/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-asyncpg/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", ] [project.optional-dependencies] @@ -36,7 +36,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-asyncpg[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-asyncpg/src/opentelemetry/instrumentation/asyncpg/version.py b/instrumentation/opentelemetry-instrumentation-asyncpg/src/opentelemetry/instrumentation/asyncpg/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-asyncpg/src/opentelemetry/instrumentation/asyncpg/version.py +++ b/instrumentation/opentelemetry-instrumentation-asyncpg/src/opentelemetry/instrumentation/asyncpg/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-aws-lambda/pyproject.toml b/instrumentation/opentelemetry-instrumentation-aws-lambda/pyproject.toml index 49b6ee8911..97c0a01245 100644 --- a/instrumentation/opentelemetry-instrumentation-aws-lambda/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-aws-lambda/pyproject.toml @@ -22,15 +22,15 @@ classifiers = [ "Programming Language :: Python :: 3.8", ] dependencies = [ - "opentelemetry-instrumentation == 0.38b0", + "opentelemetry-instrumentation == 0.41b0", "opentelemetry-propagator-aws-xray == 1.0.1", - "opentelemetry-semantic-conventions == 0.38b0", + "opentelemetry-semantic-conventions == 0.41b0", ] [project.optional-dependencies] instruments = [] test = [ - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.41b0", ] [project.urls] diff --git a/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/__init__.py b/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/__init__.py index 8b322b2a94..32c6725235 100644 --- a/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/__init__.py @@ -230,7 +230,6 @@ def _set_api_gateway_v1_proxy_attributes( span.set_attribute( SpanAttributes.HTTP_METHOD, lambda_event.get("httpMethod") ) - span.set_attribute(SpanAttributes.HTTP_ROUTE, lambda_event.get("resource")) if lambda_event.get("body"): span.set_attribute( @@ -239,27 +238,33 @@ def _set_api_gateway_v1_proxy_attributes( ) if lambda_event.get("headers"): - span.set_attribute( - SpanAttributes.HTTP_USER_AGENT, - lambda_event["headers"].get("User-Agent"), - ) - span.set_attribute( - SpanAttributes.HTTP_SCHEME, - lambda_event["headers"].get("X-Forwarded-Proto"), - ) - span.set_attribute( - SpanAttributes.NET_HOST_NAME, lambda_event["headers"].get("Host") - ) + if "User-Agent" in lambda_event["headers"]: + span.set_attribute( + SpanAttributes.HTTP_USER_AGENT, + lambda_event["headers"]["User-Agent"], + ) + if "X-Forwarded-Proto" in lambda_event["headers"]: + span.set_attribute( + SpanAttributes.HTTP_SCHEME, + lambda_event["headers"]["X-Forwarded-Proto"], + ) + if "Host" in lambda_event["headers"]: + span.set_attribute( + SpanAttributes.NET_HOST_NAME, + lambda_event["headers"]["Host"], + ) + if "resource" in lambda_event: + span.set_attribute(SpanAttributes.HTTP_ROUTE, lambda_event["resource"]) - if lambda_event.get("queryStringParameters"): - span.set_attribute( - SpanAttributes.HTTP_TARGET, - f"{lambda_event.get('resource')}?{urlencode(lambda_event.get('queryStringParameters'))}", - ) - else: - span.set_attribute( - SpanAttributes.HTTP_TARGET, lambda_event.get("resource") - ) + if lambda_event.get("queryStringParameters"): + span.set_attribute( + SpanAttributes.HTTP_TARGET, + f"{lambda_event['resource']}?{urlencode(lambda_event['queryStringParameters'])}", + ) + else: + span.set_attribute( + SpanAttributes.HTTP_TARGET, lambda_event["resource"] + ) return span @@ -272,6 +277,12 @@ def _set_api_gateway_v2_proxy_attributes( More info: https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-integrations-lambda.html """ + if "domainName" in lambda_event["requestContext"]: + span.set_attribute( + SpanAttributes.NET_HOST_NAME, + lambda_event["requestContext"]["domainName"], + ) + if lambda_event.get("body"): span.set_attribute( "http.request.body", @@ -284,29 +295,31 @@ def _set_api_gateway_v2_proxy_attributes( ) if lambda_event["requestContext"].get("http"): - span.set_attribute( - SpanAttributes.HTTP_METHOD, - lambda_event["requestContext"]["http"].get("method"), - ) - span.set_attribute( - SpanAttributes.HTTP_USER_AGENT, - lambda_event["requestContext"]["http"].get("userAgent"), - ) - span.set_attribute( - SpanAttributes.HTTP_ROUTE, - lambda_event["requestContext"]["http"].get("path"), - ) - - if lambda_event.get("rawQueryString"): + if "method" in lambda_event["requestContext"]["http"]: span.set_attribute( - SpanAttributes.HTTP_TARGET, - f"{lambda_event['requestContext']['http'].get('path')}?{lambda_event.get('rawQueryString')}", + SpanAttributes.HTTP_METHOD, + lambda_event["requestContext"]["http"]["method"], ) - else: + if "userAgent" in lambda_event["requestContext"]["http"]: span.set_attribute( - SpanAttributes.HTTP_TARGET, - lambda_event["requestContext"]["http"].get("path"), + SpanAttributes.HTTP_USER_AGENT, + lambda_event["requestContext"]["http"]["userAgent"], ) + if "path" in lambda_event["requestContext"]["http"]: + span.set_attribute( + SpanAttributes.HTTP_ROUTE, + lambda_event["requestContext"]["http"]["path"], + ) + if lambda_event.get("rawQueryString"): + span.set_attribute( + SpanAttributes.HTTP_TARGET, + f"{lambda_event['requestContext']['http']['path']}?{lambda_event['rawQueryString']}", + ) + else: + span.set_attribute( + SpanAttributes.HTTP_TARGET, + lambda_event["requestContext"]["http"]["path"], + ) return span diff --git a/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/version.py b/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/version.py +++ b/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-boto/pyproject.toml b/instrumentation/opentelemetry-instrumentation-boto/pyproject.toml index 83c23f01cf..125f311045 100644 --- a/instrumentation/opentelemetry-instrumentation-boto/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-boto/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", ] [project.optional-dependencies] @@ -38,7 +38,7 @@ test = [ "opentelemetry-instrumentation-boto[instruments]", "markupsafe==2.0.1", "moto~=2.0", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-boto/src/opentelemetry/instrumentation/boto/version.py b/instrumentation/opentelemetry-instrumentation-boto/src/opentelemetry/instrumentation/boto/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-boto/src/opentelemetry/instrumentation/boto/version.py +++ b/instrumentation/opentelemetry-instrumentation-boto/src/opentelemetry/instrumentation/boto/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-boto3sqs/pyproject.toml b/instrumentation/opentelemetry-instrumentation-boto3sqs/pyproject.toml index 386602644f..5a91c1d206 100644 --- a/instrumentation/opentelemetry-instrumentation-boto3sqs/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-boto3sqs/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", "wrapt >= 1.0.0, < 2.0.0", ] @@ -37,7 +37,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-boto3sqs[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-boto3sqs/src/opentelemetry/instrumentation/boto3sqs/version.py b/instrumentation/opentelemetry-instrumentation-boto3sqs/src/opentelemetry/instrumentation/boto3sqs/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-boto3sqs/src/opentelemetry/instrumentation/boto3sqs/version.py +++ b/instrumentation/opentelemetry-instrumentation-boto3sqs/src/opentelemetry/instrumentation/boto3sqs/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-botocore/pyproject.toml b/instrumentation/opentelemetry-instrumentation-botocore/pyproject.toml index d643d16548..c02c3bdf80 100644 --- a/instrumentation/opentelemetry-instrumentation-botocore/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-botocore/pyproject.toml @@ -26,8 +26,9 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.38b0", - "opentelemetry-semantic-conventions == 0.38b0", + "opentelemetry-instrumentation == 0.41b0", + "opentelemetry-semantic-conventions == 0.41b0", + "opentelemetry-propagator-aws-xray == 1.0.1", ] [project.optional-dependencies] @@ -38,7 +39,7 @@ test = [ "opentelemetry-instrumentation-botocore[instruments]", "markupsafe==2.0.1", "moto[all] ~= 2.2.6", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.41b0", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-botocore/src/opentelemetry/instrumentation/botocore/__init__.py b/instrumentation/opentelemetry-instrumentation-botocore/src/opentelemetry/instrumentation/botocore/__init__.py index f80f13915e..3e6c51c330 100644 --- a/instrumentation/opentelemetry-instrumentation-botocore/src/opentelemetry/instrumentation/botocore/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-botocore/src/opentelemetry/instrumentation/botocore/__init__.py @@ -220,6 +220,7 @@ def _patched_api_call(self, original_func, instance, args, kwargs): attributes["rpc.request.payload"] = limit_string_size(self.payload_size_limit,json.dumps(body, default=str)) elif call_context.service == "events" and call_context.operation == "PutEvents": call_context.span_kind = SpanKind.PRODUCER + attributes["rpc.request.payload"] = limit_string_size(self.payload_size_limit, json.dumps(call_context.params, default=str)) else: attributes["rpc.request.payload"] = limit_string_size(self.payload_size_limit, json.dumps(call_context.params, default=str)) except Exception as ex: @@ -471,7 +472,7 @@ def set( val = {"DataType": "String", "StringValue": value} carrier[key] = val -def limit_string_size(s: str, max_size: int) -> str: +def limit_string_size(max_size: int, s: str) -> str: if len(s) > max_size: return s[:max_size] else: diff --git a/instrumentation/opentelemetry-instrumentation-botocore/src/opentelemetry/instrumentation/botocore/version.py b/instrumentation/opentelemetry-instrumentation-botocore/src/opentelemetry/instrumentation/botocore/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-botocore/src/opentelemetry/instrumentation/botocore/version.py +++ b/instrumentation/opentelemetry-instrumentation-botocore/src/opentelemetry/instrumentation/botocore/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-botocore/tests/test_botocore_instrumentation.py b/instrumentation/opentelemetry-instrumentation-botocore/tests/test_botocore_instrumentation.py index 9a5d8429b5..3d25dcbf2d 100644 --- a/instrumentation/opentelemetry-instrumentation-botocore/tests/test_botocore_instrumentation.py +++ b/instrumentation/opentelemetry-instrumentation-botocore/tests/test_botocore_instrumentation.py @@ -36,9 +36,11 @@ from opentelemetry.instrumentation.botocore import BotocoreInstrumentor from opentelemetry.instrumentation.utils import _SUPPRESS_INSTRUMENTATION_KEY from opentelemetry.propagate import get_global_textmap, set_global_textmap +from opentelemetry.propagators.aws.aws_xray_propagator import TRACE_HEADER_KEY from opentelemetry.semconv.trace import SpanAttributes from opentelemetry.test.mock_textmap import MockTextMapPropagator from opentelemetry.test.test_base import TestBase +from opentelemetry.trace.span import format_span_id, format_trace_id _REQUEST_ID_REGEX_MATCH = r"[A-Z0-9]{52}" @@ -225,27 +227,21 @@ def test_unpatch(self): @mock_ec2 def test_uninstrument_does_not_inject_headers(self): headers = {} - previous_propagator = get_global_textmap() - try: - set_global_textmap(MockTextMapPropagator()) - def intercept_headers(**kwargs): - headers.update(kwargs["request"].headers) + def intercept_headers(**kwargs): + headers.update(kwargs["request"].headers) - ec2 = self._make_client("ec2") + ec2 = self._make_client("ec2") - BotocoreInstrumentor().uninstrument() + BotocoreInstrumentor().uninstrument() - ec2.meta.events.register_first( - "before-send.ec2.DescribeInstances", intercept_headers - ) - with self.tracer_provider.get_tracer("test").start_span("parent"): - ec2.describe_instances() + ec2.meta.events.register_first( + "before-send.ec2.DescribeInstances", intercept_headers + ) + with self.tracer_provider.get_tracer("test").start_span("parent"): + ec2.describe_instances() - self.assertNotIn(MockTextMapPropagator.TRACE_ID_KEY, headers) - self.assertNotIn(MockTextMapPropagator.SPAN_ID_KEY, headers) - finally: - set_global_textmap(previous_propagator) + self.assertNotIn(TRACE_HEADER_KEY, headers) @mock_sqs def test_double_patch(self): @@ -306,20 +302,42 @@ def check_headers(**kwargs): "EC2", "DescribeInstances", request_id=request_id ) - self.assertIn(MockTextMapPropagator.TRACE_ID_KEY, headers) - self.assertEqual( - str(span.get_span_context().trace_id), - headers[MockTextMapPropagator.TRACE_ID_KEY], + # only x-ray propagation is used in HTTP requests + self.assertIn(TRACE_HEADER_KEY, headers) + xray_context = headers[TRACE_HEADER_KEY] + formated_trace_id = format_trace_id( + span.get_span_context().trace_id ) - self.assertIn(MockTextMapPropagator.SPAN_ID_KEY, headers) - self.assertEqual( - str(span.get_span_context().span_id), - headers[MockTextMapPropagator.SPAN_ID_KEY], + formated_trace_id = ( + formated_trace_id[:8] + "-" + formated_trace_id[8:] ) + self.assertEqual( + xray_context.lower(), + f"root=1-{formated_trace_id};parent={format_span_id(span.get_span_context().span_id)};sampled=1".lower(), + ) finally: set_global_textmap(previous_propagator) + @mock_ec2 + def test_override_xray_propagator_injects_into_request(self): + headers = {} + + def check_headers(**kwargs): + nonlocal headers + headers = kwargs["request"].headers + + BotocoreInstrumentor().instrument() + + ec2 = self._make_client("ec2") + ec2.meta.events.register_first( + "before-send.ec2.DescribeInstances", check_headers + ) + ec2.describe_instances() + + self.assertNotIn(MockTextMapPropagator.TRACE_ID_KEY, headers) + self.assertNotIn(MockTextMapPropagator.SPAN_ID_KEY, headers) + @mock_xray def test_suppress_instrumentation_xray_client(self): xray_client = self._make_client("xray") diff --git a/instrumentation/opentelemetry-instrumentation-cassandra/LICENSE b/instrumentation/opentelemetry-instrumentation-cassandra/LICENSE new file mode 100644 index 0000000000..1ef7dad2c5 --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-cassandra/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright The OpenTelemetry Authors + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/instrumentation/opentelemetry-instrumentation-cassandra/README.rst b/instrumentation/opentelemetry-instrumentation-cassandra/README.rst new file mode 100644 index 0000000000..36e7d6202e --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-cassandra/README.rst @@ -0,0 +1,25 @@ +OpenTelemetry Cassandra Instrumentation +=================================== + +|pypi| + +.. |pypi| image:: https://badge.fury.io/py/opentelemetry-instrumentation-cassandra.svg + :target: https://pypi.org/project/opentelemetry-instrumentation-cassandra/ + +Instrumentation for Cassandra (cassandra-driver and scylla-driver libraries). + + +Installation +------------ + +:: + + pip install opentelemetry-instrumentation-cassandra + + +References +---------- +* `OpenTelemetry Cassandra Instrumentation `_ +* `OpenTelemetry Project `_ +* `OpenTelemetry Python Examples `_ + diff --git a/instrumentation/opentelemetry-instrumentation-cassandra/pyproject.toml b/instrumentation/opentelemetry-instrumentation-cassandra/pyproject.toml new file mode 100644 index 0000000000..9abb17598f --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-cassandra/pyproject.toml @@ -0,0 +1,60 @@ +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[project] +name = "opentelemetry-instrumentation-cassandra" +dynamic = ["version"] +description = "OpenTelemetry Cassandra instrumentation" +readme = "README.rst" +license = "Apache-2.0" +requires-python = ">=3.7" +authors = [ + { name = "OpenTelemetry Authors", email = "cncf-opentelemetry-contributors@lists.cncf.io" }, +] +classifiers = [ + "Development Status :: 4 - Beta", + "Intended Audience :: Developers", + "License :: OSI Approved :: Apache Software License", + "Programming Language :: Python", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.7", + "Programming Language :: Python :: 3.8", + "Programming Language :: Python :: 3.9", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", +] +dependencies = [ + "opentelemetry-api ~= 1.12", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", + "wrapt >= 1.0.0, < 2.0.0", +] + +[project.optional-dependencies] +instruments = [ + "cassandra-driver ~= 3.25", + "scylla-driver ~= 3.25", +] +test = [ + "opentelemetry-instrumentation-cassandra[instruments]", + "opentelemetry-test-utils == 0.42b0.dev", +] + +[project.entry-points.opentelemetry_instrumentor] +cassandra = "opentelemetry.instrumentation.cassandra:CassandraInstrumentor" + +[project.urls] +Homepage = "https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation/opentelemetry-instrumentation-cassandra" + +[tool.hatch.version] +path = "src/opentelemetry/instrumentation/cassandra/version.py" + +[tool.hatch.build.targets.sdist] +include = [ + "/src", + "/tests", +] + +[tool.hatch.build.targets.wheel] +packages = ["src/opentelemetry"] diff --git a/instrumentation/opentelemetry-instrumentation-cassandra/src/opentelemetry/instrumentation/cassandra/__init__.py b/instrumentation/opentelemetry-instrumentation-cassandra/src/opentelemetry/instrumentation/cassandra/__init__.py new file mode 100644 index 0000000000..6a4ee7edc5 --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-cassandra/src/opentelemetry/instrumentation/cassandra/__init__.py @@ -0,0 +1,91 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""" +Cassandra instrumentation supporting `cassandra-driver`_ and `scylla-driver`_, it can be enabled by +using ``CassandraInstrumentor``. + +.. _cassandra-driver: https://pypi.org/project/cassandra-driver/ +.. _scylla-driver: https://pypi.org/project/scylla-driver/ + +Usage +----- + +.. code:: python + + import cassandra.cluster + from opentelemetry.instrumentation.cassandra import CassandraInstrumentor + + CassandraInstrumentor().instrument() + + cluster = cassandra.cluster.Cluster() + session = cluster.connect() + rows = session.execute("SELECT * FROM test") + +API +--- +""" + +from typing import Collection + +import cassandra.cluster +from wrapt import wrap_function_wrapper + +from opentelemetry import trace +from opentelemetry.instrumentation.instrumentor import BaseInstrumentor +from opentelemetry.instrumentation.cassandra.package import _instruments +from opentelemetry.instrumentation.cassandra.version import __version__ +from opentelemetry.instrumentation.utils import unwrap +from opentelemetry.semconv.trace import SpanAttributes + + +def _instrument(tracer_provider, include_db_statement=False): + """Instruments the cassandra-driver/scylla-driver module + + Wraps cassandra.cluster.Session.execute_async(). + """ + tracer = trace.get_tracer(__name__, __version__, tracer_provider) + name = "Cassandra" + + def _traced_execute_async(func, instance, args, kwargs): + with tracer.start_as_current_span( + name, kind=trace.SpanKind.CLIENT + ) as span: + if span.is_recording(): + span.set_attribute(SpanAttributes.DB_NAME, instance.keyspace) + span.set_attribute(SpanAttributes.DB_SYSTEM, "cassandra") + span.set_attribute(SpanAttributes.NET_PEER_NAME, instance.cluster.contact_points) + + if include_db_statement: + query = args[0] + span.set_attribute(SpanAttributes.DB_STATEMENT, str(query)) + + response = func(*args, **kwargs) + return response + + wrap_function_wrapper("cassandra.cluster", "Session.execute_async", _traced_execute_async) + + +class CassandraInstrumentor(BaseInstrumentor): + def instrumentation_dependencies(self) -> Collection[str]: + return _instruments + + def _instrument(self, **kwargs): + _instrument( + tracer_provider=kwargs.get("tracer_provider"), + include_db_statement=kwargs.get("include_db_statement"), + ) + + def _uninstrument(self, **kwargs): + unwrap(cassandra.cluster.Session, "execute_async") diff --git a/instrumentation/opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/environment_variables.py b/instrumentation/opentelemetry-instrumentation-cassandra/src/opentelemetry/instrumentation/cassandra/package.py similarity index 85% rename from instrumentation/opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/environment_variables.py rename to instrumentation/opentelemetry-instrumentation-cassandra/src/opentelemetry/instrumentation/cassandra/package.py index 750b97445e..d10a3cb5ba 100644 --- a/instrumentation/opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/environment_variables.py +++ b/instrumentation/opentelemetry-instrumentation-cassandra/src/opentelemetry/instrumentation/cassandra/package.py @@ -12,6 +12,5 @@ # See the License for the specific language governing permissions and # limitations under the License. -OTEL_PYTHON_INSTRUMENTATION_SANITIZE_REDIS = ( - "OTEL_PYTHON_INSTRUMENTATION_SANITIZE_REDIS" -) + +_instruments = ("cassandra-driver ~= 3.25", "scylla-driver ~= 3.25") diff --git a/instrumentation/opentelemetry-instrumentation-cassandra/src/opentelemetry/instrumentation/cassandra/version.py b/instrumentation/opentelemetry-instrumentation-cassandra/src/opentelemetry/instrumentation/cassandra/version.py new file mode 100644 index 0000000000..c2996671d6 --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-cassandra/src/opentelemetry/instrumentation/cassandra/version.py @@ -0,0 +1,15 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-cassandra/tests/__init__.py b/instrumentation/opentelemetry-instrumentation-cassandra/tests/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/instrumentation/opentelemetry-instrumentation-cassandra/tests/test_cassandra_integration.py b/instrumentation/opentelemetry-instrumentation-cassandra/tests/test_cassandra_integration.py new file mode 100644 index 0000000000..6977e1b2a2 --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-cassandra/tests/test_cassandra_integration.py @@ -0,0 +1,121 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from unittest import mock + +import cassandra.cluster +from wrapt import BoundFunctionWrapper + +import opentelemetry.instrumentation.cassandra +from opentelemetry import trace as trace_api +from opentelemetry.instrumentation.cassandra import CassandraInstrumentor +from opentelemetry.sdk import resources +from opentelemetry.test.test_base import TestBase +from opentelemetry.trace import SpanKind + + +def connect_and_execute_query(): + cluster = cassandra.cluster.Cluster() + cluster._is_setup = True + session = cluster.connect() + session.cluster = cluster + session.keyspace = "test" + session._request_init_callbacks = [] + query = "SELECT * FROM test" + session.execute(query) + return cluster, session, query + + +class TestCassandraIntegration(TestBase): + def tearDown(self): + super().tearDown() + with self.disable_logging(): + CassandraInstrumentor().uninstrument() + + def test_instrument_uninstrument(self): + instrumentation = CassandraInstrumentor() + instrumentation.instrument() + self.assertTrue(isinstance(cassandra.cluster.Session.execute_async, BoundFunctionWrapper)) + + instrumentation.uninstrument() + self.assertFalse(isinstance(cassandra.cluster.Session.execute_async, BoundFunctionWrapper)) + + @mock.patch("cassandra.cluster.Cluster.connect") + @mock.patch("cassandra.cluster.Session.__init__") + @mock.patch("cassandra.cluster.Session._create_response_future") + def test_instrumentor(self, mock_create_response_future, mock_session_init, mock_connect): + mock_create_response_future.return_value = mock.Mock() + mock_session_init.return_value = None + mock_connect.return_value = cassandra.cluster.Session() + + CassandraInstrumentor().instrument() + + connect_and_execute_query() + + spans_list = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans_list), 1) + span = spans_list[0] + + # Check version and name in span's instrumentation info + self.assertEqualSpanInstrumentationInfo( + span, opentelemetry.instrumentation.cassandra + ) + self.assertEqual(span.name, "Cassandra") + self.assertEqual(span.kind, SpanKind.CLIENT) + + # check that no spans are generated after uninstrument + CassandraInstrumentor().uninstrument() + + connect_and_execute_query() + + spans_list = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans_list), 1) + + @mock.patch("cassandra.cluster.Cluster.connect") + @mock.patch("cassandra.cluster.Session.__init__") + @mock.patch("cassandra.cluster.Session._create_response_future") + def test_custom_tracer_provider(self, mock_create_response_future, mock_session_init, mock_connect): + mock_create_response_future.return_value = mock.Mock() + mock_session_init.return_value = None + mock_connect.return_value = cassandra.cluster.Session() + + resource = resources.Resource.create({}) + result = self.create_tracer_provider(resource=resource) + tracer_provider, exporter = result + + CassandraInstrumentor().instrument(tracer_provider=tracer_provider) + + connect_and_execute_query() + + span_list = exporter.get_finished_spans() + self.assertEqual(len(span_list), 1) + span = span_list[0] + + self.assertIs(span.resource, resource) + + @mock.patch("cassandra.cluster.Cluster.connect") + @mock.patch("cassandra.cluster.Session.__init__") + @mock.patch("cassandra.cluster.Session._create_response_future") + def test_instrument_connection_no_op_tracer_provider(self, mock_create_response_future, mock_session_init, mock_connect): + mock_create_response_future.return_value = mock.Mock() + mock_session_init.return_value = None + mock_connect.return_value = cassandra.cluster.Session() + + tracer_provider = trace_api.NoOpTracerProvider() + CassandraInstrumentor().instrument(tracer_provider=tracer_provider) + + connect_and_execute_query() + + spans_list = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans_list), 0) diff --git a/instrumentation/opentelemetry-instrumentation-celery/pyproject.toml b/instrumentation/opentelemetry-instrumentation-celery/pyproject.toml index 816e6002c8..2c0de681e1 100644 --- a/instrumentation/opentelemetry-instrumentation-celery/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-celery/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", ] [project.optional-dependencies] @@ -36,7 +36,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-celery[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", "pytest", ] diff --git a/instrumentation/opentelemetry-instrumentation-celery/src/opentelemetry/instrumentation/celery/__init__.py b/instrumentation/opentelemetry-instrumentation-celery/src/opentelemetry/instrumentation/celery/__init__.py index cb265b46f8..bb83a5c192 100644 --- a/instrumentation/opentelemetry-instrumentation-celery/src/opentelemetry/instrumentation/celery/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-celery/src/opentelemetry/instrumentation/celery/__init__.py @@ -60,8 +60,10 @@ def add(x, y): """ import logging +from timeit import default_timer from typing import Collection, Iterable +from billiard.einfo import ExceptionInfo from celery import signals # pylint: disable=no-name-in-module from opentelemetry import trace @@ -69,10 +71,18 @@ def add(x, y): from opentelemetry.instrumentation.celery.package import _instruments from opentelemetry.instrumentation.celery.version import __version__ from opentelemetry.instrumentation.instrumentor import BaseInstrumentor +from opentelemetry.metrics import get_meter from opentelemetry.propagate import extract, inject from opentelemetry.propagators.textmap import Getter from opentelemetry.semconv.trace import SpanAttributes from opentelemetry.trace.status import Status, StatusCode +from billiard import VERSION + + +if VERSION >= (4, 0, 1): + from billiard.einfo import ExceptionWithTraceback +else: + ExceptionWithTraceback = None logger = logging.getLogger(__name__) @@ -104,6 +114,11 @@ def keys(self, carrier): class CeleryInstrumentor(BaseInstrumentor): + def __init__(self): + super().__init__() + self.metrics = None + self.task_id_to_start_time = {} + def instrumentation_dependencies(self) -> Collection[str]: return _instruments @@ -113,6 +128,11 @@ def _instrument(self, **kwargs): # pylint: disable=attribute-defined-outside-init self._tracer = trace.get_tracer(__name__, __version__, tracer_provider) + meter_provider = kwargs.get("meter_provider") + meter = get_meter(__name__, __version__, meter_provider) + + self.create_celery_metrics(meter) + signals.task_prerun.connect(self._trace_prerun, weak=False) signals.task_postrun.connect(self._trace_postrun, weak=False) signals.before_task_publish.connect( @@ -139,6 +159,7 @@ def _trace_prerun(self, *args, **kwargs): if task is None or task_id is None: return + self.update_task_duration_time(task_id) request = task.request tracectx = extract(request, getter=celery_getter) or None @@ -153,8 +174,7 @@ def _trace_prerun(self, *args, **kwargs): activation.__enter__() # pylint: disable=E1101 utils.attach_span(task, task_id, (span, activation)) - @staticmethod - def _trace_postrun(*args, **kwargs): + def _trace_postrun(self, *args, **kwargs): task = utils.retrieve_task(kwargs) task_id = utils.retrieve_task_id(kwargs) @@ -178,6 +198,9 @@ def _trace_postrun(*args, **kwargs): activation.__exit__(None, None, None) utils.detach_span(task, task_id) + self.update_task_duration_time(task_id) + labels = {"task": task.name, "worker": task.request.hostname} + self._record_histograms(task_id, labels) def _trace_before_publish(self, *args, **kwargs): task = utils.retrieve_task_from_sender(kwargs) @@ -256,6 +279,18 @@ def _trace_failure(*args, **kwargs): return if ex is not None: + # Unwrap the actual exception wrapped by billiard's + # `ExceptionInfo` and `ExceptionWithTraceback`. + if isinstance(ex, ExceptionInfo) and ex.exception is not None: + ex = ex.exception + + if ( + ExceptionWithTraceback is not None + and isinstance(ex, ExceptionWithTraceback) + and ex.exc is not None + ): + ex = ex.exc + status_kwargs["description"] = str(ex) span.record_exception(ex) span.set_status(Status(**status_kwargs)) @@ -277,3 +312,30 @@ def _trace_retry(*args, **kwargs): # Use `str(reason)` instead of `reason.message` in case we get # something that isn't an `Exception` span.set_attribute(_TASK_RETRY_REASON_KEY, str(reason)) + + def update_task_duration_time(self, task_id): + cur_time = default_timer() + task_duration_time_until_now = ( + cur_time - self.task_id_to_start_time[task_id] + if task_id in self.task_id_to_start_time + else cur_time + ) + self.task_id_to_start_time[task_id] = task_duration_time_until_now + + def _record_histograms(self, task_id, metric_attributes): + if task_id is None: + return + + self.metrics["flower.task.runtime.seconds"].record( + self.task_id_to_start_time.get(task_id), + attributes=metric_attributes, + ) + + def create_celery_metrics(self, meter) -> None: + self.metrics = { + "flower.task.runtime.seconds": meter.create_histogram( + name="flower.task.runtime.seconds", + unit="seconds", + description="The time it took to run the task.", + ) + } diff --git a/instrumentation/opentelemetry-instrumentation-celery/src/opentelemetry/instrumentation/celery/utils.py b/instrumentation/opentelemetry-instrumentation-celery/src/opentelemetry/instrumentation/celery/utils.py index 6f4f9cbc3a..f92c5e03c8 100644 --- a/instrumentation/opentelemetry-instrumentation-celery/src/opentelemetry/instrumentation/celery/utils.py +++ b/instrumentation/opentelemetry-instrumentation-celery/src/opentelemetry/instrumentation/celery/utils.py @@ -15,6 +15,7 @@ import logging from celery import registry # pylint: disable=no-name-in-module +from billiard import VERSION from opentelemetry.semconv.trace import SpanAttributes diff --git a/instrumentation/opentelemetry-instrumentation-celery/src/opentelemetry/instrumentation/celery/version.py b/instrumentation/opentelemetry-instrumentation-celery/src/opentelemetry/instrumentation/celery/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-celery/src/opentelemetry/instrumentation/celery/version.py +++ b/instrumentation/opentelemetry-instrumentation-celery/src/opentelemetry/instrumentation/celery/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-celery/tests/celery_test_tasks.py b/instrumentation/opentelemetry-instrumentation-celery/tests/celery_test_tasks.py index d9660412f0..9ac78f6d8b 100644 --- a/instrumentation/opentelemetry-instrumentation-celery/tests/celery_test_tasks.py +++ b/instrumentation/opentelemetry-instrumentation-celery/tests/celery_test_tasks.py @@ -24,6 +24,15 @@ class Config: app.config_from_object(Config) +class CustomError(Exception): + pass + + @app.task def task_add(num_a, num_b): return num_a + num_b + + +@app.task +def task_raises(): + raise CustomError("The task failed!") diff --git a/instrumentation/opentelemetry-instrumentation-celery/tests/test_metrics.py b/instrumentation/opentelemetry-instrumentation-celery/tests/test_metrics.py new file mode 100644 index 0000000000..46e39a6046 --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-celery/tests/test_metrics.py @@ -0,0 +1,76 @@ +import threading +import time +from timeit import default_timer + +from opentelemetry.instrumentation.celery import CeleryInstrumentor +from opentelemetry.test.test_base import TestBase + +from .celery_test_tasks import app, task_add + + +class TestMetrics(TestBase): + def setUp(self): + super().setUp() + self._worker = app.Worker( + app=app, pool="solo", concurrency=1, hostname="celery@akochavi" + ) + self._thread = threading.Thread(target=self._worker.start) + self._thread.daemon = True + self._thread.start() + + def tearDown(self): + super().tearDown() + self._worker.stop() + self._thread.join() + + def get_metrics(self): + result = task_add.delay(1, 2) + + timeout = time.time() + 60 * 1 # 1 minutes from now + while not result.ready(): + if time.time() > timeout: + break + time.sleep(0.05) + return self.get_sorted_metrics() + + def test_basic_metric(self): + CeleryInstrumentor().instrument() + start_time = default_timer() + task_runtime_estimated = (default_timer() - start_time) * 1000 + + metrics = self.get_metrics() + CeleryInstrumentor().uninstrument() + self.assertEqual(len(metrics), 1) + + task_runtime = metrics[0] + print(task_runtime) + self.assertEqual(task_runtime.name, "flower.task.runtime.seconds") + self.assert_metric_expected( + task_runtime, + [ + self.create_histogram_data_point( + count=1, + sum_data_point=task_runtime_estimated, + max_data_point=task_runtime_estimated, + min_data_point=task_runtime_estimated, + attributes={ + "task": "tests.celery_test_tasks.task_add", + "worker": "celery@akochavi", + }, + ) + ], + est_value_delta=200, + ) + + def test_metric_uninstrument(self): + CeleryInstrumentor().instrument() + metrics = self.get_metrics() + self.assertEqual(len(metrics), 1) + CeleryInstrumentor().uninstrument() + + metrics = self.get_metrics() + self.assertEqual(len(metrics), 1) + + for metric in metrics: + for point in list(metric.data.data_points): + self.assertEqual(point.count, 1) diff --git a/instrumentation/opentelemetry-instrumentation-celery/tests/test_tasks.py b/instrumentation/opentelemetry-instrumentation-celery/tests/test_tasks.py index 47f79d7e1c..ed4dbb5b1d 100644 --- a/instrumentation/opentelemetry-instrumentation-celery/tests/test_tasks.py +++ b/instrumentation/opentelemetry-instrumentation-celery/tests/test_tasks.py @@ -18,9 +18,9 @@ from opentelemetry.instrumentation.celery import CeleryInstrumentor from opentelemetry.semconv.trace import SpanAttributes from opentelemetry.test.test_base import TestBase -from opentelemetry.trace import SpanKind +from opentelemetry.trace import SpanKind, StatusCode -from .celery_test_tasks import app, task_add +from .celery_test_tasks import app, task_add, task_raises class TestCeleryInstrumentation(TestBase): @@ -66,6 +66,10 @@ def test_task(self): }, ) + self.assertEqual(consumer.status.status_code, StatusCode.UNSET) + + self.assertEqual(0, len(consumer.events)) + self.assertEqual( producer.name, "apply_async/tests.celery_test_tasks.task_add" ) @@ -84,6 +88,70 @@ def test_task(self): self.assertEqual(consumer.parent.span_id, producer.context.span_id) self.assertEqual(consumer.context.trace_id, producer.context.trace_id) + def test_task_raises(self): + CeleryInstrumentor().instrument() + + result = task_raises.delay() + + timeout = time.time() + 60 * 1 # 1 minutes from now + while not result.ready(): + if time.time() > timeout: + break + time.sleep(0.05) + + spans = self.sorted_spans(self.memory_exporter.get_finished_spans()) + self.assertEqual(len(spans), 2) + + consumer, producer = spans + + self.assertEqual( + consumer.name, "run/tests.celery_test_tasks.task_raises" + ) + self.assertEqual(consumer.kind, SpanKind.CONSUMER) + self.assertSpanHasAttributes( + consumer, + { + "celery.action": "run", + "celery.state": "FAILURE", + SpanAttributes.MESSAGING_DESTINATION: "celery", + "celery.task_name": "tests.celery_test_tasks.task_raises", + }, + ) + + self.assertEqual(consumer.status.status_code, StatusCode.ERROR) + + self.assertEqual(1, len(consumer.events)) + event = consumer.events[0] + + self.assertIn(SpanAttributes.EXCEPTION_STACKTRACE, event.attributes) + + self.assertEqual( + event.attributes[SpanAttributes.EXCEPTION_TYPE], "CustomError" + ) + + self.assertEqual( + event.attributes[SpanAttributes.EXCEPTION_MESSAGE], + "The task failed!", + ) + + self.assertEqual( + producer.name, "apply_async/tests.celery_test_tasks.task_raises" + ) + self.assertEqual(producer.kind, SpanKind.PRODUCER) + self.assertSpanHasAttributes( + producer, + { + "celery.action": "apply_async", + "celery.task_name": "tests.celery_test_tasks.task_raises", + SpanAttributes.MESSAGING_DESTINATION_KIND: "queue", + SpanAttributes.MESSAGING_DESTINATION: "celery", + }, + ) + + self.assertNotEqual(consumer.parent, producer.context) + self.assertEqual(consumer.parent.span_id, producer.context.span_id) + self.assertEqual(consumer.context.trace_id, producer.context.trace_id) + def test_uninstrument(self): CeleryInstrumentor().instrument() CeleryInstrumentor().uninstrument() diff --git a/instrumentation/opentelemetry-instrumentation-confluent-kafka/pyproject.toml b/instrumentation/opentelemetry-instrumentation-confluent-kafka/pyproject.toml index a84f7d959e..2ef1fd6b49 100644 --- a/instrumentation/opentelemetry-instrumentation-confluent-kafka/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-confluent-kafka/pyproject.toml @@ -31,7 +31,7 @@ dependencies = [ [project.optional-dependencies] instruments = [ - "confluent-kafka >= 1.8.2, < 2.0.0", + "confluent-kafka >= 1.8.2, <= 2.2.0", ] test = [ "opentelemetry-instrumentation-confluent-kafka[instruments]", diff --git a/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/__init__.py b/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/__init__.py index 12cb363219..c4e68b33b4 100644 --- a/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/__init__.py @@ -112,6 +112,8 @@ def instrument_consumer(consumer: Consumer, tracer_provider=None) from .package import _instruments from .utils import ( KafkaPropertiesExtractor, + _end_current_consume_span, + _create_new_consume_span, _enrich_span, _get_span_name, _kafka_getter, @@ -137,6 +139,12 @@ def __init__(self, config): def poll(self, timeout=-1): # pylint: disable=useless-super-delegation return super().poll(timeout) + # This method is deliberately implemented in order to allow wrapt to wrap this function + def consume( + self, *args, **kwargs + ): # pylint: disable=useless-super-delegation + return super().consume(*args, **kwargs) + class ProxiedProducer(Producer): def __init__(self, producer: Producer, tracer: Tracer): @@ -177,10 +185,14 @@ def committed(self, partitions, timeout=-1): def commit(self, *args, **kwargs): return self._consumer.commit(*args, **kwargs) - def consume( - self, num_messages=1, *args, **kwargs - ): # pylint: disable=keyword-arg-before-vararg - return self._consumer.consume(num_messages, *args, **kwargs) + def consume(self, *args, **kwargs): + return ConfluentKafkaInstrumentor.wrap_consume( + self._consumer.consume, + self, + self._tracer, + args, + kwargs, + ) def get_watermark_offsets( self, partition, timeout=-1, *args, **kwargs @@ -275,6 +287,11 @@ def _inner_wrap_poll(func, instance, args, kwargs): func, instance, self._tracer, args, kwargs ) + def _inner_wrap_consume(func, instance, args, kwargs): + return ConfluentKafkaInstrumentor.wrap_consume( + func, instance, self._tracer, args, kwargs + ) + wrapt.wrap_function_wrapper( AutoInstrumentedProducer, "produce", @@ -287,6 +304,12 @@ def _inner_wrap_poll(func, instance, args, kwargs): _inner_wrap_poll, ) + wrapt.wrap_function_wrapper( + AutoInstrumentedConsumer, + "consume", + _inner_wrap_consume, + ) + def _uninstrument(self, **kwargs): confluent_kafka.Producer = self._original_kafka_producer confluent_kafka.Consumer = self._original_kafka_consumer @@ -326,29 +349,14 @@ def wrap_produce(func, instance, tracer, args, kwargs): @staticmethod def wrap_poll(func, instance, tracer, args, kwargs): if instance._current_consume_span: - context.detach(instance._current_context_token) - instance._current_context_token = None - instance._current_consume_span.end() - instance._current_consume_span = None + _end_current_consume_span(instance) with tracer.start_as_current_span( "recv", end_on_exit=True, kind=trace.SpanKind.CONSUMER ): record = func(*args, **kwargs) if record: - links = [] - ctx = propagate.extract(record.headers(), getter=_kafka_getter) - if ctx: - for item in ctx.values(): - if hasattr(item, "get_span_context"): - links.append(Link(context=item.get_span_context())) - - instance._current_consume_span = tracer.start_span( - name=f"{record.topic()} process", - links=links, - kind=SpanKind.CONSUMER, - ) - + _create_new_consume_span(instance, tracer, [record]) _enrich_span( instance._current_consume_span, record.topic(), @@ -361,3 +369,26 @@ def wrap_poll(func, instance, tracer, args, kwargs): ) return record + + @staticmethod + def wrap_consume(func, instance, tracer, args, kwargs): + if instance._current_consume_span: + _end_current_consume_span(instance) + + with tracer.start_as_current_span( + "recv", end_on_exit=True, kind=trace.SpanKind.CONSUMER + ): + records = func(*args, **kwargs) + if len(records) > 0: + _create_new_consume_span(instance, tracer, records) + _enrich_span( + instance._current_consume_span, + records[0].topic(), + operation=MessagingOperationValues.PROCESS, + ) + + instance._current_context_token = context.attach( + trace.set_span_in_context(instance._current_consume_span) + ) + + return records diff --git a/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/package.py b/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/package.py index eab664d9ee..21d37ebfae 100644 --- a/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/package.py +++ b/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/package.py @@ -13,4 +13,4 @@ # limitations under the License. -_instruments = ("confluent-kafka >= 1.8.2, < 2.0.0",) +_instruments = ("confluent-kafka >= 1.8.2, <= 2.2.0",) diff --git a/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/utils.py b/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/utils.py index 77fce03cd8..2029960703 100644 --- a/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/utils.py +++ b/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/utils.py @@ -1,6 +1,8 @@ from logging import getLogger from typing import List, Optional +from opentelemetry import context, propagate +from opentelemetry.trace import SpanKind, Link from opentelemetry.propagators import textmap from opentelemetry.semconv.trace import ( MessagingDestinationKindValues, @@ -81,6 +83,34 @@ def set(self, carrier: textmap.CarrierT, key: str, value: str) -> None: _kafka_getter = KafkaContextGetter() +def _end_current_consume_span(instance): + context.detach(instance._current_context_token) + instance._current_context_token = None + instance._current_consume_span.end() + instance._current_consume_span = None + + +def _create_new_consume_span(instance, tracer, records): + links = _get_links_from_records(records) + instance._current_consume_span = tracer.start_span( + name=f"{records[0].topic()} process", + links=links, + kind=SpanKind.CONSUMER, + ) + + +def _get_links_from_records(records): + links = [] + for record in records: + ctx = propagate.extract(record.headers(), getter=_kafka_getter) + if ctx: + for item in ctx.values(): + if hasattr(item, "get_span_context"): + links.append(Link(context=item.get_span_context())) + + return links + + def _enrich_span( span, topic, @@ -94,7 +124,7 @@ def _enrich_span( span.set_attribute(SpanAttributes.MESSAGING_SYSTEM, "kafka") span.set_attribute(SpanAttributes.MESSAGING_DESTINATION, topic) - if partition: + if partition is not None: span.set_attribute(SpanAttributes.MESSAGING_KAFKA_PARTITION, partition) span.set_attribute( @@ -109,7 +139,7 @@ def _enrich_span( # https://stackoverflow.com/questions/65935155/identify-and-find-specific-message-in-kafka-topic # A message within Kafka is uniquely defined by its topic name, topic partition and offset. - if partition and offset and topic: + if partition is not None and offset is not None and topic: span.set_attribute( SpanAttributes.MESSAGING_MESSAGE_ID, f"{topic}.{partition}.{offset}", diff --git a/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/version.py b/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/version.py +++ b/instrumentation/opentelemetry-instrumentation-confluent-kafka/src/opentelemetry/instrumentation/confluent_kafka/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-confluent-kafka/tests/test_instrumentation.py b/instrumentation/opentelemetry-instrumentation-confluent-kafka/tests/test_instrumentation.py index 1e3f304188..d7ac343dbf 100644 --- a/instrumentation/opentelemetry-instrumentation-confluent-kafka/tests/test_instrumentation.py +++ b/instrumentation/opentelemetry-instrumentation-confluent-kafka/tests/test_instrumentation.py @@ -14,7 +14,12 @@ # pylint: disable=no-name-in-module -from unittest import TestCase +from opentelemetry.semconv.trace import ( + SpanAttributes, + MessagingDestinationKindValues, +) +from opentelemetry.test.test_base import TestBase +from .utils import MockConsumer, MockedMessage from confluent_kafka import Consumer, Producer @@ -29,7 +34,7 @@ ) -class TestConfluentKafka(TestCase): +class TestConfluentKafka(TestBase): def test_instrument_api(self) -> None: instrumentation = ConfluentKafkaInstrumentor() @@ -104,3 +109,140 @@ def test_context_getter(self) -> None: context_setter.set(carrier_list, "key1", "val1") self.assertEqual(context_getter.get(carrier_list, "key1"), ["val1"]) self.assertEqual(["key1"], context_getter.keys(carrier_list)) + + def test_poll(self) -> None: + instrumentation = ConfluentKafkaInstrumentor() + mocked_messages = [ + MockedMessage("topic-10", 0, 0, []), + MockedMessage("topic-20", 2, 4, []), + MockedMessage("topic-30", 1, 3, []), + ] + expected_spans = [ + {"name": "recv", "attributes": {}}, + { + "name": "topic-10 process", + "attributes": { + SpanAttributes.MESSAGING_OPERATION: "process", + SpanAttributes.MESSAGING_KAFKA_PARTITION: 0, + SpanAttributes.MESSAGING_SYSTEM: "kafka", + SpanAttributes.MESSAGING_DESTINATION: "topic-10", + SpanAttributes.MESSAGING_DESTINATION_KIND: MessagingDestinationKindValues.QUEUE.value, + SpanAttributes.MESSAGING_MESSAGE_ID: "topic-10.0.0", + }, + }, + {"name": "recv", "attributes": {}}, + { + "name": "topic-20 process", + "attributes": { + SpanAttributes.MESSAGING_OPERATION: "process", + SpanAttributes.MESSAGING_KAFKA_PARTITION: 2, + SpanAttributes.MESSAGING_SYSTEM: "kafka", + SpanAttributes.MESSAGING_DESTINATION: "topic-20", + SpanAttributes.MESSAGING_DESTINATION_KIND: MessagingDestinationKindValues.QUEUE.value, + SpanAttributes.MESSAGING_MESSAGE_ID: "topic-20.2.4", + }, + }, + {"name": "recv", "attributes": {}}, + { + "name": "topic-30 process", + "attributes": { + SpanAttributes.MESSAGING_OPERATION: "process", + SpanAttributes.MESSAGING_KAFKA_PARTITION: 1, + SpanAttributes.MESSAGING_SYSTEM: "kafka", + SpanAttributes.MESSAGING_DESTINATION: "topic-30", + SpanAttributes.MESSAGING_DESTINATION_KIND: MessagingDestinationKindValues.QUEUE.value, + SpanAttributes.MESSAGING_MESSAGE_ID: "topic-30.1.3", + }, + }, + {"name": "recv", "attributes": {}}, + ] + + consumer = MockConsumer( + mocked_messages, + { + "bootstrap.servers": "localhost:29092", + "group.id": "mygroup", + "auto.offset.reset": "earliest", + }, + ) + self.memory_exporter.clear() + consumer = instrumentation.instrument_consumer(consumer) + consumer.poll() + consumer.poll() + consumer.poll() + consumer.poll() + + span_list = self.memory_exporter.get_finished_spans() + self._compare_spans(span_list, expected_spans) + + def test_consume(self) -> None: + instrumentation = ConfluentKafkaInstrumentor() + mocked_messages = [ + MockedMessage("topic-1", 0, 0, []), + MockedMessage("topic-1", 2, 1, []), + MockedMessage("topic-1", 3, 2, []), + MockedMessage("topic-2", 0, 0, []), + MockedMessage("topic-3", 0, 3, []), + MockedMessage("topic-2", 0, 1, []), + ] + expected_spans = [ + {"name": "recv", "attributes": {}}, + { + "name": "topic-1 process", + "attributes": { + SpanAttributes.MESSAGING_OPERATION: "process", + SpanAttributes.MESSAGING_SYSTEM: "kafka", + SpanAttributes.MESSAGING_DESTINATION: "topic-1", + SpanAttributes.MESSAGING_DESTINATION_KIND: MessagingDestinationKindValues.QUEUE.value, + }, + }, + {"name": "recv", "attributes": {}}, + { + "name": "topic-2 process", + "attributes": { + SpanAttributes.MESSAGING_OPERATION: "process", + SpanAttributes.MESSAGING_SYSTEM: "kafka", + SpanAttributes.MESSAGING_DESTINATION: "topic-2", + SpanAttributes.MESSAGING_DESTINATION_KIND: MessagingDestinationKindValues.QUEUE.value, + }, + }, + {"name": "recv", "attributes": {}}, + { + "name": "topic-3 process", + "attributes": { + SpanAttributes.MESSAGING_OPERATION: "process", + SpanAttributes.MESSAGING_SYSTEM: "kafka", + SpanAttributes.MESSAGING_DESTINATION: "topic-3", + SpanAttributes.MESSAGING_DESTINATION_KIND: MessagingDestinationKindValues.QUEUE.value, + }, + }, + {"name": "recv", "attributes": {}}, + ] + + consumer = MockConsumer( + mocked_messages, + { + "bootstrap.servers": "localhost:29092", + "group.id": "mygroup", + "auto.offset.reset": "earliest", + }, + ) + + self.memory_exporter.clear() + consumer = instrumentation.instrument_consumer(consumer) + consumer.consume(3) + consumer.consume(1) + consumer.consume(2) + consumer.consume(1) + span_list = self.memory_exporter.get_finished_spans() + self._compare_spans(span_list, expected_spans) + + def _compare_spans(self, spans, expected_spans): + for span, expected_span in zip(spans, expected_spans): + self.assertEqual(expected_span["name"], span.name) + for attribute_key, expected_attribute_value in expected_span[ + "attributes" + ].items(): + self.assertEqual( + expected_attribute_value, span.attributes[attribute_key] + ) diff --git a/instrumentation/opentelemetry-instrumentation-confluent-kafka/tests/utils.py b/instrumentation/opentelemetry-instrumentation-confluent-kafka/tests/utils.py new file mode 100644 index 0000000000..798daaeff4 --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-confluent-kafka/tests/utils.py @@ -0,0 +1,39 @@ +from confluent_kafka import Consumer + + +class MockConsumer(Consumer): + def __init__(self, queue, config): + self._queue = queue + super().__init__(config) + + def consume( + self, num_messages=1, *args, **kwargs + ): # pylint: disable=keyword-arg-before-vararg + messages = self._queue[:num_messages] + self._queue = self._queue[num_messages:] + return messages + + def poll(self, timeout=None): + if len(self._queue) > 0: + return self._queue.pop(0) + return None + + +class MockedMessage: + def __init__(self, topic: str, partition: int, offset: int, headers): + self._topic = topic + self._partition = partition + self._offset = offset + self._headers = headers + + def topic(self): + return self._topic + + def partition(self): + return self._partition + + def offset(self): + return self._offset + + def headers(self): + return self._headers diff --git a/instrumentation/opentelemetry-instrumentation-dbapi/pyproject.toml b/instrumentation/opentelemetry-instrumentation-dbapi/pyproject.toml index e87b80a728..5f9df3983e 100644 --- a/instrumentation/opentelemetry-instrumentation-dbapi/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-dbapi/pyproject.toml @@ -26,15 +26,15 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", "wrapt >= 1.0.0, < 2.0.0", ] [project.optional-dependencies] instruments = [] test = [ - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.urls] diff --git a/instrumentation/opentelemetry-instrumentation-dbapi/src/opentelemetry/instrumentation/dbapi/version.py b/instrumentation/opentelemetry-instrumentation-dbapi/src/opentelemetry/instrumentation/dbapi/version.py index 16c754f1d3..084725a38e 100644 --- a/instrumentation/opentelemetry-instrumentation-dbapi/src/opentelemetry/instrumentation/dbapi/version.py +++ b/instrumentation/opentelemetry-instrumentation-dbapi/src/opentelemetry/instrumentation/dbapi/version.py @@ -12,6 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" _instruments = tuple() diff --git a/instrumentation/opentelemetry-instrumentation-django/pyproject.toml b/instrumentation/opentelemetry-instrumentation-django/pyproject.toml index 5f900599ca..e235d6e6a9 100644 --- a/instrumentation/opentelemetry-instrumentation-django/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-django/pyproject.toml @@ -26,22 +26,22 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-instrumentation-wsgi == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", - "opentelemetry-util-http == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-instrumentation-wsgi == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", + "opentelemetry-util-http == 0.42b0.dev", ] [project.optional-dependencies] asgi = [ - "opentelemetry-instrumentation-asgi == 0.39b0.dev", + "opentelemetry-instrumentation-asgi == 0.42b0.dev", ] instruments = [ "django >= 1.10", ] test = [ "opentelemetry-instrumentation-django[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/middleware/otel_middleware.py b/instrumentation/opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/middleware/otel_middleware.py index 1baa05eca9..491e78cab5 100644 --- a/instrumentation/opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/middleware/otel_middleware.py +++ b/instrumentation/opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/middleware/otel_middleware.py @@ -172,19 +172,16 @@ def _get_span_name(request): else: match = resolve(request.path) - if hasattr(match, "route"): - return match.route + if hasattr(match, "route") and match.route: + return f"{request.method} {match.route}" - # Instead of using `view_name`, better to use `_func_name` as some applications can use similar - # view names in different modules - if hasattr(match, "_func_name"): - return match._func_name # pylint: disable=protected-access + if hasattr(match, "url_name") and match.url_name: + return f"{request.method} {match.url_name}" - # Fallback for safety as `_func_name` private field - return match.view_name + return request.method except Resolver404: - return f"HTTP {request.method}" + return request.method # pylint: disable=too-many-locals def process_request(self, request): diff --git a/instrumentation/opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/version.py b/instrumentation/opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/version.py +++ b/instrumentation/opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-django/tests/test_middleware.py b/instrumentation/opentelemetry-instrumentation-django/tests/test_middleware.py index e235c8840e..d7bb1e544f 100644 --- a/instrumentation/opentelemetry-instrumentation-django/tests/test_middleware.py +++ b/instrumentation/opentelemetry-instrumentation-django/tests/test_middleware.py @@ -74,10 +74,14 @@ DJANGO_3_0 = VERSION >= (3, 0) if DJANGO_2_0: - from django.urls import re_path + from django.urls import path, re_path else: from django.conf.urls import url as re_path + def path(path_argument, *args, **kwargs): + return re_path(rf"^{path_argument}$", *args, **kwargs) + + urlpatterns = [ re_path(r"^traced/", traced), re_path(r"^traced_custom_header/", response_with_custom_header), @@ -87,6 +91,7 @@ re_path(r"^excluded_noarg/", excluded_noarg), re_path(r"^excluded_noarg2/", excluded_noarg2), re_path(r"^span_name/([0-9]{4})/$", route_span_name), + path("", traced, name="empty"), ] _django_instrumentor = DjangoInstrumentor() @@ -150,9 +155,9 @@ def test_templated_route_get(self): self.assertEqual( span.name, - "^route/(?P[0-9]{4})/template/$" + "GET ^route/(?P[0-9]{4})/template/$" if DJANGO_2_2 - else "tests.views.traced_template", + else "GET", ) self.assertEqual(span.kind, SpanKind.SERVER) self.assertEqual(span.status.status_code, StatusCode.UNSET) @@ -177,9 +182,7 @@ def test_traced_get(self): span = spans[0] - self.assertEqual( - span.name, "^traced/" if DJANGO_2_2 else "tests.views.traced" - ) + self.assertEqual(span.name, "GET ^traced/" if DJANGO_2_2 else "GET") self.assertEqual(span.kind, SpanKind.SERVER) self.assertEqual(span.status.status_code, StatusCode.UNSET) self.assertEqual(span.attributes[SpanAttributes.HTTP_METHOD], "GET") @@ -207,6 +210,16 @@ def test_not_recording(self): self.assertFalse(mock_span.set_attribute.called) self.assertFalse(mock_span.set_status.called) + def test_empty_path(self): + Client().get("/") + + spans = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans), 1) + + span = spans[0] + + self.assertEqual(span.name, "GET empty") + def test_traced_post(self): Client().post("/traced/") @@ -215,9 +228,7 @@ def test_traced_post(self): span = spans[0] - self.assertEqual( - span.name, "^traced/" if DJANGO_2_2 else "tests.views.traced" - ) + self.assertEqual(span.name, "POST ^traced/" if DJANGO_2_2 else "POST") self.assertEqual(span.kind, SpanKind.SERVER) self.assertEqual(span.status.status_code, StatusCode.UNSET) self.assertEqual(span.attributes[SpanAttributes.HTTP_METHOD], "POST") @@ -241,9 +252,7 @@ def test_error(self): span = spans[0] - self.assertEqual( - span.name, "^error/" if DJANGO_2_2 else "tests.views.error" - ) + self.assertEqual(span.name, "GET ^error/" if DJANGO_2_2 else "GET") self.assertEqual(span.kind, SpanKind.SERVER) self.assertEqual(span.status.status_code, StatusCode.ERROR) self.assertEqual(span.attributes[SpanAttributes.HTTP_METHOD], "GET") @@ -307,9 +316,7 @@ def test_span_name(self): span = span_list[0] self.assertEqual( span.name, - "^span_name/([0-9]{4})/$" - if DJANGO_2_2 - else "tests.views.route_span_name", + "GET ^span_name/([0-9]{4})/$" if DJANGO_2_2 else "GET", ) def test_span_name_for_query_string(self): @@ -323,9 +330,7 @@ def test_span_name_for_query_string(self): span = span_list[0] self.assertEqual( span.name, - "^span_name/([0-9]{4})/$" - if DJANGO_2_2 - else "tests.views.route_span_name", + "GET ^span_name/([0-9]{4})/$" if DJANGO_2_2 else "GET", ) def test_span_name_404(self): @@ -334,7 +339,7 @@ def test_span_name_404(self): self.assertEqual(len(span_list), 1) span = span_list[0] - self.assertEqual(span.name, "HTTP GET") + self.assertEqual(span.name, "GET") def test_traced_request_attrs(self): Client().get("/span_name/1234/", CONTENT_TYPE="test/ct") diff --git a/instrumentation/opentelemetry-instrumentation-django/tests/test_middleware_asgi.py b/instrumentation/opentelemetry-instrumentation-django/tests/test_middleware_asgi.py index a78501bcb8..0e2472d15e 100644 --- a/instrumentation/opentelemetry-instrumentation-django/tests/test_middleware_asgi.py +++ b/instrumentation/opentelemetry-instrumentation-django/tests/test_middleware_asgi.py @@ -137,7 +137,7 @@ async def test_templated_route_get(self): span = spans[0] - self.assertEqual(span.name, "^route/(?P[0-9]{4})/template/$") + self.assertEqual(span.name, "GET ^route/(?P[0-9]{4})/template/$") self.assertEqual(span.kind, SpanKind.SERVER) self.assertEqual(span.status.status_code, StatusCode.UNSET) self.assertEqual(span.attributes[SpanAttributes.HTTP_METHOD], "GET") @@ -160,7 +160,7 @@ async def test_traced_get(self): span = spans[0] - self.assertEqual(span.name, "^traced/") + self.assertEqual(span.name, "GET ^traced/") self.assertEqual(span.kind, SpanKind.SERVER) self.assertEqual(span.status.status_code, StatusCode.UNSET) self.assertEqual(span.attributes[SpanAttributes.HTTP_METHOD], "GET") @@ -195,7 +195,7 @@ async def test_traced_post(self): span = spans[0] - self.assertEqual(span.name, "^traced/") + self.assertEqual(span.name, "POST ^traced/") self.assertEqual(span.kind, SpanKind.SERVER) self.assertEqual(span.status.status_code, StatusCode.UNSET) self.assertEqual(span.attributes[SpanAttributes.HTTP_METHOD], "POST") @@ -218,7 +218,7 @@ async def test_error(self): span = spans[0] - self.assertEqual(span.name, "^error/") + self.assertEqual(span.name, "GET ^error/") self.assertEqual(span.kind, SpanKind.SERVER) self.assertEqual(span.status.status_code, StatusCode.ERROR) self.assertEqual(span.attributes[SpanAttributes.HTTP_METHOD], "GET") @@ -264,7 +264,7 @@ async def test_span_name(self): self.assertEqual(len(span_list), 1) span = span_list[0] - self.assertEqual(span.name, "^span_name/([0-9]{4})/$") + self.assertEqual(span.name, "GET ^span_name/([0-9]{4})/$") async def test_span_name_for_query_string(self): """ @@ -275,7 +275,7 @@ async def test_span_name_for_query_string(self): self.assertEqual(len(span_list), 1) span = span_list[0] - self.assertEqual(span.name, "^span_name/([0-9]{4})/$") + self.assertEqual(span.name, "GET ^span_name/([0-9]{4})/$") async def test_span_name_404(self): await self.async_client.get("/span_name/1234567890/") @@ -283,7 +283,7 @@ async def test_span_name_404(self): self.assertEqual(len(span_list), 1) span = span_list[0] - self.assertEqual(span.name, "HTTP GET") + self.assertEqual(span.name, "GET") async def test_traced_request_attrs(self): await self.async_client.get("/span_name/1234/", CONTENT_TYPE="test/ct") diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/pyproject.toml b/instrumentation/opentelemetry-instrumentation-elasticsearch/pyproject.toml index 739524f7be..bee1d44d2a 100644 --- a/instrumentation/opentelemetry-instrumentation-elasticsearch/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-elasticsearch/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", "wrapt >= 1.0.0, < 2.0.0", ] @@ -38,7 +38,7 @@ instruments = [ test = [ "opentelemetry-instrumentation-elasticsearch[instruments]", "elasticsearch-dsl >= 2.0", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/__init__.py b/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/__init__.py index d39c172c6a..480ccb6402 100644 --- a/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/__init__.py @@ -98,6 +98,12 @@ def response_hook(span, response): from .utils import sanitize_body +# Split of elasticsearch and elastic_transport in 8.0.0+ +# https://www.elastic.co/guide/en/elasticsearch/client/python-api/master/release-notes.html#rn-8-0-0 +es_transport_split = elasticsearch.VERSION[0] > 7 +if es_transport_split: + import elastic_transport + logger = getLogger(__name__) @@ -137,16 +143,28 @@ def _instrument(self, **kwargs): tracer = get_tracer(__name__, __version__, tracer_provider) request_hook = kwargs.get("request_hook") response_hook = kwargs.get("response_hook") - _wrap( - elasticsearch, - "Transport.perform_request", - _wrap_perform_request( - tracer, - self._span_name_prefix, - request_hook, - response_hook, - ), - ) + if es_transport_split: + _wrap( + elastic_transport, + "Transport.perform_request", + _wrap_perform_request( + tracer, + self._span_name_prefix, + request_hook, + response_hook, + ), + ) + else: + _wrap( + elasticsearch, + "Transport.perform_request", + _wrap_perform_request( + tracer, + self._span_name_prefix, + request_hook, + response_hook, + ), + ) def _uninstrument(self, **kwargs): unwrap(elasticsearch.Transport, "perform_request") diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/utils.py b/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/utils.py index ca4a466bab..97f2bc3b87 100644 --- a/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/utils.py +++ b/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/utils.py @@ -11,6 +11,7 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. +import json sanitized_keys = ( "message", @@ -51,6 +52,9 @@ def _unflatten_dict(d): def sanitize_body(body) -> str: + if isinstance(body, str): + body = json.loads(body) + flatten_body = _flatten_dict(body) for key in flatten_body: diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/version.py b/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/version.py +++ b/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/helpers_es8.py b/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/helpers_es8.py new file mode 100644 index 0000000000..04ed2efda2 --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/helpers_es8.py @@ -0,0 +1,41 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from elasticsearch_dsl import Document, Keyword, Text + + +class Article(Document): + title = Text(analyzer="snowball", fields={"raw": Keyword()}) + body = Text(analyzer="snowball") + + class Index: + name = "test-index" + + +dsl_create_statement = { + "mappings": { + "properties": { + "title": { + "analyzer": "snowball", + "fields": {"raw": {"type": "keyword"}}, + "type": "text", + }, + "body": {"analyzer": "snowball", "type": "text"}, + } + } +} +dsl_index_result = (1, {}, '{"result": "created"}') +dsl_index_span_name = "Elasticsearch/test-index/_doc/2" +dsl_index_url = "/test-index/_doc/2" +dsl_search_method = "POST" diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py b/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py index 02bf3eb591..37dd5f9cd7 100644 --- a/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py +++ b/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py @@ -37,7 +37,9 @@ major_version = elasticsearch.VERSION[0] -if major_version == 7: +if major_version == 8: + from . import helpers_es8 as helpers # pylint: disable=no-name-in-module +elif major_version == 7: from . import helpers_es7 as helpers # pylint: disable=no-name-in-module elif major_version == 6: from . import helpers_es6 as helpers # pylint: disable=no-name-in-module @@ -479,3 +481,7 @@ def test_body_sanitization(self, _): sanitize_body(sanitization_queries.filter_query), str(sanitization_queries.filter_query_sanitized), ) + self.assertEqual( + sanitize_body(json.dumps(sanitization_queries.interval_query)), + str(sanitization_queries.interval_query_sanitized), + ) diff --git a/instrumentation/opentelemetry-instrumentation-falcon/pyproject.toml b/instrumentation/opentelemetry-instrumentation-falcon/pyproject.toml index 60c495bd15..b6c482ed1f 100644 --- a/instrumentation/opentelemetry-instrumentation-falcon/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-falcon/pyproject.toml @@ -26,10 +26,10 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-instrumentation-wsgi == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", - "opentelemetry-util-http == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-instrumentation-wsgi == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", + "opentelemetry-util-http == 0.42b0.dev", "packaging >= 20.0", ] @@ -39,7 +39,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-falcon[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", "parameterized == 0.7.4", ] diff --git a/instrumentation/opentelemetry-instrumentation-falcon/src/opentelemetry/instrumentation/falcon/__init__.py b/instrumentation/opentelemetry-instrumentation-falcon/src/opentelemetry/instrumentation/falcon/__init__.py index 3a6a86e4fb..669f41b0ab 100644 --- a/instrumentation/opentelemetry-instrumentation-falcon/src/opentelemetry/instrumentation/falcon/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-falcon/src/opentelemetry/instrumentation/falcon/__init__.py @@ -208,7 +208,7 @@ def response_hook(span, req, resp): from opentelemetry.metrics import get_meter from opentelemetry.semconv.metrics import MetricInstruments from opentelemetry.semconv.trace import SpanAttributes -from opentelemetry.trace.status import Status +from opentelemetry.trace.status import Status, StatusCode from opentelemetry.util.http import get_excluded_urls, get_traced_request_attrs _logger = getLogger(__name__) @@ -428,7 +428,6 @@ def process_resource(self, req, resp, resource, params): resource_name = resource.__class__.__name__ span.set_attribute("falcon.resource", resource_name) - span.update_name(f"{resource_name}.on_{req.method.lower()}") def process_response( self, req, resp, resource, req_succeeded=None @@ -461,11 +460,17 @@ def process_response( try: status_code = int(status) span.set_attribute(SpanAttributes.HTTP_STATUS_CODE, status_code) + otel_status_code = http_status_to_status_code( + status_code, server_span=True + ) + + # set the description only when the status code is ERROR + if otel_status_code is not StatusCode.ERROR: + reason = None + span.set_status( Status( - status_code=http_status_to_status_code( - status_code, server_span=True - ), + status_code=otel_status_code, description=reason, ) ) @@ -477,6 +482,12 @@ def process_response( response_headers = resp.headers if span.is_recording() and span.kind == trace.SpanKind.SERVER: + # Check if low-cardinality route is available as per semantic-conventions + if req.uri_template: + span.update_name(f"{req.method} {req.uri_template}") + else: + span.update_name(f"{req.method}") + custom_attributes = ( otel_wsgi.collect_custom_response_headers_attributes( response_headers.items() diff --git a/instrumentation/opentelemetry-instrumentation-falcon/src/opentelemetry/instrumentation/falcon/version.py b/instrumentation/opentelemetry-instrumentation-falcon/src/opentelemetry/instrumentation/falcon/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-falcon/src/opentelemetry/instrumentation/falcon/version.py +++ b/instrumentation/opentelemetry-instrumentation-falcon/src/opentelemetry/instrumentation/falcon/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-falcon/tests/app.py b/instrumentation/opentelemetry-instrumentation-falcon/tests/app.py index 3e4c62ec3e..a4d279149d 100644 --- a/instrumentation/opentelemetry-instrumentation-falcon/tests/app.py +++ b/instrumentation/opentelemetry-instrumentation-falcon/tests/app.py @@ -61,6 +61,13 @@ def on_get(self, _, resp): resp.set_header("my-secret-header", "my-secret-value") +class UserResource: + def on_get(self, req, resp, user_id): + # pylint: disable=no-member + resp.status = falcon.HTTP_200 + resp.body = f"Hello user {user_id}" + + def make_app(): _parsed_falcon_version = package_version.parse(falcon.__version__) if _parsed_falcon_version < package_version.parse("3.0.0"): @@ -76,4 +83,6 @@ def make_app(): app.add_route( "/test_custom_response_headers", CustomResponseHeaderResource() ) + app.add_route("/user/{user_id}", UserResource()) + return app diff --git a/instrumentation/opentelemetry-instrumentation-falcon/tests/test_falcon.py b/instrumentation/opentelemetry-instrumentation-falcon/tests/test_falcon.py index aeba57a9b5..2245dbfd80 100644 --- a/instrumentation/opentelemetry-instrumentation-falcon/tests/test_falcon.py +++ b/instrumentation/opentelemetry-instrumentation-falcon/tests/test_falcon.py @@ -110,7 +110,7 @@ def _test_method(self, method): spans = self.memory_exporter.get_finished_spans() self.assertEqual(len(spans), 1) span = spans[0] - self.assertEqual(span.name, f"HelloWorldResource.on_{method.lower()}") + self.assertEqual(span.name, f"{method} /hello") self.assertEqual(span.status.status_code, StatusCode.UNSET) self.assertEqual( span.status.description, @@ -145,7 +145,7 @@ def test_404(self): spans = self.memory_exporter.get_finished_spans() self.assertEqual(len(spans), 1) span = spans[0] - self.assertEqual(span.name, "HTTP GET") + self.assertEqual(span.name, "GET") self.assertEqual(span.status.status_code, StatusCode.UNSET) self.assertSpanHasAttributes( span, @@ -177,7 +177,7 @@ def test_500(self): spans = self.memory_exporter.get_finished_spans() self.assertEqual(len(spans), 1) span = spans[0] - self.assertEqual(span.name, "ErrorResource.on_get") + self.assertEqual(span.name, "GET /error") self.assertFalse(span.status.is_ok) self.assertEqual(span.status.status_code, StatusCode.ERROR) self.assertEqual( @@ -206,6 +206,33 @@ def test_500(self): span.attributes[SpanAttributes.NET_PEER_IP], "127.0.0.1" ) + def test_url_template(self): + self.client().simulate_get("/user/123") + spans = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans), 1) + span = spans[0] + self.assertEqual(span.name, "GET /user/{user_id}") + self.assertEqual(span.status.status_code, StatusCode.UNSET) + self.assertEqual( + span.status.description, + None, + ) + self.assertSpanHasAttributes( + span, + { + SpanAttributes.HTTP_METHOD: "GET", + SpanAttributes.HTTP_SERVER_NAME: "falconframework.org", + SpanAttributes.HTTP_SCHEME: "http", + SpanAttributes.NET_HOST_PORT: 80, + SpanAttributes.HTTP_HOST: "falconframework.org", + SpanAttributes.HTTP_TARGET: "/", + SpanAttributes.NET_PEER_PORT: "65133", + SpanAttributes.HTTP_FLAVOR: "1.1", + "falcon.resource": "UserResource", + SpanAttributes.HTTP_STATUS_CODE: 200, + }, + ) + def test_uninstrument(self): self.client().simulate_get(path="/hello") spans = self.memory_exporter.get_finished_spans() diff --git a/instrumentation/opentelemetry-instrumentation-fastapi/pyproject.toml b/instrumentation/opentelemetry-instrumentation-fastapi/pyproject.toml index b837f82891..4938baaffb 100644 --- a/instrumentation/opentelemetry-instrumentation-fastapi/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-fastapi/pyproject.toml @@ -26,10 +26,10 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-instrumentation-asgi == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", - "opentelemetry-util-http == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-instrumentation-asgi == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", + "opentelemetry-util-http == 0.42b0.dev", ] [project.optional-dependencies] @@ -38,7 +38,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-fastapi[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", "requests ~= 2.23", # needed for testclient "httpx ~= 0.22", # needed for testclient ] diff --git a/instrumentation/opentelemetry-instrumentation-fastapi/src/opentelemetry/instrumentation/fastapi/__init__.py b/instrumentation/opentelemetry-instrumentation-fastapi/src/opentelemetry/instrumentation/fastapi/__init__.py index 934c11e110..e99c8be6ed 100644 --- a/instrumentation/opentelemetry-instrumentation-fastapi/src/opentelemetry/instrumentation/fastapi/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-fastapi/src/opentelemetry/instrumentation/fastapi/__init__.py @@ -227,7 +227,7 @@ def instrument_app( app.add_middleware( OpenTelemetryMiddleware, excluded_urls=excluded_urls, - default_span_details=_get_route_details, + default_span_details=_get_default_span_details, server_request_hook=server_request_hook, client_request_hook=client_request_hook, client_response_hook=client_response_hook, @@ -300,7 +300,7 @@ def __init__(self, *args, **kwargs): self.add_middleware( OpenTelemetryMiddleware, excluded_urls=_InstrumentedFastAPI._excluded_urls, - default_span_details=_get_route_details, + default_span_details=_get_default_span_details, server_request_hook=_InstrumentedFastAPI._server_request_hook, client_request_hook=_InstrumentedFastAPI._client_request_hook, client_response_hook=_InstrumentedFastAPI._client_response_hook, @@ -316,15 +316,21 @@ def __del__(self): def _get_route_details(scope): - """Callback to retrieve the fastapi route being served. + """ + Function to retrieve Starlette route from scope. TODO: there is currently no way to retrieve http.route from a starlette application from scope. - See: https://github.com/encode/starlette/pull/804 + + Args: + scope: A Starlette scope + Returns: + A string containing the route or None """ app = scope["app"] route = None + for starlette_route in app.routes: match, _ = starlette_route.matches(scope) if match == Match.FULL: @@ -332,10 +338,27 @@ def _get_route_details(scope): break if match == Match.PARTIAL: route = starlette_route.path - # method only exists for http, if websocket - # leave it blank. - span_name = route or scope.get("method", "") + return route + + +def _get_default_span_details(scope): + """ + Callback to retrieve span name and attributes from scope. + + Args: + scope: A Starlette scope + Returns: + A tuple of span name and attributes + """ + route = _get_route_details(scope) + method = scope.get("method", "") attributes = {} if route: attributes[SpanAttributes.HTTP_ROUTE] = route + if method and route: # http + span_name = f"{method} {route}" + elif route: # websocket + span_name = route + else: # fallback + span_name = method return span_name, attributes diff --git a/instrumentation/opentelemetry-instrumentation-fastapi/src/opentelemetry/instrumentation/fastapi/version.py b/instrumentation/opentelemetry-instrumentation-fastapi/src/opentelemetry/instrumentation/fastapi/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-fastapi/src/opentelemetry/instrumentation/fastapi/version.py +++ b/instrumentation/opentelemetry-instrumentation-fastapi/src/opentelemetry/instrumentation/fastapi/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-fastapi/tests/test_fastapi_instrumentation.py b/instrumentation/opentelemetry-instrumentation-fastapi/tests/test_fastapi_instrumentation.py index 261b2e025f..4269dfa2e4 100644 --- a/instrumentation/opentelemetry-instrumentation-fastapi/tests/test_fastapi_instrumentation.py +++ b/instrumentation/opentelemetry-instrumentation-fastapi/tests/test_fastapi_instrumentation.py @@ -44,10 +44,20 @@ _expected_metric_names = [ "http.server.active_requests", "http.server.duration", + "http.server.response.size", + "http.server.request.size", ] _recommended_attrs = { "http.server.active_requests": _active_requests_count_attrs, "http.server.duration": {*_duration_attrs, SpanAttributes.HTTP_TARGET}, + "http.server.response.size": { + *_duration_attrs, + SpanAttributes.HTTP_TARGET, + }, + "http.server.request.size": { + *_duration_attrs, + SpanAttributes.HTTP_TARGET, + }, } @@ -106,7 +116,7 @@ def test_instrument_app_with_instrument(self): spans = self.memory_exporter.get_finished_spans() self.assertEqual(len(spans), 3) for span in spans: - self.assertIn("/foobar", span.name) + self.assertIn("GET /foobar", span.name) def test_uninstrument_app(self): self._client.get("/foobar") @@ -138,7 +148,7 @@ def test_basic_fastapi_call(self): spans = self.memory_exporter.get_finished_spans() self.assertEqual(len(spans), 3) for span in spans: - self.assertIn("/foobar", span.name) + self.assertIn("GET /foobar", span.name) def test_fastapi_route_attribute_added(self): """Ensure that fastapi routes are used as the span name.""" @@ -146,7 +156,7 @@ def test_fastapi_route_attribute_added(self): spans = self.memory_exporter.get_finished_spans() self.assertEqual(len(spans), 3) for span in spans: - self.assertIn("/user/{username}", span.name) + self.assertIn("GET /user/{username}", span.name) self.assertEqual( spans[-1].attributes[SpanAttributes.HTTP_ROUTE], "/user/{username}" ) @@ -187,7 +197,7 @@ def test_fastapi_metrics(self): for resource_metric in metrics_list.resource_metrics: self.assertTrue(len(resource_metric.scope_metrics) == 1) for scope_metric in resource_metric.scope_metrics: - self.assertTrue(len(scope_metric.metrics) == 2) + self.assertTrue(len(scope_metric.metrics) == 3) for metric in scope_metric.metrics: self.assertIn(metric.name, _expected_metric_names) data_points = list(metric.data.data_points) @@ -246,8 +256,13 @@ def test_basic_metric_success(self): def test_basic_post_request_metric_success(self): start = default_timer() - self._client.post("/foobar") + response = self._client.post( + "/foobar", + json={"foo": "bar"}, + ) duration = max(round((default_timer() - start) * 1000), 0) + response_size = int(response.headers.get("content-length")) + request_size = int(response.request.headers.get("content-length")) metrics_list = self.memory_metrics_reader.get_metrics_data() for metric in ( metrics_list.resource_metrics[0].scope_metrics[0].metrics @@ -255,7 +270,12 @@ def test_basic_post_request_metric_success(self): for point in list(metric.data.data_points): if isinstance(point, HistogramDataPoint): self.assertEqual(point.count, 1) - self.assertAlmostEqual(duration, point.sum, delta=30) + if metric.name == "http.server.duration": + self.assertAlmostEqual(duration, point.sum, delta=30) + elif metric.name == "http.server.response.size": + self.assertEqual(response_size, point.sum) + elif metric.name == "http.server.request.size": + self.assertEqual(request_size, point.sum) if isinstance(point, NumberDataPoint): self.assertEqual(point.value, 0) diff --git a/instrumentation/opentelemetry-instrumentation-flask/pyproject.toml b/instrumentation/opentelemetry-instrumentation-flask/pyproject.toml index 9dd61a7ef7..62115c83d1 100644 --- a/instrumentation/opentelemetry-instrumentation-flask/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-flask/pyproject.toml @@ -26,10 +26,11 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-instrumentation-wsgi == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", - "opentelemetry-util-http == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-instrumentation-wsgi == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", + "opentelemetry-util-http == 0.42b0.dev", + "packaging >= 21.0", ] [project.optional-dependencies] @@ -38,8 +39,8 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-flask[instruments]", - "markupsafe==2.0.1", - "opentelemetry-test-utils == 0.39b0.dev", + "markupsafe==2.1.2", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-flask/src/opentelemetry/instrumentation/flask/__init__.py b/instrumentation/opentelemetry-instrumentation-flask/src/opentelemetry/instrumentation/flask/__init__.py index fd3c40aab3..432c6b1fbf 100644 --- a/instrumentation/opentelemetry-instrumentation-flask/src/opentelemetry/instrumentation/flask/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-flask/src/opentelemetry/instrumentation/flask/__init__.py @@ -238,13 +238,14 @@ def response_hook(span: Span, status: str, response_headers: List): API --- """ +import weakref from logging import getLogger -from threading import get_ident from time import time_ns from timeit import default_timer from typing import Collection import flask +from packaging import version as package_version import opentelemetry.instrumentation.wsgi as otel_wsgi from opentelemetry import context, trace @@ -265,11 +266,21 @@ def response_hook(span: Span, status: str, response_headers: List): _ENVIRON_STARTTIME_KEY = "opentelemetry-flask.starttime_key" _ENVIRON_SPAN_KEY = "opentelemetry-flask.span_key" _ENVIRON_ACTIVATION_KEY = "opentelemetry-flask.activation_key" -_ENVIRON_THREAD_ID_KEY = "opentelemetry-flask.thread_id_key" +_ENVIRON_REQCTX_REF_KEY = "opentelemetry-flask.reqctx_ref_key" _ENVIRON_TOKEN = "opentelemetry-flask.token" _excluded_urls_from_env = get_excluded_urls("FLASK") +if package_version.parse(flask.__version__) >= package_version.parse("2.2.0"): + + def _request_ctx_ref() -> weakref.ReferenceType: + return weakref.ref(flask.globals.request_ctx._get_current_object()) + +else: + + def _request_ctx_ref() -> weakref.ReferenceType: + return weakref.ref(flask._request_ctx_stack.top) + def get_default_span_name(): try: @@ -364,27 +375,26 @@ def _before_request(): flask_request_environ = flask.request.environ span_name = get_default_span_name() + attributes = otel_wsgi.collect_request_attributes( + flask_request_environ + ) + if flask.request.url_rule: + # For 404 that result from no route found, etc, we + # don't have a url_rule. + attributes[SpanAttributes.HTTP_ROUTE] = flask.request.url_rule.rule span, token = _start_internal_or_server_span( tracer=tracer, span_name=span_name, start_time=flask_request_environ.get(_ENVIRON_STARTTIME_KEY), context_carrier=flask_request_environ, context_getter=otel_wsgi.wsgi_getter, + attributes=attributes, ) if request_hook: request_hook(span, flask_request_environ) if span.is_recording(): - attributes = otel_wsgi.collect_request_attributes( - flask_request_environ - ) - if flask.request.url_rule: - # For 404 that result from no route found, etc, we - # don't have a url_rule. - attributes[ - SpanAttributes.HTTP_ROUTE - ] = flask.request.url_rule.rule for key, value in attributes.items(): span.set_attribute(key, value) if span.is_recording() and span.kind == trace.SpanKind.SERVER: @@ -399,7 +409,7 @@ def _before_request(): activation = trace.use_span(span, end_on_exit=True) activation.__enter__() # pylint: disable=E1101 flask_request_environ[_ENVIRON_ACTIVATION_KEY] = activation - flask_request_environ[_ENVIRON_THREAD_ID_KEY] = get_ident() + flask_request_environ[_ENVIRON_REQCTX_REF_KEY] = _request_ctx_ref() flask_request_environ[_ENVIRON_SPAN_KEY] = span flask_request_environ[_ENVIRON_TOKEN] = token @@ -439,17 +449,22 @@ def _teardown_request(exc): return activation = flask.request.environ.get(_ENVIRON_ACTIVATION_KEY) - thread_id = flask.request.environ.get(_ENVIRON_THREAD_ID_KEY) - if not activation or thread_id != get_ident(): + + original_reqctx_ref = flask.request.environ.get( + _ENVIRON_REQCTX_REF_KEY + ) + current_reqctx_ref = _request_ctx_ref() + if not activation or original_reqctx_ref != current_reqctx_ref: # This request didn't start a span, maybe because it was created in # a way that doesn't run `before_request`, like when it is created # with `app.test_request_context`. # - # Similarly, check the thread_id against the current thread to ensure - # tear down only happens on the original thread. This situation can - # arise if the original thread handling the request spawn children - # threads and then uses something like copy_current_request_context - # to copy the request context. + # Similarly, check that the request_ctx that created the span + # matches the current request_ctx, and only tear down if they match. + # This situation can arise if the original request_ctx handling + # the request calls functions that push new request_ctx's, + # like any decorated with `flask.copy_current_request_context`. + return if exc is None: activation.__exit__(None, None, None) diff --git a/instrumentation/opentelemetry-instrumentation-flask/src/opentelemetry/instrumentation/flask/version.py b/instrumentation/opentelemetry-instrumentation-flask/src/opentelemetry/instrumentation/flask/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-flask/src/opentelemetry/instrumentation/flask/version.py +++ b/instrumentation/opentelemetry-instrumentation-flask/src/opentelemetry/instrumentation/flask/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-flask/tests/base_test.py b/instrumentation/opentelemetry-instrumentation-flask/tests/base_test.py index a9cc4e55f7..6117521bb9 100644 --- a/instrumentation/opentelemetry-instrumentation-flask/tests/base_test.py +++ b/instrumentation/opentelemetry-instrumentation-flask/tests/base_test.py @@ -19,7 +19,7 @@ from werkzeug.test import Client from werkzeug.wrappers import Response -from opentelemetry import context +from opentelemetry import context, trace class InstrumentationTest: @@ -37,6 +37,21 @@ def _sqlcommenter_endpoint(): ) return sqlcommenter_flask_values + @staticmethod + def _copy_context_endpoint(): + @flask.copy_current_request_context + def _extract_header(): + return flask.request.headers["x-req"] + + # Despite `_extract_header` copying the request context, + # calling it shouldn't detach the parent Flask span's contextvar + request_header = _extract_header() + + return { + "span_name": trace.get_current_span().name, + "request_header": request_header, + } + @staticmethod def _multithreaded_endpoint(count): def do_random_stuff(): @@ -84,6 +99,7 @@ def excluded2_endpoint(): self.app.route("/hello/")(self._hello_endpoint) self.app.route("/sqlcommenter")(self._sqlcommenter_endpoint) self.app.route("/multithreaded")(self._multithreaded_endpoint) + self.app.route("/copy_context")(self._copy_context_endpoint) self.app.route("/excluded/")(self._hello_endpoint) self.app.route("/excluded")(excluded_endpoint) self.app.route("/excluded2")(excluded2_endpoint) diff --git a/instrumentation/opentelemetry-instrumentation-flask/tests/test_copy_context.py b/instrumentation/opentelemetry-instrumentation-flask/tests/test_copy_context.py new file mode 100644 index 0000000000..96268de5e7 --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-flask/tests/test_copy_context.py @@ -0,0 +1,48 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +import flask +from werkzeug.test import Client +from werkzeug.wrappers import Response + +from opentelemetry.instrumentation.flask import FlaskInstrumentor +from opentelemetry.test.wsgitestutil import WsgiTestBase + +from .base_test import InstrumentationTest + + +class TestCopyContext(InstrumentationTest, WsgiTestBase): + def setUp(self): + super().setUp() + FlaskInstrumentor().instrument() + self.app = flask.Flask(__name__) + self._common_initialization() + + def tearDown(self): + super().tearDown() + with self.disable_logging(): + FlaskInstrumentor().uninstrument() + + def test_copycontext(self): + """Test that instrumentation tear down does not blow up + when the request calls functions where the context has been + copied via `flask.copy_current_request_context` + """ + self.app = flask.Flask(__name__) + self.app.route("/copy_context")(self._copy_context_endpoint) + client = Client(self.app, Response) + resp = client.get("/copy_context", headers={"x-req": "a-header"}) + + self.assertEqual(200, resp.status_code) + self.assertEqual("/copy_context", resp.json["span_name"]) + self.assertEqual("a-header", resp.json["request_header"]) diff --git a/instrumentation/opentelemetry-instrumentation-flask/tests/test_programmatic.py b/instrumentation/opentelemetry-instrumentation-flask/tests/test_programmatic.py index 8c231b1d08..bf641aaed4 100644 --- a/instrumentation/opentelemetry-instrumentation-flask/tests/test_programmatic.py +++ b/instrumentation/opentelemetry-instrumentation-flask/tests/test_programmatic.py @@ -40,6 +40,8 @@ OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS, OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST, OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE, + OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE, + OTEL_PYTHON_INSTRUMENTATION_HTTP_CAPTURE_ALL_METHODS, get_excluded_urls, ) @@ -214,7 +216,7 @@ def test_404(self): resp.close() span_list = self.memory_exporter.get_finished_spans() self.assertEqual(len(span_list), 1) - self.assertEqual(span_list[0].name, "HTTP POST") + self.assertEqual(span_list[0].name, "POST /bye") self.assertEqual(span_list[0].kind, trace.SpanKind.SERVER) self.assertEqual(span_list[0].attributes, expected_attrs) @@ -326,6 +328,25 @@ def test_flask_metric_values(self): if isinstance(point, NumberDataPoint): self.assertEqual(point.value, 0) + def _assert_basic_metric(self, expected_duration_attributes, expected_requests_count_attributes): + metrics_list = self.memory_metrics_reader.get_metrics_data() + for resource_metric in metrics_list.resource_metrics: + for scope_metrics in resource_metric.scope_metrics: + for metric in scope_metrics.metrics: + for point in list(metric.data.data_points): + if isinstance(point, HistogramDataPoint): + self.assertDictEqual( + expected_duration_attributes, + dict(point.attributes), + ) + self.assertEqual(point.count, 1) + elif isinstance(point, NumberDataPoint): + self.assertDictEqual( + expected_requests_count_attributes, + dict(point.attributes), + ) + self.assertEqual(point.value, 0) + def test_basic_metric_success(self): self.client.get("/hello/756") expected_duration_attributes = { @@ -344,23 +365,62 @@ def test_basic_metric_success(self): "http.flavor": "1.1", "http.server_name": "localhost", } - metrics_list = self.memory_metrics_reader.get_metrics_data() - for resource_metric in metrics_list.resource_metrics: - for scope_metrics in resource_metric.scope_metrics: - for metric in scope_metrics.metrics: - for point in list(metric.data.data_points): - if isinstance(point, HistogramDataPoint): - self.assertDictEqual( - expected_duration_attributes, - dict(point.attributes), - ) - self.assertEqual(point.count, 1) - elif isinstance(point, NumberDataPoint): - self.assertDictEqual( - expected_requests_count_attributes, - dict(point.attributes), - ) - self.assertEqual(point.value, 0) + self._assert_basic_metric( + expected_duration_attributes, + expected_requests_count_attributes, + ) + + def test_basic_metric_nonstandard_http_method_success(self): + self.client.open("/hello/756", method="NONSTANDARD") + expected_duration_attributes = { + "http.method": "UNKNOWN", + "http.host": "localhost", + "http.scheme": "http", + "http.flavor": "1.1", + "http.server_name": "localhost", + "net.host.port": 80, + "http.status_code": 405, + } + expected_requests_count_attributes = { + "http.method": "UNKNOWN", + "http.host": "localhost", + "http.scheme": "http", + "http.flavor": "1.1", + "http.server_name": "localhost", + } + self._assert_basic_metric( + expected_duration_attributes, + expected_requests_count_attributes, + ) + + @patch.dict( + "os.environ", + { + OTEL_PYTHON_INSTRUMENTATION_HTTP_CAPTURE_ALL_METHODS: "1", + }, + ) + def test_basic_metric_nonstandard_http_method_allowed_success(self): + self.client.open("/hello/756", method="NONSTANDARD") + expected_duration_attributes = { + "http.method": "NONSTANDARD", + "http.host": "localhost", + "http.scheme": "http", + "http.flavor": "1.1", + "http.server_name": "localhost", + "net.host.port": 80, + "http.status_code": 405, + } + expected_requests_count_attributes = { + "http.method": "NONSTANDARD", + "http.host": "localhost", + "http.scheme": "http", + "http.flavor": "1.1", + "http.server_name": "localhost", + } + self._assert_basic_metric( + expected_duration_attributes, + expected_requests_count_attributes, + ) def test_metric_uninstrument(self): self.client.delete("/hello/756") diff --git a/instrumentation/opentelemetry-instrumentation-grpc/pyproject.toml b/instrumentation/opentelemetry-instrumentation-grpc/pyproject.toml index 7351d49340..37023eae4d 100644 --- a/instrumentation/opentelemetry-instrumentation-grpc/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-grpc/pyproject.toml @@ -26,9 +26,9 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", "opentelemetry-sdk ~= 1.12", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", "wrapt >= 1.0.0, < 2.0.0", ] @@ -39,7 +39,7 @@ instruments = [ test = [ "opentelemetry-instrumentation-grpc[instruments]", "opentelemetry-sdk ~= 1.12", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", "protobuf ~= 3.13", ] diff --git a/instrumentation/opentelemetry-instrumentation-grpc/src/opentelemetry/instrumentation/grpc/_server.py b/instrumentation/opentelemetry-instrumentation-grpc/src/opentelemetry/instrumentation/grpc/_server.py index a34cac0b3c..dcee959b4d 100644 --- a/instrumentation/opentelemetry-instrumentation-grpc/src/opentelemetry/instrumentation/grpc/_server.py +++ b/instrumentation/opentelemetry-instrumentation-grpc/src/opentelemetry/instrumentation/grpc/_server.py @@ -250,24 +250,30 @@ def _start_span( # * ipv4:127.0.0.1:57284 # * ipv4:10.2.1.1:57284,127.0.0.1:57284 # - try: - ip, port = ( - context.peer().split(",")[0].split(":", 1)[1].rsplit(":", 1) - ) - ip = unquote(ip) - attributes.update( - { - SpanAttributes.NET_PEER_IP: ip, - SpanAttributes.NET_PEER_PORT: port, - } - ) + if context.peer() != "unix:": + try: + ip, port = ( + context.peer() + .split(",")[0] + .split(":", 1)[1] + .rsplit(":", 1) + ) + ip = unquote(ip) + attributes.update( + { + SpanAttributes.NET_PEER_IP: ip, + SpanAttributes.NET_PEER_PORT: port, + } + ) - # other telemetry sources add this, so we will too - if ip in ("[::1]", "127.0.0.1"): - attributes[SpanAttributes.NET_PEER_NAME] = "localhost" + # other telemetry sources add this, so we will too + if ip in ("[::1]", "127.0.0.1"): + attributes[SpanAttributes.NET_PEER_NAME] = "localhost" - except IndexError: - logger.warning("Failed to parse peer address '%s'", context.peer()) + except IndexError: + logger.warning( + "Failed to parse peer address '%s'", context.peer() + ) return self._tracer.start_as_current_span( name=handler_call_details.method, diff --git a/instrumentation/opentelemetry-instrumentation-grpc/src/opentelemetry/instrumentation/grpc/version.py b/instrumentation/opentelemetry-instrumentation-grpc/src/opentelemetry/instrumentation/grpc/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-grpc/src/opentelemetry/instrumentation/grpc/version.py +++ b/instrumentation/opentelemetry-instrumentation-grpc/src/opentelemetry/instrumentation/grpc/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-grpc/tests/test_server_interceptor.py b/instrumentation/opentelemetry-instrumentation-grpc/tests/test_server_interceptor.py index b48d887f5a..57f27c89d6 100644 --- a/instrumentation/opentelemetry-instrumentation-grpc/tests/test_server_interceptor.py +++ b/instrumentation/opentelemetry-instrumentation-grpc/tests/test_server_interceptor.py @@ -15,6 +15,8 @@ # pylint:disable=unused-argument # pylint:disable=no-self-use +import contextlib +import tempfile import threading from concurrent import futures @@ -78,23 +80,32 @@ def ServerStreamingMethod(self, request, context): class TestOpenTelemetryServerInterceptor(TestBase): - def test_instrumentor(self): - def handler(request, context): - return b"" - - grpc_server_instrumentor = GrpcInstrumentorServer() - grpc_server_instrumentor.instrument() - with futures.ThreadPoolExecutor(max_workers=1) as executor: + net_peer_span_attributes = { + SpanAttributes.NET_PEER_IP: "[::1]", + SpanAttributes.NET_PEER_NAME: "localhost", + } + + @contextlib.contextmanager + def server(self, max_workers=1, interceptors=None): + with futures.ThreadPoolExecutor(max_workers=max_workers) as executor: server = grpc.server( executor, options=(("grpc.so_reuseport", 0),), + interceptors=interceptors or [], ) - server.add_generic_rpc_handlers((UnaryUnaryRpcHandler(handler),)) - port = server.add_insecure_port("[::]:0") channel = grpc.insecure_channel(f"localhost:{port:d}") + yield server, channel + + def test_instrumentor(self): + def handler(request, context): + return b"" + grpc_server_instrumentor = GrpcInstrumentorServer() + grpc_server_instrumentor.instrument() + with self.server(max_workers=1) as (server, channel): + server.add_generic_rpc_handlers((UnaryUnaryRpcHandler(handler),)) rpc_call = "TestServicer/handler" try: server.start() @@ -117,8 +128,7 @@ def handler(request, context): self.assertSpanHasAttributes( span, { - SpanAttributes.NET_PEER_IP: "[::1]", - SpanAttributes.NET_PEER_NAME: "localhost", + **self.net_peer_span_attributes, SpanAttributes.RPC_METHOD: "handler", SpanAttributes.RPC_SERVICE: "TestServicer", SpanAttributes.RPC_SYSTEM: "grpc", @@ -137,17 +147,8 @@ def handler(request, context): grpc_server_instrumentor = GrpcInstrumentorServer() grpc_server_instrumentor.instrument() grpc_server_instrumentor.uninstrument() - with futures.ThreadPoolExecutor(max_workers=1) as executor: - server = grpc.server( - executor, - options=(("grpc.so_reuseport", 0),), - ) - + with self.server(max_workers=1) as (server, channel): server.add_generic_rpc_handlers((UnaryUnaryRpcHandler(handler),)) - - port = server.add_insecure_port("[::]:0") - channel = grpc.insecure_channel(f"localhost:{port:d}") - rpc_call = "TestServicer/test" try: server.start() @@ -164,15 +165,11 @@ def test_create_span(self): # Intercept gRPC calls... interceptor = server_interceptor() - with futures.ThreadPoolExecutor(max_workers=1) as executor: - server = grpc.server( - executor, - options=(("grpc.so_reuseport", 0),), - interceptors=[interceptor], - ) + with self.server( + max_workers=1, + interceptors=[interceptor], + ) as (server, channel): add_GRPCTestServerServicer_to_server(Servicer(), server) - port = server.add_insecure_port("[::]:0") - channel = grpc.insecure_channel(f"localhost:{port:d}") rpc_call = "/GRPCTestServer/SimpleMethod" request = Request(client_id=1, request_data="test") @@ -199,8 +196,7 @@ def test_create_span(self): self.assertSpanHasAttributes( span, { - SpanAttributes.NET_PEER_IP: "[::1]", - SpanAttributes.NET_PEER_NAME: "localhost", + **self.net_peer_span_attributes, SpanAttributes.RPC_METHOD: "SimpleMethod", SpanAttributes.RPC_SERVICE: "GRPCTestServer", SpanAttributes.RPC_SYSTEM: "grpc", @@ -231,15 +227,11 @@ def SimpleMethod(self, request, context): interceptor = server_interceptor() # setup the server - with futures.ThreadPoolExecutor(max_workers=1) as executor: - server = grpc.server( - executor, - options=(("grpc.so_reuseport", 0),), - interceptors=[interceptor], - ) + with self.server( + max_workers=1, + interceptors=[interceptor], + ) as (server, channel): add_GRPCTestServerServicer_to_server(TwoSpanServicer(), server) - port = server.add_insecure_port("[::]:0") - channel = grpc.insecure_channel(f"localhost:{port:d}") # setup the RPC rpc_call = "/GRPCTestServer/SimpleMethod" @@ -268,8 +260,7 @@ def SimpleMethod(self, request, context): self.assertSpanHasAttributes( parent_span, { - SpanAttributes.NET_PEER_IP: "[::1]", - SpanAttributes.NET_PEER_NAME: "localhost", + **self.net_peer_span_attributes, SpanAttributes.RPC_METHOD: "SimpleMethod", SpanAttributes.RPC_SERVICE: "GRPCTestServer", SpanAttributes.RPC_SYSTEM: "grpc", @@ -292,15 +283,11 @@ def test_create_span_streaming(self): # Intercept gRPC calls... interceptor = server_interceptor() - with futures.ThreadPoolExecutor(max_workers=1) as executor: - server = grpc.server( - executor, - options=(("grpc.so_reuseport", 0),), - interceptors=[interceptor], - ) + with self.server( + max_workers=1, + interceptors=[interceptor], + ) as (server, channel): add_GRPCTestServerServicer_to_server(Servicer(), server) - port = server.add_insecure_port("[::]:0") - channel = grpc.insecure_channel(f"localhost:{port:d}") # setup the RPC rpc_call = "/GRPCTestServer/ServerStreamingMethod" @@ -328,8 +315,7 @@ def test_create_span_streaming(self): self.assertSpanHasAttributes( span, { - SpanAttributes.NET_PEER_IP: "[::1]", - SpanAttributes.NET_PEER_NAME: "localhost", + **self.net_peer_span_attributes, SpanAttributes.RPC_METHOD: "ServerStreamingMethod", SpanAttributes.RPC_SERVICE: "GRPCTestServer", SpanAttributes.RPC_SYSTEM: "grpc", @@ -360,15 +346,11 @@ def ServerStreamingMethod(self, request, context): # Intercept gRPC calls... interceptor = server_interceptor() - with futures.ThreadPoolExecutor(max_workers=1) as executor: - server = grpc.server( - executor, - options=(("grpc.so_reuseport", 0),), - interceptors=[interceptor], - ) + with self.server( + max_workers=1, + interceptors=[interceptor], + ) as (server, channel): add_GRPCTestServerServicer_to_server(TwoSpanServicer(), server) - port = server.add_insecure_port("[::]:0") - channel = grpc.insecure_channel(f"localhost:{port:d}") # setup the RPC rpc_call = "/GRPCTestServer/ServerStreamingMethod" @@ -397,8 +379,7 @@ def ServerStreamingMethod(self, request, context): self.assertSpanHasAttributes( parent_span, { - SpanAttributes.NET_PEER_IP: "[::1]", - SpanAttributes.NET_PEER_NAME: "localhost", + **self.net_peer_span_attributes, SpanAttributes.RPC_METHOD: "ServerStreamingMethod", SpanAttributes.RPC_SERVICE: "GRPCTestServer", SpanAttributes.RPC_SYSTEM: "grpc", @@ -427,17 +408,12 @@ def handler(request, context): active_span_in_handler = trace.get_current_span() return b"" - with futures.ThreadPoolExecutor(max_workers=1) as executor: - server = grpc.server( - executor, - options=(("grpc.so_reuseport", 0),), - interceptors=[interceptor], - ) + with self.server( + max_workers=1, + interceptors=[interceptor], + ) as (server, channel): server.add_generic_rpc_handlers((UnaryUnaryRpcHandler(handler),)) - port = server.add_insecure_port("[::]:0") - channel = grpc.insecure_channel(f"localhost:{port:d}") - active_span_before_call = trace.get_current_span() try: server.start() @@ -463,17 +439,12 @@ def handler(request, context): active_spans_in_handler.append(trace.get_current_span()) return b"" - with futures.ThreadPoolExecutor(max_workers=1) as executor: - server = grpc.server( - executor, - options=(("grpc.so_reuseport", 0),), - interceptors=[interceptor], - ) + with self.server( + max_workers=1, + interceptors=[interceptor], + ) as (server, channel): server.add_generic_rpc_handlers((UnaryUnaryRpcHandler(handler),)) - port = server.add_insecure_port("[::]:0") - channel = grpc.insecure_channel(f"localhost:{port:d}") - try: server.start() channel.unary_unary("TestServicer/handler")(b"") @@ -496,8 +467,7 @@ def handler(request, context): self.assertSpanHasAttributes( span, { - SpanAttributes.NET_PEER_IP: "[::1]", - SpanAttributes.NET_PEER_NAME: "localhost", + **self.net_peer_span_attributes, SpanAttributes.RPC_METHOD: "handler", SpanAttributes.RPC_SERVICE: "TestServicer", SpanAttributes.RPC_SYSTEM: "grpc", @@ -527,17 +497,12 @@ def handler(request, context): active_spans_in_handler.append(trace.get_current_span()) return b"" - with futures.ThreadPoolExecutor(max_workers=2) as executor: - server = grpc.server( - executor, - options=(("grpc.so_reuseport", 0),), - interceptors=[interceptor], - ) + with self.server( + max_workers=2, + interceptors=[interceptor], + ) as (server, channel): server.add_generic_rpc_handlers((UnaryUnaryRpcHandler(handler),)) - port = server.add_insecure_port("[::]:0") - channel = grpc.insecure_channel(f"localhost:{port:d}") - try: server.start() # Interleave calls so spans are active on each thread at the same @@ -568,8 +533,7 @@ def handler(request, context): self.assertSpanHasAttributes( span, { - SpanAttributes.NET_PEER_IP: "[::1]", - SpanAttributes.NET_PEER_NAME: "localhost", + **self.net_peer_span_attributes, SpanAttributes.RPC_METHOD: "handler", SpanAttributes.RPC_SERVICE: "TestServicer", SpanAttributes.RPC_SYSTEM: "grpc", @@ -592,18 +556,11 @@ def test_abort(self): def handler(request, context): context.abort(grpc.StatusCode.FAILED_PRECONDITION, failure_message) - with futures.ThreadPoolExecutor(max_workers=1) as executor: - server = grpc.server( - executor, - options=(("grpc.so_reuseport", 0),), - interceptors=[interceptor], - ) - + with self.server( + max_workers=1, + interceptors=[interceptor], + ) as (server, channel): server.add_generic_rpc_handlers((UnaryUnaryRpcHandler(handler),)) - - port = server.add_insecure_port("[::]:0") - channel = grpc.insecure_channel(f"localhost:{port:d}") - rpc_call = "TestServicer/handler" server.start() @@ -635,8 +592,7 @@ def handler(request, context): self.assertSpanHasAttributes( span, { - SpanAttributes.NET_PEER_IP: "[::1]", - SpanAttributes.NET_PEER_NAME: "localhost", + **self.net_peer_span_attributes, SpanAttributes.RPC_METHOD: "handler", SpanAttributes.RPC_SERVICE: "TestServicer", SpanAttributes.RPC_SYSTEM: "grpc", @@ -647,6 +603,28 @@ def handler(request, context): ) +class TestOpenTelemetryServerInterceptorUnix( + TestOpenTelemetryServerInterceptor, +): + net_peer_span_attributes = {} + + @contextlib.contextmanager + def server(self, max_workers=1, interceptors=None): + with futures.ThreadPoolExecutor( + max_workers=max_workers + ) as executor, tempfile.TemporaryDirectory() as tmp: + server = grpc.server( + executor, + options=(("grpc.so_reuseport", 0),), + interceptors=interceptors or [], + ) + + sock = f"unix://{tmp}/grpc.sock" + server.add_insecure_port(sock) + channel = grpc.insecure_channel(sock) + yield server, channel + + def get_latch(num): """Get a countdown latch function for use in n threads.""" cv = threading.Condition() diff --git a/instrumentation/opentelemetry-instrumentation-httpx/README.rst b/instrumentation/opentelemetry-instrumentation-httpx/README.rst index ffa86cb4bc..1e03eb128e 100644 --- a/instrumentation/opentelemetry-instrumentation-httpx/README.rst +++ b/instrumentation/opentelemetry-instrumentation-httpx/README.rst @@ -30,7 +30,7 @@ When using the instrumentor, all clients will automatically trace requests. import httpx from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor - url = "https://httpbin.org/get" + url = "https://some.url/get" HTTPXClientInstrumentor().instrument() with httpx.Client() as client: @@ -51,7 +51,7 @@ use the `instrument_client` method. import httpx from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor - url = "https://httpbin.org/get" + url = "https://some.url/get" with httpx.Client(transport=telemetry_transport) as client: HTTPXClientInstrumentor.instrument_client(client) @@ -96,7 +96,7 @@ If you don't want to use the instrumentor class, you can use the transport class SyncOpenTelemetryTransport, ) - url = "https://httpbin.org/get" + url = "https://some.url/get" transport = httpx.HTTPTransport() telemetry_transport = SyncOpenTelemetryTransport(transport) diff --git a/instrumentation/opentelemetry-instrumentation-httpx/pyproject.toml b/instrumentation/opentelemetry-instrumentation-httpx/pyproject.toml index fbbc9ba718..a50b2f4754 100644 --- a/instrumentation/opentelemetry-instrumentation-httpx/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-httpx/pyproject.toml @@ -26,18 +26,18 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", ] [project.optional-dependencies] instruments = [ - "httpx >= 0.18.0, <= 0.23.0", + "httpx >= 0.18.0", ] test = [ "opentelemetry-instrumentation-httpx[instruments]", "opentelemetry-sdk ~= 1.12", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-httpx/src/opentelemetry/instrumentation/httpx/__init__.py b/instrumentation/opentelemetry-instrumentation-httpx/src/opentelemetry/instrumentation/httpx/__init__.py index b603cbcdd6..bb40adbc26 100644 --- a/instrumentation/opentelemetry-instrumentation-httpx/src/opentelemetry/instrumentation/httpx/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-httpx/src/opentelemetry/instrumentation/httpx/__init__.py @@ -25,7 +25,7 @@ import httpx from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor - url = "https://httpbin.org/get" + url = "https://some.url/get" HTTPXClientInstrumentor().instrument() with httpx.Client() as client: @@ -46,7 +46,7 @@ import httpx from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor - url = "https://httpbin.org/get" + url = "https://some.url/get" with httpx.Client(transport=telemetry_transport) as client: HTTPXClientInstrumentor.instrument_client(client) @@ -91,7 +91,7 @@ SyncOpenTelemetryTransport, ) - url = "https://httpbin.org/get" + url = "https://some.url/get" transport = httpx.HTTPTransport() telemetry_transport = SyncOpenTelemetryTransport(transport) @@ -209,7 +209,7 @@ class ResponseInfo(typing.NamedTuple): def _get_default_span_name(method: str) -> str: - return f"HTTP {method.strip()}" + return method.strip() def _apply_status_code(span: Span, status_code: int) -> None: diff --git a/instrumentation/opentelemetry-instrumentation-httpx/src/opentelemetry/instrumentation/httpx/version.py b/instrumentation/opentelemetry-instrumentation-httpx/src/opentelemetry/instrumentation/httpx/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-httpx/src/opentelemetry/instrumentation/httpx/version.py +++ b/instrumentation/opentelemetry-instrumentation-httpx/src/opentelemetry/instrumentation/httpx/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-httpx/tests/test_httpx_integration.py b/instrumentation/opentelemetry-instrumentation-httpx/tests/test_httpx_integration.py index 3cac4c45a7..daddaad306 100644 --- a/instrumentation/opentelemetry-instrumentation-httpx/tests/test_httpx_integration.py +++ b/instrumentation/opentelemetry-instrumentation-httpx/tests/test_httpx_integration.py @@ -59,7 +59,7 @@ def _async_call(coro: typing.Coroutine) -> asyncio.Task: def _response_hook(span, request: "RequestInfo", response: "ResponseInfo"): span.set_attribute( HTTP_RESPONSE_BODY, - response[2].read(), + b"".join(response[2]), ) @@ -68,7 +68,7 @@ async def _async_response_hook( ): span.set_attribute( HTTP_RESPONSE_BODY, - await response[2].aread(), + b"".join([part async for part in response[2]]), ) @@ -97,7 +97,7 @@ class BaseTestCases: class BaseTest(TestBase, metaclass=abc.ABCMeta): # pylint: disable=no-member - URL = "http://httpbin.org/status/200" + URL = "http://mock/status/200" response_hook = staticmethod(_response_hook) request_hook = staticmethod(_request_hook) no_update_request_hook = staticmethod(_no_update_request_hook) @@ -142,7 +142,7 @@ def test_basic(self): span = self.assert_span() self.assertIs(span.kind, trace.SpanKind.CLIENT) - self.assertEqual(span.name, "HTTP GET") + self.assertEqual(span.name, "GET") self.assertEqual( span.attributes, @@ -165,7 +165,7 @@ def test_basic_multiple(self): self.assert_span(num_spans=2) def test_not_foundbasic(self): - url_404 = "http://httpbin.org/status/404" + url_404 = "http://mock/status/404" with respx.mock: respx.get(url_404).mock(httpx.Response(404)) @@ -258,7 +258,7 @@ def test_invalid_url(self): span = self.assert_span() - self.assertEqual(span.name, "HTTP POST") + self.assertEqual(span.name, "POST") self.assertEqual( span.attributes[SpanAttributes.HTTP_METHOD], "POST" ) @@ -350,7 +350,7 @@ def test_request_hook_no_span_change(self): self.assertEqual(result.text, "Hello!") span = self.assert_span() - self.assertEqual(span.name, "HTTP GET") + self.assertEqual(span.name, "GET") def test_not_recording(self): with mock.patch("opentelemetry.trace.INVALID_SPAN") as mock_span: @@ -444,7 +444,7 @@ def test_request_hook_no_span_update(self): self.assertEqual(result.text, "Hello!") span = self.assert_span() - self.assertEqual(span.name, "HTTP GET") + self.assertEqual(span.name, "GET") HTTPXClientInstrumentor().uninstrument() def test_not_recording(self): diff --git a/instrumentation/opentelemetry-instrumentation-jinja2/pyproject.toml b/instrumentation/opentelemetry-instrumentation-jinja2/pyproject.toml index b6f6f2cdee..c0f0c51d37 100644 --- a/instrumentation/opentelemetry-instrumentation-jinja2/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-jinja2/pyproject.toml @@ -25,7 +25,7 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", "wrapt >= 1.0.0, < 2.0.0", ] @@ -36,7 +36,7 @@ instruments = [ test = [ "opentelemetry-instrumentation-jinja2[instruments]", "markupsafe==2.0.1", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-jinja2/src/opentelemetry/instrumentation/jinja2/version.py b/instrumentation/opentelemetry-instrumentation-jinja2/src/opentelemetry/instrumentation/jinja2/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-jinja2/src/opentelemetry/instrumentation/jinja2/version.py +++ b/instrumentation/opentelemetry-instrumentation-jinja2/src/opentelemetry/instrumentation/jinja2/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-kafka-python/pyproject.toml b/instrumentation/opentelemetry-instrumentation-kafka-python/pyproject.toml index 1d83e30c57..9fbde3057d 100644 --- a/instrumentation/opentelemetry-instrumentation-kafka-python/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-kafka-python/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.5", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", ] [project.optional-dependencies] @@ -36,7 +36,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-kafka-python[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", "wrapt >= 1.0.0, < 2.0.0", ] diff --git a/instrumentation/opentelemetry-instrumentation-kafka-python/src/opentelemetry/instrumentation/kafka/version.py b/instrumentation/opentelemetry-instrumentation-kafka-python/src/opentelemetry/instrumentation/kafka/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-kafka-python/src/opentelemetry/instrumentation/kafka/version.py +++ b/instrumentation/opentelemetry-instrumentation-kafka-python/src/opentelemetry/instrumentation/kafka/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-logging/pyproject.toml b/instrumentation/opentelemetry-instrumentation-logging/pyproject.toml index 26d8867652..76bde24c55 100644 --- a/instrumentation/opentelemetry-instrumentation-logging/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-logging/pyproject.toml @@ -25,13 +25,13 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", ] [project.optional-dependencies] instruments = [] test = [ - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-logging/src/opentelemetry/instrumentation/logging/__init__.py b/instrumentation/opentelemetry-instrumentation-logging/src/opentelemetry/instrumentation/logging/__init__.py index d419aaa2f9..ce332d0113 100644 --- a/instrumentation/opentelemetry-instrumentation-logging/src/opentelemetry/instrumentation/logging/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-logging/src/opentelemetry/instrumentation/logging/__init__.py @@ -94,6 +94,7 @@ def record_factory(*args, **kwargs): record.otelSpanID = "0" record.otelTraceID = "0" + record.otelTraceSampled = False nonlocal service_name if service_name is None: @@ -113,6 +114,7 @@ def record_factory(*args, **kwargs): if ctx != INVALID_SPAN_CONTEXT: record.otelSpanID = format(ctx.span_id, "016x") record.otelTraceID = format(ctx.trace_id, "032x") + record.otelTraceSampled = ctx.trace_flags.sampled if callable(LoggingInstrumentor._log_hook): try: LoggingInstrumentor._log_hook( # pylint: disable=E1102 diff --git a/instrumentation/opentelemetry-instrumentation-logging/src/opentelemetry/instrumentation/logging/constants.py b/instrumentation/opentelemetry-instrumentation-logging/src/opentelemetry/instrumentation/logging/constants.py index ff50d46086..b18f93364f 100644 --- a/instrumentation/opentelemetry-instrumentation-logging/src/opentelemetry/instrumentation/logging/constants.py +++ b/instrumentation/opentelemetry-instrumentation-logging/src/opentelemetry/instrumentation/logging/constants.py @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -DEFAULT_LOGGING_FORMAT = "%(asctime)s %(levelname)s [%(name)s] [%(filename)s:%(lineno)d] [trace_id=%(otelTraceID)s span_id=%(otelSpanID)s resource.service.name=%(otelServiceName)s] - %(message)s" +DEFAULT_LOGGING_FORMAT = "%(asctime)s %(levelname)s [%(name)s] [%(filename)s:%(lineno)d] [trace_id=%(otelTraceID)s span_id=%(otelSpanID)s resource.service.name=%(otelServiceName)s trace_sampled=%(otelTraceSampled)s] - %(message)s" _MODULE_DOC = """ @@ -27,6 +27,7 @@ - ``otelSpanID`` - ``otelTraceID`` - ``otelServiceName`` +- ``otelTraceSampled`` The integration uses the following logging format by default: @@ -113,7 +114,7 @@ .. code-block:: - %(otelSpanID)s %(otelTraceID)s %(otelServiceName)s + %(otelSpanID)s %(otelTraceID)s %(otelServiceName)s %(otelTraceSampled)s diff --git a/instrumentation/opentelemetry-instrumentation-logging/src/opentelemetry/instrumentation/logging/version.py b/instrumentation/opentelemetry-instrumentation-logging/src/opentelemetry/instrumentation/logging/version.py index 16c754f1d3..084725a38e 100644 --- a/instrumentation/opentelemetry-instrumentation-logging/src/opentelemetry/instrumentation/logging/version.py +++ b/instrumentation/opentelemetry-instrumentation-logging/src/opentelemetry/instrumentation/logging/version.py @@ -12,6 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" _instruments = tuple() diff --git a/instrumentation/opentelemetry-instrumentation-logging/tests/test_logging.py b/instrumentation/opentelemetry-instrumentation-logging/tests/test_logging.py index bddee9bf84..a5a0d5adff 100644 --- a/instrumentation/opentelemetry-instrumentation-logging/tests/test_logging.py +++ b/instrumentation/opentelemetry-instrumentation-logging/tests/test_logging.py @@ -62,6 +62,7 @@ def test_trace_context_injection(self): self.assertEqual(record.otelSpanID, "0") self.assertEqual(record.otelTraceID, "0") self.assertEqual(record.otelServiceName, "") + self.assertEqual(record.otelTraceSampled, False) def log_hook(span, record): @@ -82,7 +83,7 @@ def tearDown(self): super().tearDown() LoggingInstrumentor().uninstrument() - def assert_trace_context_injected(self, span_id, trace_id): + def assert_trace_context_injected(self, span_id, trace_id, trace_sampled): with self.caplog.at_level(level=logging.INFO): logger = logging.getLogger("test logger") logger.info("hello") @@ -90,16 +91,20 @@ def assert_trace_context_injected(self, span_id, trace_id): record = self.caplog.records[0] self.assertEqual(record.otelSpanID, span_id) self.assertEqual(record.otelTraceID, trace_id) + self.assertEqual(record.otelTraceSampled, trace_sampled) self.assertEqual(record.otelServiceName, "unknown_service") def test_trace_context_injection(self): with self.tracer.start_as_current_span("s1") as span: span_id = format(span.get_span_context().span_id, "016x") trace_id = format(span.get_span_context().trace_id, "032x") - self.assert_trace_context_injected(span_id, trace_id) + trace_sampled = span.get_span_context().trace_flags.sampled + self.assert_trace_context_injected( + span_id, trace_id, trace_sampled + ) def test_trace_context_injection_without_span(self): - self.assert_trace_context_injected("0", "0") + self.assert_trace_context_injected("0", "0", False) @mock.patch("logging.basicConfig") def test_basic_config_called(self, basic_config_mock): @@ -163,6 +168,7 @@ def test_log_hook(self): with self.tracer.start_as_current_span("s1") as span: span_id = format(span.get_span_context().span_id, "016x") trace_id = format(span.get_span_context().trace_id, "032x") + trace_sampled = span.get_span_context().trace_flags.sampled with self.caplog.at_level(level=logging.INFO): logger = logging.getLogger("test logger") logger.info("hello") @@ -171,6 +177,7 @@ def test_log_hook(self): self.assertEqual(record.otelSpanID, span_id) self.assertEqual(record.otelTraceID, trace_id) self.assertEqual(record.otelServiceName, "unknown_service") + self.assertEqual(record.otelTraceSampled, trace_sampled) self.assertEqual( record.custom_user_attribute_from_log_hook, "some-value" ) @@ -179,7 +186,10 @@ def test_uninstrumented(self): with self.tracer.start_as_current_span("s1") as span: span_id = format(span.get_span_context().span_id, "016x") trace_id = format(span.get_span_context().trace_id, "032x") - self.assert_trace_context_injected(span_id, trace_id) + trace_sampled = span.get_span_context().trace_flags.sampled + self.assert_trace_context_injected( + span_id, trace_id, trace_sampled + ) LoggingInstrumentor().uninstrument() @@ -187,6 +197,7 @@ def test_uninstrumented(self): with self.tracer.start_as_current_span("s1") as span: span_id = format(span.get_span_context().span_id, "016x") trace_id = format(span.get_span_context().trace_id, "032x") + trace_sampled = span.get_span_context().trace_flags.sampled with self.caplog.at_level(level=logging.INFO): logger = logging.getLogger("test logger") logger.info("hello") @@ -195,3 +206,4 @@ def test_uninstrumented(self): self.assertFalse(hasattr(record, "otelSpanID")) self.assertFalse(hasattr(record, "otelTraceID")) self.assertFalse(hasattr(record, "otelServiceName")) + self.assertFalse(hasattr(record, "otelTraceSampled")) diff --git a/instrumentation/opentelemetry-instrumentation-mysql/pyproject.toml b/instrumentation/opentelemetry-instrumentation-mysql/pyproject.toml index 7d2b050d78..e155147d5c 100644 --- a/instrumentation/opentelemetry-instrumentation-mysql/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-mysql/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-instrumentation-dbapi == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-instrumentation-dbapi == 0.42b0.dev", ] [project.optional-dependencies] @@ -36,7 +36,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-mysql[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-mysql/src/opentelemetry/instrumentation/mysql/version.py b/instrumentation/opentelemetry-instrumentation-mysql/src/opentelemetry/instrumentation/mysql/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-mysql/src/opentelemetry/instrumentation/mysql/version.py +++ b/instrumentation/opentelemetry-instrumentation-mysql/src/opentelemetry/instrumentation/mysql/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-mysqlclient/LICENSE b/instrumentation/opentelemetry-instrumentation-mysqlclient/LICENSE new file mode 100644 index 0000000000..1ef7dad2c5 --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-mysqlclient/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright The OpenTelemetry Authors + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/instrumentation/opentelemetry-instrumentation-mysqlclient/README.rst b/instrumentation/opentelemetry-instrumentation-mysqlclient/README.rst new file mode 100644 index 0000000000..cce21d8bca --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-mysqlclient/README.rst @@ -0,0 +1,21 @@ +OpenTelemetry mysqlclient Instrumentation +========================================= + +|pypi| + +.. |pypi| image:: https://badge.fury.io/py/opentelemetry-instrumentation-mysqlclient.svg + :target: https://pypi.org/project/opentelemetry-instrumentation-mysqlclient/ + +Installation +------------ + +:: + + pip install opentelemetry-instrumentation-mysqlclient + + +References +---------- +* `OpenTelemetry mysqlclient Instrumentation `_ +* `OpenTelemetry Project `_ +* `OpenTelemetry Python Examples `_ diff --git a/instrumentation/opentelemetry-instrumentation-mysqlclient/pyproject.toml b/instrumentation/opentelemetry-instrumentation-mysqlclient/pyproject.toml new file mode 100644 index 0000000000..72d4bc67cc --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-mysqlclient/pyproject.toml @@ -0,0 +1,58 @@ +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[project] +name = "opentelemetry-instrumentation-mysqlclient" +dynamic = ["version"] +description = "OpenTelemetry mysqlclient instrumentation" +readme = "README.rst" +license = "Apache-2.0" +requires-python = ">=3.7" +authors = [ + { name = "OpenTelemetry Authors", email = "cncf-opentelemetry-contributors@lists.cncf.io" }, +] +classifiers = [ + "Development Status :: 4 - Beta", + "Intended Audience :: Developers", + "License :: OSI Approved :: Apache Software License", + "Programming Language :: Python", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.7", + "Programming Language :: Python :: 3.8", + "Programming Language :: Python :: 3.9", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", +] +dependencies = [ + "opentelemetry-api ~= 1.12", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-instrumentation-dbapi == 0.42b0.dev", +] + +[project.optional-dependencies] +instruments = [ + "mysqlclient < 3", +] +test = [ + "opentelemetry-instrumentation-mysqlclient[instruments]", + "opentelemetry-test-utils == 0.42b0.dev", +] + +[project.entry-points.opentelemetry_instrumentor] +mysqlclient = "opentelemetry.instrumentation.mysqlclient:MySQLClientInstrumentor" + +[project.urls] +Homepage = "https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation/opentelemetry-instrumentation-mysqlclient" + +[tool.hatch.version] +path = "src/opentelemetry/instrumentation/mysqlclient/version.py" + +[tool.hatch.build.targets.sdist] +include = [ + "/src", + "/tests", +] + +[tool.hatch.build.targets.wheel] +packages = ["src/opentelemetry"] diff --git a/instrumentation/opentelemetry-instrumentation-mysqlclient/src/opentelemetry/instrumentation/mysqlclient/__init__.py b/instrumentation/opentelemetry-instrumentation-mysqlclient/src/opentelemetry/instrumentation/mysqlclient/__init__.py new file mode 100644 index 0000000000..85083cff2e --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-mysqlclient/src/opentelemetry/instrumentation/mysqlclient/__init__.py @@ -0,0 +1,117 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""" +The integration with MySQLClient supports the `MySQLClient`_ library and can be enabled +by using ``MySQLClientInstrumentor``. + +.. _MySQLClient: https://pypi.org/project/MySQLClient/ + +Usage +----- + +.. code:: python + + import MySQLdb + from opentelemetry.instrumentation.mysqlclient import MySQLClientInstrumentor + + + MySQLClientInstrumentor().instrument() + + cnx = MySQLdb.connect(database="MySQL_Database") + cursor = cnx.cursor() + cursor.execute("INSERT INTO test (testField) VALUES (123)" + cnx.commit() + cursor.close() + cnx.close() + +API +--- +""" + +from typing import Collection + +import MySQLdb + +from opentelemetry.instrumentation import dbapi +from opentelemetry.instrumentation.instrumentor import BaseInstrumentor +from opentelemetry.instrumentation.mysqlclient.package import _instruments +from opentelemetry.instrumentation.mysqlclient.version import __version__ + +_CONNECTION_ATTRIBUTES = { + "database": "db", + "port": "port", + "host": "host", + "user": "user", +} +_DATABASE_SYSTEM = "mysql" + + +class MySQLClientInstrumentor(BaseInstrumentor): + def instrumentation_dependencies(self) -> Collection[str]: + return _instruments + + def _instrument(self, **kwargs): + """Integrate with the mysqlclient library. + https://github.com/PyMySQL/mysqlclient/ + """ + tracer_provider = kwargs.get("tracer_provider") + + dbapi.wrap_connect( + __name__, + MySQLdb, + "connect", + _DATABASE_SYSTEM, + _CONNECTION_ATTRIBUTES, + version=__version__, + tracer_provider=tracer_provider, + ) + + def _uninstrument(self, **kwargs): + """ "Disable mysqlclient instrumentation""" + dbapi.unwrap_connect(MySQLdb, "connect") + + @staticmethod + def instrument_connection(connection, tracer_provider=None): + """Enable instrumentation in a mysqlclient connection. + + Args: + connection: The connection to instrument. + tracer_provider: The optional tracer provider to use. If omitted + the current globally configured one is used. + + Returns: + An instrumented connection. + """ + + return dbapi.instrument_connection( + __name__, + connection, + _DATABASE_SYSTEM, + _CONNECTION_ATTRIBUTES, + version=__version__, + tracer_provider=tracer_provider, + ) + + @staticmethod + def uninstrument_connection(connection): + """Disable instrumentation in a mysqlclient connection. + + Args: + connection: The connection to uninstrument. + + Returns: + An uninstrumented connection. + """ + return dbapi.uninstrument_connection(connection) diff --git a/instrumentation/opentelemetry-instrumentation-mysqlclient/src/opentelemetry/instrumentation/mysqlclient/package.py b/instrumentation/opentelemetry-instrumentation-mysqlclient/src/opentelemetry/instrumentation/mysqlclient/package.py new file mode 100644 index 0000000000..9469194728 --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-mysqlclient/src/opentelemetry/instrumentation/mysqlclient/package.py @@ -0,0 +1,16 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +_instruments = ("mysqlclient < 3",) diff --git a/instrumentation/opentelemetry-instrumentation-mysqlclient/src/opentelemetry/instrumentation/mysqlclient/version.py b/instrumentation/opentelemetry-instrumentation-mysqlclient/src/opentelemetry/instrumentation/mysqlclient/version.py new file mode 100644 index 0000000000..c2996671d6 --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-mysqlclient/src/opentelemetry/instrumentation/mysqlclient/version.py @@ -0,0 +1,15 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-mysqlclient/tests/__init__.py b/instrumentation/opentelemetry-instrumentation-mysqlclient/tests/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/instrumentation/opentelemetry-instrumentation-mysqlclient/tests/test_mysqlclient_integration.py b/instrumentation/opentelemetry-instrumentation-mysqlclient/tests/test_mysqlclient_integration.py new file mode 100644 index 0000000000..35fdecc8e1 --- /dev/null +++ b/instrumentation/opentelemetry-instrumentation-mysqlclient/tests/test_mysqlclient_integration.py @@ -0,0 +1,118 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from unittest import mock + +import MySQLdb + +import opentelemetry.instrumentation.mysqlclient +from opentelemetry.instrumentation.mysqlclient import MySQLClientInstrumentor +from opentelemetry.sdk import resources +from opentelemetry.test.test_base import TestBase + + +class TestMySQLClientIntegration(TestBase): + def tearDown(self): + super().tearDown() + with self.disable_logging(): + MySQLClientInstrumentor().uninstrument() + + @mock.patch("MySQLdb.connect") + # pylint: disable=unused-argument + def test_instrumentor(self, mock_connect): + MySQLClientInstrumentor().instrument() + + cnx = MySQLdb.connect(database="test") + cursor = cnx.cursor() + query = "SELECT * FROM test" + cursor.execute(query) + + spans_list = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans_list), 1) + span = spans_list[0] + + # Check version and name in span's instrumentation info + self.assertEqualSpanInstrumentationInfo( + span, opentelemetry.instrumentation.mysqlclient + ) + + # check that no spans are generated after uninstrument + MySQLClientInstrumentor().uninstrument() + + cnx = MySQLdb.connect(database="test") + cursor = cnx.cursor() + query = "SELECT * FROM test" + cursor.execute(query) + + spans_list = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans_list), 1) + + @mock.patch("MySQLdb.connect") + # pylint: disable=unused-argument + def test_custom_tracer_provider(self, mock_connect): + resource = resources.Resource.create({}) + result = self.create_tracer_provider(resource=resource) + tracer_provider, exporter = result + + MySQLClientInstrumentor().instrument(tracer_provider=tracer_provider) + + cnx = MySQLdb.connect(database="test") + cursor = cnx.cursor() + query = "SELECT * FROM test" + cursor.execute(query) + + spans_list = exporter.get_finished_spans() + self.assertEqual(len(spans_list), 1) + span = spans_list[0] + + self.assertIs(span.resource, resource) + + @mock.patch("MySQLdb.connect") + # pylint: disable=unused-argument + def test_instrument_connection(self, mock_connect): + cnx = MySQLdb.connect(database="test") + query = "SELECT * FROM test" + cursor = cnx.cursor() + cursor.execute(query) + + spans_list = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans_list), 0) + + cnx = MySQLClientInstrumentor().instrument_connection(cnx) + cursor = cnx.cursor() + cursor.execute(query) + + spans_list = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans_list), 1) + + @mock.patch("MySQLdb.connect") + # pylint: disable=unused-argument + def test_uninstrument_connection(self, mock_connect): + MySQLClientInstrumentor().instrument() + cnx = MySQLdb.connect(database="test") + query = "SELECT * FROM test" + cursor = cnx.cursor() + cursor.execute(query) + + spans_list = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans_list), 1) + + self.memory_exporter.clear() + + cnx = MySQLClientInstrumentor().uninstrument_connection(cnx) + cursor = cnx.cursor() + cursor.execute(query) + + spans_list = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans_list), 0) diff --git a/instrumentation/opentelemetry-instrumentation-pika/pyproject.toml b/instrumentation/opentelemetry-instrumentation-pika/pyproject.toml index f53b18a083..8ee033e58d 100644 --- a/instrumentation/opentelemetry-instrumentation-pika/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-pika/pyproject.toml @@ -36,7 +36,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-pika[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", "pytest", "wrapt >= 1.0.0, < 2.0.0", ] diff --git a/instrumentation/opentelemetry-instrumentation-pika/src/opentelemetry/instrumentation/pika/version.py b/instrumentation/opentelemetry-instrumentation-pika/src/opentelemetry/instrumentation/pika/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-pika/src/opentelemetry/instrumentation/pika/version.py +++ b/instrumentation/opentelemetry-instrumentation-pika/src/opentelemetry/instrumentation/pika/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-psycopg2/pyproject.toml b/instrumentation/opentelemetry-instrumentation-psycopg2/pyproject.toml index 702f7b3c84..d709c2bad1 100644 --- a/instrumentation/opentelemetry-instrumentation-psycopg2/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-psycopg2/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-instrumentation-dbapi == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-instrumentation-dbapi == 0.42b0.dev", ] [project.optional-dependencies] @@ -36,7 +36,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-psycopg2[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-psycopg2/src/opentelemetry/instrumentation/psycopg2/version.py b/instrumentation/opentelemetry-instrumentation-psycopg2/src/opentelemetry/instrumentation/psycopg2/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-psycopg2/src/opentelemetry/instrumentation/psycopg2/version.py +++ b/instrumentation/opentelemetry-instrumentation-psycopg2/src/opentelemetry/instrumentation/psycopg2/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-pymemcache/pyproject.toml b/instrumentation/opentelemetry-instrumentation-pymemcache/pyproject.toml index 55a2ef077d..f6b61bd4d0 100644 --- a/instrumentation/opentelemetry-instrumentation-pymemcache/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-pymemcache/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", "wrapt >= 1.0.0, < 2.0.0", ] @@ -37,7 +37,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-pymemcache[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-pymemcache/src/opentelemetry/instrumentation/pymemcache/version.py b/instrumentation/opentelemetry-instrumentation-pymemcache/src/opentelemetry/instrumentation/pymemcache/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-pymemcache/src/opentelemetry/instrumentation/pymemcache/version.py +++ b/instrumentation/opentelemetry-instrumentation-pymemcache/src/opentelemetry/instrumentation/pymemcache/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-pymongo/pyproject.toml b/instrumentation/opentelemetry-instrumentation-pymongo/pyproject.toml index dbb7f64a46..2c40ed26bb 100644 --- a/instrumentation/opentelemetry-instrumentation-pymongo/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-pymongo/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", ] [project.optional-dependencies] @@ -36,7 +36,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-pymongo[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-pymongo/src/opentelemetry/instrumentation/pymongo/version.py b/instrumentation/opentelemetry-instrumentation-pymongo/src/opentelemetry/instrumentation/pymongo/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-pymongo/src/opentelemetry/instrumentation/pymongo/version.py +++ b/instrumentation/opentelemetry-instrumentation-pymongo/src/opentelemetry/instrumentation/pymongo/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-pymysql/pyproject.toml b/instrumentation/opentelemetry-instrumentation-pymysql/pyproject.toml index 83bf9052cf..62404ed4f1 100644 --- a/instrumentation/opentelemetry-instrumentation-pymysql/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-pymysql/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-instrumentation-dbapi == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-instrumentation-dbapi == 0.42b0.dev", ] [project.optional-dependencies] @@ -36,7 +36,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-pymysql[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-pymysql/src/opentelemetry/instrumentation/pymysql/version.py b/instrumentation/opentelemetry-instrumentation-pymysql/src/opentelemetry/instrumentation/pymysql/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-pymysql/src/opentelemetry/instrumentation/pymysql/version.py +++ b/instrumentation/opentelemetry-instrumentation-pymysql/src/opentelemetry/instrumentation/pymysql/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-pyramid/pyproject.toml b/instrumentation/opentelemetry-instrumentation-pyramid/pyproject.toml index 4148e45a7e..59483c972b 100644 --- a/instrumentation/opentelemetry-instrumentation-pyramid/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-pyramid/pyproject.toml @@ -26,10 +26,10 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-instrumentation-wsgi == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", - "opentelemetry-util-http == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-instrumentation-wsgi == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", + "opentelemetry-util-http == 0.42b0.dev", "wrapt >= 1.0.0, < 2.0.0", ] @@ -39,7 +39,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-pyramid[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", "werkzeug == 0.16.1", ] diff --git a/instrumentation/opentelemetry-instrumentation-pyramid/src/opentelemetry/instrumentation/pyramid/version.py b/instrumentation/opentelemetry-instrumentation-pyramid/src/opentelemetry/instrumentation/pyramid/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-pyramid/src/opentelemetry/instrumentation/pyramid/version.py +++ b/instrumentation/opentelemetry-instrumentation-pyramid/src/opentelemetry/instrumentation/pyramid/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-pyramid/tests/pyramid_base_test.py b/instrumentation/opentelemetry-instrumentation-pyramid/tests/pyramid_base_test.py index f5dd9fd7d7..c6b9faa196 100644 --- a/instrumentation/opentelemetry-instrumentation-pyramid/tests/pyramid_base_test.py +++ b/instrumentation/opentelemetry-instrumentation-pyramid/tests/pyramid_base_test.py @@ -15,7 +15,11 @@ import pyramid.httpexceptions as exc from pyramid.response import Response from werkzeug.test import Client -from werkzeug.wrappers import BaseResponse + +# opentelemetry-instrumentation-pyramid uses werkzeug==0.16.1 which has +# werkzeug.wrappers.BaseResponse. This is not the case for newer versions of +# werkzeug like the one lint uses. +from werkzeug.wrappers import BaseResponse # pylint: disable=no-name-in-module class InstrumentationTest: diff --git a/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_programmatic.py b/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_programmatic.py index d3a4fa91db..478eab1937 100644 --- a/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_programmatic.py +++ b/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_programmatic.py @@ -145,7 +145,7 @@ def test_404(self): resp.close() span_list = self.memory_exporter.get_finished_spans() self.assertEqual(len(span_list), 1) - self.assertEqual(span_list[0].name, "HTTP POST") + self.assertEqual(span_list[0].name, "POST /bye") self.assertEqual(span_list[0].kind, trace.SpanKind.SERVER) self.assertEqual(span_list[0].attributes, expected_attrs) diff --git a/instrumentation/opentelemetry-instrumentation-redis/pyproject.toml b/instrumentation/opentelemetry-instrumentation-redis/pyproject.toml index ed0ebaf149..3ae0609147 100644 --- a/instrumentation/opentelemetry-instrumentation-redis/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-redis/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", "wrapt >= 1.12.1", ] @@ -38,7 +38,7 @@ instruments = [ test = [ "opentelemetry-instrumentation-redis[instruments]", "opentelemetry-sdk ~= 1.3", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/__init__.py b/instrumentation/opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/__init__.py index c1068bda27..ba4b8d529e 100644 --- a/instrumentation/opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/__init__.py @@ -16,7 +16,7 @@ Instrument `redis`_ to report Redis queries. There are two options for instrumenting code. The first option is to use the -``opentelemetry-instrumentation`` executable which will automatically +``opentelemetry-instrument`` executable which will automatically instrument your Redis client. The second is to programmatically enable instrumentation via the following code: @@ -64,8 +64,6 @@ async def redis_get(): response_hook (Callable) - a function with extra user-defined logic to be performed after performing the request this function signature is: def response_hook(span: Span, instance: redis.connection.Connection, response) -> None -sanitize_query (Boolean) - default False, enable the Redis query sanitization - for example: .. code: python @@ -88,27 +86,11 @@ def response_hook(span, instance, response): client = redis.StrictRedis(host="localhost", port=6379) client.get("my-key") -Configuration -------------- - -Query sanitization -****************** -To enable query sanitization with an environment variable, set -``OTEL_PYTHON_INSTRUMENTATION_SANITIZE_REDIS`` to "true". - -For example, - -:: - - export OTEL_PYTHON_INSTRUMENTATION_SANITIZE_REDIS="true" - -will result in traced queries like "SET ? ?". API --- """ import typing -from os import environ from typing import Any, Collection import redis @@ -116,9 +98,6 @@ def response_hook(span, instance, response): from opentelemetry import trace from opentelemetry.instrumentation.instrumentor import BaseInstrumentor -from opentelemetry.instrumentation.redis.environment_variables import ( - OTEL_PYTHON_INSTRUMENTATION_SANITIZE_REDIS, -) from opentelemetry.instrumentation.redis.package import _instruments from opentelemetry.instrumentation.redis.util import ( _extract_conn_attributes, @@ -157,19 +136,52 @@ def _set_connection_attributes(span, conn): span.set_attribute(key, value) +def _build_span_name(instance, cmd_args): + if len(cmd_args) > 0 and cmd_args[0]: + name = cmd_args[0] + else: + name = instance.connection_pool.connection_kwargs.get("db", 0) + return name + + +def _build_span_meta_data_for_pipeline(instance): + try: + command_stack = ( + instance.command_stack + if hasattr(instance, "command_stack") + else instance._command_stack + ) + + cmds = [ + _format_command_args(c.args if hasattr(c, "args") else c[0]) + for c in command_stack + ] + resource = "\n".join(cmds) + + span_name = " ".join( + [ + (c.args[0] if hasattr(c, "args") else c[0][0]) + for c in command_stack + ] + ) + except (AttributeError, IndexError): + command_stack = [] + resource = "" + span_name = "" + + return command_stack, resource, span_name + + +# pylint: disable=R0915 def _instrument( tracer, request_hook: _RequestHookT = None, response_hook: _ResponseHookT = None, - sanitize_query: bool = False, ): def _traced_execute_command(func, instance, args, kwargs): - query = _format_command_args(args, sanitize_query) + query = _format_command_args(args) + name = _build_span_name(instance, args) - if len(args) > 0 and args[0]: - name = args[0] - else: - name = instance.connection_pool.connection_kwargs.get("db", 0) with tracer.start_as_current_span( name, kind=trace.SpanKind.CLIENT ) as span: @@ -185,31 +197,11 @@ def _traced_execute_command(func, instance, args, kwargs): return response def _traced_execute_pipeline(func, instance, args, kwargs): - try: - command_stack = ( - instance.command_stack - if hasattr(instance, "command_stack") - else instance._command_stack - ) - - cmds = [ - _format_command_args( - c.args if hasattr(c, "args") else c[0], sanitize_query - ) - for c in command_stack - ] - resource = "\n".join(cmds) - - span_name = " ".join( - [ - (c.args[0] if hasattr(c, "args") else c[0][0]) - for c in command_stack - ] - ) - except (AttributeError, IndexError): - command_stack = [] - resource = "" - span_name = "" + ( + command_stack, + resource, + span_name, + ) = _build_span_meta_data_for_pipeline(instance) with tracer.start_as_current_span( span_name, kind=trace.SpanKind.CLIENT @@ -254,32 +246,72 @@ def _traced_execute_pipeline(func, instance, args, kwargs): "ClusterPipeline.execute", _traced_execute_pipeline, ) + + async def _async_traced_execute_command(func, instance, args, kwargs): + query = _format_command_args(args) + name = _build_span_name(instance, args) + + with tracer.start_as_current_span( + name, kind=trace.SpanKind.CLIENT + ) as span: + if span.is_recording(): + span.set_attribute(SpanAttributes.DB_STATEMENT, query) + _set_connection_attributes(span, instance) + span.set_attribute("db.redis.args_length", len(args)) + if callable(request_hook): + request_hook(span, instance, args, kwargs) + response = await func(*args, **kwargs) + if callable(response_hook): + response_hook(span, instance, response) + return response + + async def _async_traced_execute_pipeline(func, instance, args, kwargs): + ( + command_stack, + resource, + span_name, + ) = _build_span_meta_data_for_pipeline(instance) + + with tracer.start_as_current_span( + span_name, kind=trace.SpanKind.CLIENT + ) as span: + if span.is_recording(): + span.set_attribute(SpanAttributes.DB_STATEMENT, resource) + _set_connection_attributes(span, instance) + span.set_attribute( + "db.redis.pipeline_length", len(command_stack) + ) + response = await func(*args, **kwargs) + if callable(response_hook): + response_hook(span, instance, response) + return response + if redis.VERSION >= _REDIS_ASYNCIO_VERSION: wrap_function_wrapper( "redis.asyncio", f"{redis_class}.execute_command", - _traced_execute_command, + _async_traced_execute_command, ) wrap_function_wrapper( "redis.asyncio.client", f"{pipeline_class}.execute", - _traced_execute_pipeline, + _async_traced_execute_pipeline, ) wrap_function_wrapper( "redis.asyncio.client", f"{pipeline_class}.immediate_execute_command", - _traced_execute_command, + _async_traced_execute_command, ) if redis.VERSION >= _REDIS_ASYNCIO_CLUSTER_VERSION: wrap_function_wrapper( "redis.asyncio.cluster", "RedisCluster.execute_command", - _traced_execute_command, + _async_traced_execute_command, ) wrap_function_wrapper( "redis.asyncio.cluster", "ClusterPipeline.execute", - _traced_execute_pipeline, + _async_traced_execute_pipeline, ) @@ -307,15 +339,6 @@ def _instrument(self, **kwargs): tracer, request_hook=kwargs.get("request_hook"), response_hook=kwargs.get("response_hook"), - sanitize_query=kwargs.get( - "sanitize_query", - environ.get( - OTEL_PYTHON_INSTRUMENTATION_SANITIZE_REDIS, "false" - ) - .lower() - .strip() - == "true", - ), ) def _uninstrument(self, **kwargs): diff --git a/instrumentation/opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/util.py b/instrumentation/opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/util.py index 1eadaba718..b24f9b2655 100644 --- a/instrumentation/opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/util.py +++ b/instrumentation/opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/util.py @@ -48,41 +48,23 @@ def _extract_conn_attributes(conn_kwargs): return attributes -def _format_command_args(args, sanitize_query): +def _format_command_args(args): """Format and sanitize command arguments, and trim them as needed""" cmd_max_len = 1000 value_too_long_mark = "..." - if sanitize_query: - # Sanitized query format: "COMMAND ? ?" - args_length = len(args) - if args_length > 0: - out = [str(args[0])] + ["?"] * (args_length - 1) - out_str = " ".join(out) - if len(out_str) > cmd_max_len: - out_str = ( - out_str[: cmd_max_len - len(value_too_long_mark)] - + value_too_long_mark - ) - else: - out_str = "" - return out_str + # Sanitized query format: "COMMAND ? ?" + args_length = len(args) + if args_length > 0: + out = [str(args[0])] + ["?"] * (args_length - 1) + out_str = " ".join(out) - value_max_len = 100 - length = 0 - out = [] - for arg in args: - cmd = str(arg) + if len(out_str) > cmd_max_len: + out_str = ( + out_str[: cmd_max_len - len(value_too_long_mark)] + + value_too_long_mark + ) + else: + out_str = "" - if len(cmd) > value_max_len: - cmd = cmd[:value_max_len] + value_too_long_mark - - if length + len(cmd) > cmd_max_len: - prefix = cmd[: cmd_max_len - length] - out.append(f"{prefix}{value_too_long_mark}") - break - - out.append(cmd) - length += len(cmd) - - return " ".join(out) + return out_str diff --git a/instrumentation/opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/version.py b/instrumentation/opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/version.py +++ b/instrumentation/opentelemetry-instrumentation-redis/src/opentelemetry/instrumentation/redis/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-redis/tests/test_redis.py b/instrumentation/opentelemetry-instrumentation-redis/tests/test_redis.py index 56a0df6a0a..11e56ad953 100644 --- a/instrumentation/opentelemetry-instrumentation-redis/tests/test_redis.py +++ b/instrumentation/opentelemetry-instrumentation-redis/tests/test_redis.py @@ -11,9 +11,11 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. +import asyncio from unittest import mock import redis +import redis.asyncio from opentelemetry import trace from opentelemetry.instrumentation.redis import RedisInstrumentor @@ -21,6 +23,24 @@ from opentelemetry.trace import SpanKind +class AsyncMock: + """A sufficient async mock implementation. + + Python 3.7 doesn't have an inbuilt async mock class, so this is used. + """ + + def __init__(self): + self.mock = mock.Mock() + + async def __call__(self, *args, **kwargs): + future = asyncio.Future() + future.set_result("random") + return future + + def __getattr__(self, item): + return AsyncMock() + + class TestRedis(TestBase): def setUp(self): super().setUp() @@ -87,6 +107,35 @@ def test_instrument_uninstrument(self): spans = self.memory_exporter.get_finished_spans() self.assertEqual(len(spans), 1) + def test_instrument_uninstrument_async_client_command(self): + redis_client = redis.asyncio.Redis() + + with mock.patch.object(redis_client, "connection", AsyncMock()): + asyncio.run(redis_client.get("key")) + + spans = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans), 1) + self.memory_exporter.clear() + + # Test uninstrument + RedisInstrumentor().uninstrument() + + with mock.patch.object(redis_client, "connection", AsyncMock()): + asyncio.run(redis_client.get("key")) + + spans = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans), 0) + self.memory_exporter.clear() + + # Test instrument again + RedisInstrumentor().instrument() + + with mock.patch.object(redis_client, "connection", AsyncMock()): + asyncio.run(redis_client.get("key")) + + spans = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans), 1) + def test_response_hook(self): redis_client = redis.Redis() connection = redis.connection.Connection() @@ -168,22 +217,11 @@ def test_query_sanitizer_enabled(self): span = spans[0] self.assertEqual(span.attributes.get("db.statement"), "SET ? ?") - def test_query_sanitizer_enabled_env(self): + def test_query_sanitizer(self): redis_client = redis.Redis() connection = redis.connection.Connection() redis_client.connection = connection - RedisInstrumentor().uninstrument() - - env_patch = mock.patch.dict( - "os.environ", - {"OTEL_PYTHON_INSTRUMENTATION_SANITIZE_REDIS": "true"}, - ) - env_patch.start() - RedisInstrumentor().instrument( - tracer_provider=self.tracer_provider, - ) - with mock.patch.object(redis_client, "connection"): redis_client.set("key", "value") @@ -192,21 +230,6 @@ def test_query_sanitizer_enabled_env(self): span = spans[0] self.assertEqual(span.attributes.get("db.statement"), "SET ? ?") - env_patch.stop() - - def test_query_sanitizer_disabled(self): - redis_client = redis.Redis() - connection = redis.connection.Connection() - redis_client.connection = connection - - with mock.patch.object(redis_client, "connection"): - redis_client.set("key", "value") - - spans = self.memory_exporter.get_finished_spans() - self.assertEqual(len(spans), 1) - - span = spans[0] - self.assertEqual(span.attributes.get("db.statement"), "SET key value") def test_no_op_tracer_provider(self): RedisInstrumentor().uninstrument() diff --git a/instrumentation/opentelemetry-instrumentation-remoulade/pyproject.toml b/instrumentation/opentelemetry-instrumentation-remoulade/pyproject.toml index 72c127733a..2a3a596700 100644 --- a/instrumentation/opentelemetry-instrumentation-remoulade/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-remoulade/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", ] [project.optional-dependencies] @@ -36,7 +36,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-remoulade[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", "opentelemetry-sdk ~= 1.10" ] diff --git a/instrumentation/opentelemetry-instrumentation-remoulade/src/opentelemetry/instrumentation/remoulade/version.py b/instrumentation/opentelemetry-instrumentation-remoulade/src/opentelemetry/instrumentation/remoulade/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-remoulade/src/opentelemetry/instrumentation/remoulade/version.py +++ b/instrumentation/opentelemetry-instrumentation-remoulade/src/opentelemetry/instrumentation/remoulade/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-requests/pyproject.toml b/instrumentation/opentelemetry-instrumentation-requests/pyproject.toml index 7990017918..184bd1ca7e 100644 --- a/instrumentation/opentelemetry-instrumentation-requests/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-requests/pyproject.toml @@ -26,9 +26,9 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", - "opentelemetry-util-http == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", + "opentelemetry-util-http == 0.42b0.dev", ] [project.optional-dependencies] @@ -38,7 +38,7 @@ instruments = [ test = [ "opentelemetry-instrumentation-requests[instruments]", "httpretty ~= 1.0", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/__init__.py b/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/__init__.py index e5bb24223c..c3dabf05a5 100644 --- a/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/__init__.py @@ -245,8 +245,16 @@ def _uninstrument_from(instr_root, restore_as_bound_func=False): def get_default_span_name(method): - """Default implementation for name_callback, returns HTTP {method_name}.""" - return f"HTTP {method.strip()}" + """ + Default implementation for name_callback, returns HTTP {method_name}. + https://opentelemetry.io/docs/reference/specification/trace/semantic_conventions/http/#name + + Args: + method: string representing HTTP method + Returns: + span name + """ + return method.strip() class RequestsInstrumentor(BaseInstrumentor): diff --git a/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/version.py b/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/version.py +++ b/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-requests/tests/test_requests_integration.py b/instrumentation/opentelemetry-instrumentation-requests/tests/test_requests_integration.py index 8d6ee7c04d..3bd76a6995 100644 --- a/instrumentation/opentelemetry-instrumentation-requests/tests/test_requests_integration.py +++ b/instrumentation/opentelemetry-instrumentation-requests/tests/test_requests_integration.py @@ -63,7 +63,7 @@ class RequestsIntegrationTestBase(abc.ABC): # pylint: disable=no-member # pylint: disable=too-many-public-methods - URL = "http://httpbin.org/status/200" + URL = "http://mock/status/200" # pylint: disable=invalid-name def setUp(self): @@ -116,7 +116,7 @@ def test_basic(self): span = self.assert_span() self.assertIs(span.kind, trace.SpanKind.CLIENT) - self.assertEqual(span.name, "HTTP GET") + self.assertEqual(span.name, "GET") self.assertEqual( span.attributes, @@ -152,7 +152,7 @@ def response_hook(span, request_obj, response): self.assertEqual(span.attributes["response_hook_attr"], "value") def test_excluded_urls_explicit(self): - url_404 = "http://httpbin.org/status/404" + url_404 = "http://mock/status/404" httpretty.register_uri( httpretty.GET, url_404, @@ -191,10 +191,10 @@ def name_callback(method, url): self.assertEqual(result.text, "Hello!") span = self.assert_span() - self.assertEqual(span.name, "HTTP GET") + self.assertEqual(span.name, "GET") def test_not_foundbasic(self): - url_404 = "http://httpbin.org/status/404" + url_404 = "http://mock/status/404" httpretty.register_uri( httpretty.GET, url_404, @@ -460,7 +460,7 @@ def perform_request(url: str, session: requests.Session = None): return session.get(url) def test_credential_removal(self): - new_url = "http://username:password@httpbin.org/status/200" + new_url = "http://username:password@mock/status/200" self.perform_request(new_url) span = self.assert_span() diff --git a/instrumentation/opentelemetry-instrumentation-requests/tests/test_requests_ip_support.py b/instrumentation/opentelemetry-instrumentation-requests/tests/test_requests_ip_support.py index 593ed92fe9..cf2e7fb4dd 100644 --- a/instrumentation/opentelemetry-instrumentation-requests/tests/test_requests_ip_support.py +++ b/instrumentation/opentelemetry-instrumentation-requests/tests/test_requests_ip_support.py @@ -71,7 +71,7 @@ def assert_success_span(self, response: requests.Response): span = self.assert_span() self.assertIs(trace.SpanKind.CLIENT, span.kind) - self.assertEqual("HTTP GET", span.name) + self.assertEqual("GET", span.name) attributes = { "http.status_code": 200, diff --git a/instrumentation/opentelemetry-instrumentation-sklearn/pyproject.toml b/instrumentation/opentelemetry-instrumentation-sklearn/pyproject.toml index c5d16f1239..85a145f662 100644 --- a/instrumentation/opentelemetry-instrumentation-sklearn/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-sklearn/pyproject.toml @@ -26,7 +26,7 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", ] [project.optional-dependencies] @@ -35,7 +35,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-sklearn[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-sklearn/src/opentelemetry/instrumentation/sklearn/version.py b/instrumentation/opentelemetry-instrumentation-sklearn/src/opentelemetry/instrumentation/sklearn/version.py index 88ab55ca31..e9ca2a1777 100644 --- a/instrumentation/opentelemetry-instrumentation-sklearn/src/opentelemetry/instrumentation/sklearn/version.py +++ b/instrumentation/opentelemetry-instrumentation-sklearn/src/opentelemetry/instrumentation/sklearn/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-sqlalchemy/pyproject.toml b/instrumentation/opentelemetry-instrumentation-sqlalchemy/pyproject.toml index beeba209e9..3b96607b41 100644 --- a/instrumentation/opentelemetry-instrumentation-sqlalchemy/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-sqlalchemy/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", "packaging >= 21.0", "wrapt >= 1.11.2", ] diff --git a/instrumentation/opentelemetry-instrumentation-sqlalchemy/src/opentelemetry/instrumentation/sqlalchemy/__init__.py b/instrumentation/opentelemetry-instrumentation-sqlalchemy/src/opentelemetry/instrumentation/sqlalchemy/__init__.py index 77db23b417..e14ac9600c 100644 --- a/instrumentation/opentelemetry-instrumentation-sqlalchemy/src/opentelemetry/instrumentation/sqlalchemy/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-sqlalchemy/src/opentelemetry/instrumentation/sqlalchemy/__init__.py @@ -134,6 +134,9 @@ def _instrument(self, **kwargs): ``engine``: a SQLAlchemy engine instance ``engines``: a list of SQLAlchemy engine instances ``tracer_provider``: a TracerProvider, defaults to global + ``meter_provider``: a MeterProvider, defaults to global + ``enable_commenter``: bool to enable sqlcommenter, defaults to False + ``commenter_options``: dict of sqlcommenter config, defaults to {} Returns: An instrumented engine if passed in as an argument or list of instrumented engines, None otherwise. @@ -151,16 +154,21 @@ def _instrument(self, **kwargs): ) enable_commenter = kwargs.get("enable_commenter", False) + commenter_options = kwargs.get("commenter_options", {}) _w( "sqlalchemy", "create_engine", - _wrap_create_engine(tracer, connections_usage, enable_commenter), + _wrap_create_engine( + tracer, connections_usage, enable_commenter, commenter_options + ), ) _w( "sqlalchemy.engine", "create_engine", - _wrap_create_engine(tracer, connections_usage, enable_commenter), + _wrap_create_engine( + tracer, connections_usage, enable_commenter, commenter_options + ), ) _w( "sqlalchemy.engine.base", @@ -172,7 +180,10 @@ def _instrument(self, **kwargs): "sqlalchemy.ext.asyncio", "create_async_engine", _wrap_create_async_engine( - tracer, connections_usage, enable_commenter + tracer, + connections_usage, + enable_commenter, + commenter_options, ), ) if kwargs.get("engine") is not None: diff --git a/instrumentation/opentelemetry-instrumentation-sqlalchemy/src/opentelemetry/instrumentation/sqlalchemy/engine.py b/instrumentation/opentelemetry-instrumentation-sqlalchemy/src/opentelemetry/instrumentation/sqlalchemy/engine.py index ca691fc052..1cf980929b 100644 --- a/instrumentation/opentelemetry-instrumentation-sqlalchemy/src/opentelemetry/instrumentation/sqlalchemy/engine.py +++ b/instrumentation/opentelemetry-instrumentation-sqlalchemy/src/opentelemetry/instrumentation/sqlalchemy/engine.py @@ -13,6 +13,7 @@ # limitations under the License. import os import re +import weakref from sqlalchemy.event import ( # pylint: disable=no-name-in-module listen, @@ -42,7 +43,7 @@ def _normalize_vendor(vendor): def _wrap_create_async_engine( - tracer, connections_usage, enable_commenter=False + tracer, connections_usage, enable_commenter=False, commenter_options=None ): # pylint: disable=unused-argument def _wrap_create_async_engine_internal(func, module, args, kwargs): @@ -51,20 +52,32 @@ def _wrap_create_async_engine_internal(func, module, args, kwargs): """ engine = func(*args, **kwargs) EngineTracer( - tracer, engine.sync_engine, connections_usage, enable_commenter + tracer, + engine.sync_engine, + connections_usage, + enable_commenter, + commenter_options, ) return engine return _wrap_create_async_engine_internal -def _wrap_create_engine(tracer, connections_usage, enable_commenter=False): +def _wrap_create_engine( + tracer, connections_usage, enable_commenter=False, commenter_options=None +): def _wrap_create_engine_internal(func, _module, args, kwargs): """Trace the SQLAlchemy engine, creating an `EngineTracer` object that will listen to SQLAlchemy events. """ engine = func(*args, **kwargs) - EngineTracer(tracer, engine, connections_usage, enable_commenter) + EngineTracer( + tracer, + engine, + connections_usage, + enable_commenter, + commenter_options, + ) return engine return _wrap_create_engine_internal @@ -99,11 +112,11 @@ def __init__( commenter_options=None, ): self.tracer = tracer - self.engine = engine self.connections_usage = connections_usage self.vendor = _normalize_vendor(engine.name) self.enable_commenter = enable_commenter self.commenter_options = commenter_options if commenter_options else {} + self._engine_attrs = _get_attributes_from_engine(engine) self._leading_comment_remover = re.compile(r"^/\*.*?\*/") self._register_event_listener( @@ -118,14 +131,11 @@ def __init__( self._register_event_listener(engine, "checkin", self._pool_checkin) self._register_event_listener(engine, "checkout", self._pool_checkout) - def _get_pool_name(self): - return self.engine.pool.logging_name or "" - def _add_idle_to_connection_usage(self, value): self.connections_usage.add( value, attributes={ - "pool.name": self._get_pool_name(), + **self._engine_attrs, "state": "idle", }, ) @@ -134,7 +144,7 @@ def _add_used_to_connection_usage(self, value): self.connections_usage.add( value, attributes={ - "pool.name": self._get_pool_name(), + **self._engine_attrs, "state": "used", }, ) @@ -160,12 +170,21 @@ def _pool_checkout( @classmethod def _register_event_listener(cls, target, identifier, func, *args, **kw): listen(target, identifier, func, *args, **kw) - cls._remove_event_listener_params.append((target, identifier, func)) + cls._remove_event_listener_params.append( + (weakref.ref(target), identifier, func) + ) @classmethod def remove_all_event_listeners(cls): - for remove_params in cls._remove_event_listener_params: - remove(*remove_params) + for ( + weak_ref_target, + identifier, + func, + ) in cls._remove_event_listener_params: + # Remove an event listener only if saved weak reference points to an object + # which has not been garbage collected + if weak_ref_target() is not None: + remove(weak_ref_target(), identifier, func) cls._remove_event_listener_params.clear() def _operation_name(self, db_name, statement): @@ -291,3 +310,22 @@ def _get_attributes_from_cursor(vendor, cursor, attrs): if info.port: attrs[SpanAttributes.NET_PEER_PORT] = int(info.port) return attrs + + +def _get_connection_string(engine): + drivername = engine.url.drivername or "" + host = engine.url.host or "" + port = engine.url.port or "" + database = engine.url.database or "" + return f"{drivername}://{host}:{port}/{database}" + + +def _get_attributes_from_engine(engine): + """Set metadata attributes of the database engine""" + attrs = {} + + attrs["pool.name"] = getattr( + getattr(engine, "pool", None), "logging_name", None + ) or _get_connection_string(engine) + + return attrs diff --git a/instrumentation/opentelemetry-instrumentation-sqlalchemy/src/opentelemetry/instrumentation/sqlalchemy/version.py b/instrumentation/opentelemetry-instrumentation-sqlalchemy/src/opentelemetry/instrumentation/sqlalchemy/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-sqlalchemy/src/opentelemetry/instrumentation/sqlalchemy/version.py +++ b/instrumentation/opentelemetry-instrumentation-sqlalchemy/src/opentelemetry/instrumentation/sqlalchemy/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlalchemy.py b/instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlalchemy.py index 981da107db..f729fa6d80 100644 --- a/instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlalchemy.py +++ b/instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlalchemy.py @@ -12,6 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. import asyncio +import logging from unittest import mock import pytest @@ -176,6 +177,43 @@ def test_create_engine_wrapper(self): "opentelemetry.instrumentation.sqlalchemy", ) + def test_create_engine_wrapper_enable_commenter(self): + logging.getLogger("sqlalchemy.engine").setLevel(logging.INFO) + SQLAlchemyInstrumentor().instrument( + enable_commenter=True, + commenter_options={"db_framework": False}, + ) + from sqlalchemy import create_engine # pylint: disable-all + + engine = create_engine("sqlite:///:memory:") + cnx = engine.connect() + cnx.execute("SELECT 1;").fetchall() + # sqlcommenter + self.assertRegex( + self.caplog.records[-2].getMessage(), + r"SELECT 1 /\*db_driver='(.*)',traceparent='\d{1,2}-[a-zA-Z0-9_]{32}-[a-zA-Z0-9_]{16}-\d{1,2}'\*/;", + ) + + def test_create_engine_wrapper_enable_commenter_otel_values_false(self): + logging.getLogger("sqlalchemy.engine").setLevel(logging.INFO) + SQLAlchemyInstrumentor().instrument( + enable_commenter=True, + commenter_options={ + "db_framework": False, + "opentelemetry_values": False, + }, + ) + from sqlalchemy import create_engine # pylint: disable-all + + engine = create_engine("sqlite:///:memory:") + cnx = engine.connect() + cnx.execute("SELECT 1;").fetchall() + # sqlcommenter + self.assertRegex( + self.caplog.records[-2].getMessage(), + r"SELECT 1 /\*db_driver='(.*)'\*/;", + ) + def test_custom_tracer_provider(self): provider = TracerProvider( resource=Resource.create( @@ -242,6 +280,65 @@ async def run(): asyncio.get_event_loop().run_until_complete(run()) + @pytest.mark.skipif( + not sqlalchemy.__version__.startswith("1.4"), + reason="only run async tests for 1.4", + ) + def test_create_async_engine_wrapper_enable_commenter(self): + async def run(): + logging.getLogger("sqlalchemy.engine").setLevel(logging.INFO) + SQLAlchemyInstrumentor().instrument( + enable_commenter=True, + commenter_options={ + "db_framework": False, + }, + ) + from sqlalchemy.ext.asyncio import ( # pylint: disable-all + create_async_engine, + ) + + engine = create_async_engine("sqlite+aiosqlite:///:memory:") + async with engine.connect() as cnx: + await cnx.execute(sqlalchemy.text("SELECT 1;")) + # sqlcommenter + self.assertRegex( + self.caplog.records[1].getMessage(), + r"SELECT 1 /\*db_driver='(.*)',traceparent='\d{1,2}-[a-zA-Z0-9_]{32}-[a-zA-Z0-9_]{16}-\d{1,2}'\*/;", + ) + + asyncio.get_event_loop().run_until_complete(run()) + + @pytest.mark.skipif( + not sqlalchemy.__version__.startswith("1.4"), + reason="only run async tests for 1.4", + ) + def test_create_async_engine_wrapper_enable_commenter_otel_values_false( + self, + ): + async def run(): + logging.getLogger("sqlalchemy.engine").setLevel(logging.INFO) + SQLAlchemyInstrumentor().instrument( + enable_commenter=True, + commenter_options={ + "db_framework": False, + "opentelemetry_values": False, + }, + ) + from sqlalchemy.ext.asyncio import ( # pylint: disable-all + create_async_engine, + ) + + engine = create_async_engine("sqlite+aiosqlite:///:memory:") + async with engine.connect() as cnx: + await cnx.execute(sqlalchemy.text("SELECT 1;")) + # sqlcommenter + self.assertRegex( + self.caplog.records[1].getMessage(), + r"SELECT 1 /\*db_driver='(.*)'\*/;", + ) + + asyncio.get_event_loop().run_until_complete(run()) + def test_uninstrument(self): engine = create_engine("sqlite:///:memory:") SQLAlchemyInstrumentor().instrument( @@ -307,3 +404,26 @@ def test_no_op_tracer_provider(self): cnx.execute("SELECT 1 + 1;").fetchall() spans = self.memory_exporter.get_finished_spans() self.assertEqual(len(spans), 0) + + def test_no_memory_leakage_if_engine_diposed(self): + SQLAlchemyInstrumentor().instrument() + import gc + import weakref + + from sqlalchemy import create_engine + + callback = mock.Mock() + + def make_shortlived_engine(): + engine = create_engine("sqlite:///:memory:") + # Callback will be called if engine is deallocated during garbage + # collection + weakref.finalize(engine, callback) + with engine.connect() as conn: + conn.execute("SELECT 1 + 1;").fetchall() + + for _ in range(0, 5): + make_shortlived_engine() + + gc.collect() + assert callback.call_count == 5 diff --git a/instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlalchemy_metrics.py b/instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlalchemy_metrics.py index 39e45945f7..2d753c3c42 100644 --- a/instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlalchemy_metrics.py +++ b/instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlalchemy_metrics.py @@ -70,11 +70,12 @@ def test_metrics_one_connection(self): ) def test_metrics_without_pool_name(self): - pool_name = "" + pool_name = "pool_test_name" engine = sqlalchemy.create_engine( "sqlite:///:memory:", pool_size=5, poolclass=QueuePool, + pool_logging_name=pool_name, ) metrics = self.get_sorted_metrics() diff --git a/instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlcommenter.py b/instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlcommenter.py index 5f9e75a1aa..f13c552bf4 100644 --- a/instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlcommenter.py +++ b/instrumentation/opentelemetry-instrumentation-sqlalchemy/tests/test_sqlcommenter.py @@ -56,6 +56,24 @@ def test_sqlcommenter_enabled(self): r"SELECT 1 /\*db_driver='(.*)',traceparent='\d{1,2}-[a-zA-Z0-9_]{32}-[a-zA-Z0-9_]{16}-\d{1,2}'\*/;", ) + def test_sqlcommenter_enabled_otel_values_false(self): + engine = create_engine("sqlite:///:memory:") + SQLAlchemyInstrumentor().instrument( + engine=engine, + tracer_provider=self.tracer_provider, + enable_commenter=True, + commenter_options={ + "db_framework": False, + "opentelemetry_values": False, + }, + ) + cnx = engine.connect() + cnx.execute("SELECT 1;").fetchall() + self.assertRegex( + self.caplog.records[-2].getMessage(), + r"SELECT 1 /\*db_driver='(.*)'\*/;", + ) + def test_sqlcommenter_flask_integration(self): engine = create_engine("sqlite:///:memory:") SQLAlchemyInstrumentor().instrument( diff --git a/instrumentation/opentelemetry-instrumentation-sqlite3/pyproject.toml b/instrumentation/opentelemetry-instrumentation-sqlite3/pyproject.toml index dd3a33c72d..d7f6d6aae9 100644 --- a/instrumentation/opentelemetry-instrumentation-sqlite3/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-sqlite3/pyproject.toml @@ -26,14 +26,14 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-instrumentation-dbapi == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-instrumentation-dbapi == 0.42b0.dev", ] [project.optional-dependencies] instruments = [] test = [ - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-sqlite3/src/opentelemetry/instrumentation/sqlite3/version.py b/instrumentation/opentelemetry-instrumentation-sqlite3/src/opentelemetry/instrumentation/sqlite3/version.py index 16c754f1d3..084725a38e 100644 --- a/instrumentation/opentelemetry-instrumentation-sqlite3/src/opentelemetry/instrumentation/sqlite3/version.py +++ b/instrumentation/opentelemetry-instrumentation-sqlite3/src/opentelemetry/instrumentation/sqlite3/version.py @@ -12,6 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" _instruments = tuple() diff --git a/instrumentation/opentelemetry-instrumentation-starlette/pyproject.toml b/instrumentation/opentelemetry-instrumentation-starlette/pyproject.toml index fb24345dbb..4c884bcc5e 100644 --- a/instrumentation/opentelemetry-instrumentation-starlette/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-starlette/pyproject.toml @@ -26,10 +26,10 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-instrumentation-asgi == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", - "opentelemetry-util-http == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-instrumentation-asgi == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", + "opentelemetry-util-http == 0.42b0.dev", ] [project.optional-dependencies] @@ -38,7 +38,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-starlette[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", "requests ~= 2.23", # needed for testclient "httpx ~= 0.22", # needed for testclient ] diff --git a/instrumentation/opentelemetry-instrumentation-starlette/src/opentelemetry/instrumentation/starlette/__init__.py b/instrumentation/opentelemetry-instrumentation-starlette/src/opentelemetry/instrumentation/starlette/__init__.py index 30f24cc65c..2d123aa70e 100644 --- a/instrumentation/opentelemetry-instrumentation-starlette/src/opentelemetry/instrumentation/starlette/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-starlette/src/opentelemetry/instrumentation/starlette/__init__.py @@ -212,7 +212,7 @@ def instrument_app( app.add_middleware( OpenTelemetryMiddleware, excluded_urls=_excluded_urls, - default_span_details=_get_route_details, + default_span_details=_get_default_span_details, server_request_hook=server_request_hook, client_request_hook=client_request_hook, client_response_hook=client_response_hook, @@ -278,7 +278,7 @@ def __init__(self, *args, **kwargs): self.add_middleware( OpenTelemetryMiddleware, excluded_urls=_excluded_urls, - default_span_details=_get_route_details, + default_span_details=_get_default_span_details, server_request_hook=_InstrumentedStarlette._server_request_hook, client_request_hook=_InstrumentedStarlette._client_request_hook, client_response_hook=_InstrumentedStarlette._client_response_hook, @@ -294,15 +294,21 @@ def __del__(self): def _get_route_details(scope): - """Callback to retrieve the starlette route being served. + """ + Function to retrieve Starlette route from scope. TODO: there is currently no way to retrieve http.route from a starlette application from scope. - See: https://github.com/encode/starlette/pull/804 + + Args: + scope: A Starlette scope + Returns: + A string containing the route or None """ app = scope["app"] route = None + for starlette_route in app.routes: match, _ = starlette_route.matches(scope) if match == Match.FULL: @@ -310,10 +316,27 @@ def _get_route_details(scope): break if match == Match.PARTIAL: route = starlette_route.path - # method only exists for http, if websocket - # leave it blank. - span_name = route or scope.get("method", "") + return route + + +def _get_default_span_details(scope): + """ + Callback to retrieve span name and attributes from scope. + + Args: + scope: A Starlette scope + Returns: + A tuple of span name and attributes + """ + route = _get_route_details(scope) + method = scope.get("method", "") attributes = {} if route: attributes[SpanAttributes.HTTP_ROUTE] = route + if method and route: # http + span_name = f"{method} {route}" + elif route: # websocket + span_name = route + else: # fallback + span_name = method return span_name, attributes diff --git a/instrumentation/opentelemetry-instrumentation-starlette/src/opentelemetry/instrumentation/starlette/version.py b/instrumentation/opentelemetry-instrumentation-starlette/src/opentelemetry/instrumentation/starlette/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-starlette/src/opentelemetry/instrumentation/starlette/version.py +++ b/instrumentation/opentelemetry-instrumentation-starlette/src/opentelemetry/instrumentation/starlette/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-starlette/tests/test_starlette_instrumentation.py b/instrumentation/opentelemetry-instrumentation-starlette/tests/test_starlette_instrumentation.py index 9c658e0092..e1c77312a4 100644 --- a/instrumentation/opentelemetry-instrumentation-starlette/tests/test_starlette_instrumentation.py +++ b/instrumentation/opentelemetry-instrumentation-starlette/tests/test_starlette_instrumentation.py @@ -49,10 +49,14 @@ _expected_metric_names = [ "http.server.active_requests", "http.server.duration", + "http.server.response.size", + "http.server.request.size", ] _recommended_attrs = { "http.server.active_requests": _active_requests_count_attrs, "http.server.duration": _duration_attrs, + "http.server.response.size": _duration_attrs, + "http.server.request.size": _duration_attrs, } @@ -93,7 +97,7 @@ def test_basic_starlette_call(self): spans = self.memory_exporter.get_finished_spans() self.assertEqual(len(spans), 3) for span in spans: - self.assertIn("/foobar", span.name) + self.assertIn("GET /foobar", span.name) def test_starlette_route_attribute_added(self): """Ensure that starlette routes are used as the span name.""" @@ -101,7 +105,7 @@ def test_starlette_route_attribute_added(self): spans = self.memory_exporter.get_finished_spans() self.assertEqual(len(spans), 3) for span in spans: - self.assertIn("/user/{username}", span.name) + self.assertIn("GET /user/{username}", span.name) self.assertEqual( spans[-1].attributes[SpanAttributes.HTTP_ROUTE], "/user/{username}" ) @@ -128,7 +132,7 @@ def test_starlette_metrics(self): for resource_metric in metrics_list.resource_metrics: self.assertTrue(len(resource_metric.scope_metrics) == 1) for scope_metric in resource_metric.scope_metrics: - self.assertTrue(len(scope_metric.metrics) == 2) + self.assertTrue(len(scope_metric.metrics) == 3) for metric in scope_metric.metrics: self.assertIn(metric.name, _expected_metric_names) data_points = list(metric.data.data_points) @@ -163,8 +167,13 @@ def test_basic_post_request_metric_success(self): "http.scheme": "http", "http.server_name": "testserver", } - self._client.post("/foobar") + response = self._client.post( + "/foobar", + json={"foo": "bar"}, + ) duration = max(round((default_timer() - start) * 1000), 0) + response_size = int(response.headers.get("content-length")) + request_size = int(response.request.headers.get("content-length")) metrics_list = self.memory_metrics_reader.get_metrics_data() for metric in ( metrics_list.resource_metrics[0].scope_metrics[0].metrics @@ -172,10 +181,15 @@ def test_basic_post_request_metric_success(self): for point in list(metric.data.data_points): if isinstance(point, HistogramDataPoint): self.assertEqual(point.count, 1) - self.assertAlmostEqual(duration, point.sum, delta=30) self.assertDictEqual( dict(point.attributes), expected_duration_attributes ) + if metric.name == "http.server.duration": + self.assertAlmostEqual(duration, point.sum, delta=30) + elif metric.name == "http.server.response.size": + self.assertEqual(response_size, point.sum) + elif metric.name == "http.server.request.size": + self.assertEqual(request_size, point.sum) if isinstance(point, NumberDataPoint): self.assertDictEqual( expected_requests_count_attributes, diff --git a/instrumentation/opentelemetry-instrumentation-system-metrics/pyproject.toml b/instrumentation/opentelemetry-instrumentation-system-metrics/pyproject.toml index 0a01e8bd27..3c9f363a4c 100644 --- a/instrumentation/opentelemetry-instrumentation-system-metrics/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-system-metrics/pyproject.toml @@ -36,7 +36,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-system-metrics[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-system-metrics/src/opentelemetry/instrumentation/system_metrics/version.py b/instrumentation/opentelemetry-instrumentation-system-metrics/src/opentelemetry/instrumentation/system_metrics/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-system-metrics/src/opentelemetry/instrumentation/system_metrics/version.py +++ b/instrumentation/opentelemetry-instrumentation-system-metrics/src/opentelemetry/instrumentation/system_metrics/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-tornado/pyproject.toml b/instrumentation/opentelemetry-instrumentation-tornado/pyproject.toml index b01e594cd1..1a286e5114 100644 --- a/instrumentation/opentelemetry-instrumentation-tornado/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-tornado/pyproject.toml @@ -25,9 +25,9 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", - "opentelemetry-util-http == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", + "opentelemetry-util-http == 0.42b0.dev", ] [project.optional-dependencies] @@ -36,7 +36,8 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-tornado[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", + "http-server-mock" ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-tornado/src/opentelemetry/instrumentation/tornado/__init__.py b/instrumentation/opentelemetry-instrumentation-tornado/src/opentelemetry/instrumentation/tornado/__init__.py index dd8da74f3f..1e2f0e5162 100644 --- a/instrumentation/opentelemetry-instrumentation-tornado/src/opentelemetry/instrumentation/tornado/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-tornado/src/opentelemetry/instrumentation/tornado/__init__.py @@ -454,10 +454,23 @@ def _get_attributes_from_request(request): ) -def _get_operation_name(handler, request): - full_class_name = type(handler).__name__ - class_name = full_class_name.rsplit(".")[-1] - return f"{class_name}.{request.method.lower()}" +def _get_default_span_name(request): + """ + Default span name is the HTTP method and URL path, or just the method. + https://github.com/open-telemetry/opentelemetry-specification/pull/3165 + https://opentelemetry.io/docs/reference/specification/trace/semantic_conventions/http/#name + + Args: + request: Tornado request object. + Returns: + Default span name. + """ + + path = request.path + method = request.method + if method and path: + return f"{method} {path}" + return f"{method}" def _get_full_handler_name(handler): @@ -468,7 +481,7 @@ def _get_full_handler_name(handler): def _start_span(tracer, handler) -> _TraceContext: span, token = _start_internal_or_server_span( tracer=tracer, - span_name=_get_operation_name(handler, handler.request), + span_name=_get_default_span_name(handler.request), start_time=time_ns(), context_carrier=handler.request.headers, context_getter=textmap.default_getter, diff --git a/instrumentation/opentelemetry-instrumentation-tornado/src/opentelemetry/instrumentation/tornado/version.py b/instrumentation/opentelemetry-instrumentation-tornado/src/opentelemetry/instrumentation/tornado/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-tornado/src/opentelemetry/instrumentation/tornado/version.py +++ b/instrumentation/opentelemetry-instrumentation-tornado/src/opentelemetry/instrumentation/tornado/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-tornado/tests/test_instrumentation.py b/instrumentation/opentelemetry-instrumentation-tornado/tests/test_instrumentation.py index 9fb3608572..0baaa348ab 100644 --- a/instrumentation/opentelemetry-instrumentation-tornado/tests/test_instrumentation.py +++ b/instrumentation/opentelemetry-instrumentation-tornado/tests/test_instrumentation.py @@ -15,6 +15,7 @@ from unittest.mock import Mock, patch +from http_server_mock import HttpServerMock from tornado.testing import AsyncHTTPTestCase from opentelemetry import trace @@ -135,7 +136,7 @@ def _test_http_method_call(self, method): self.assertEqual(manual.parent, server.context) self.assertEqual(manual.context.trace_id, client.context.trace_id) - self.assertEqual(server.name, "MainHandler." + method.lower()) + self.assertEqual(server.name, f"{method} /") self.assertTrue(server.parent.is_remote) self.assertNotEqual(server.parent, client.context) self.assertEqual(server.parent.span_id, client.context.span_id) @@ -196,7 +197,7 @@ def _test_async_handler(self, url, handler_name): self.assertEqual(len(spans), 5) client = spans.by_name("GET") - server = spans.by_name(handler_name + ".get") + server = spans.by_name(f"GET {url}") sub_wrapper = spans.by_name("sub-task-wrapper") sub2 = spans.by_name("sub-task-2") @@ -213,7 +214,7 @@ def _test_async_handler(self, url, handler_name): self.assertEqual(sub_wrapper.parent, server.context) self.assertEqual(sub_wrapper.context.trace_id, client.context.trace_id) - self.assertEqual(server.name, handler_name + ".get") + self.assertEqual(server.name, f"GET {url}") self.assertTrue(server.parent.is_remote) self.assertNotEqual(server.parent, client.context) self.assertEqual(server.parent.span_id, client.context.span_id) @@ -229,6 +230,7 @@ def _test_async_handler(self, url, handler_name): SpanAttributes.HTTP_TARGET: url, SpanAttributes.HTTP_CLIENT_IP: "127.0.0.1", SpanAttributes.HTTP_STATUS_CODE: 201, + "tornado.handler": f"tests.tornado_test_app.{handler_name}", }, ) @@ -253,9 +255,9 @@ def test_500(self): self.assertEqual(len(spans), 2) client = spans.by_name("GET") - server = spans.by_name("BadHandler.get") + server = spans.by_name("GET /error") - self.assertEqual(server.name, "BadHandler.get") + self.assertEqual(server.name, "GET /error") self.assertEqual(server.kind, SpanKind.SERVER) self.assertSpanHasAttributes( server, @@ -290,7 +292,7 @@ def test_404(self): self.assertEqual(len(spans), 2) server, client = spans - self.assertEqual(server.name, "ErrorHandler.get") + self.assertEqual(server.name, "GET /missing-url") self.assertEqual(server.kind, SpanKind.SERVER) self.assertSpanHasAttributes( server, @@ -325,7 +327,7 @@ def test_http_error(self): self.assertEqual(len(spans), 2) server, client = spans - self.assertEqual(server.name, "RaiseHTTPErrorHandler.get") + self.assertEqual(server.name, "GET /raise_403") self.assertEqual(server.kind, SpanKind.SERVER) self.assertSpanHasAttributes( server, @@ -366,7 +368,7 @@ def test_dynamic_handler(self): self.assertEqual(len(spans), 2) server, client = spans - self.assertEqual(server.name, "DynamicHandler.get") + self.assertEqual(server.name, "GET /dyna") self.assertTrue(server.parent.is_remote) self.assertNotEqual(server.parent, client.context) self.assertEqual(server.parent.span_id, client.context.span_id) @@ -407,7 +409,7 @@ def test_handler_on_finish(self): self.assertEqual(len(spans), 3) auditor, server, client = spans - self.assertEqual(server.name, "FinishedHandler.get") + self.assertEqual(server.name, "GET /on_finish") self.assertTrue(server.parent.is_remote) self.assertNotEqual(server.parent, client.context) self.assertEqual(server.parent.span_id, client.context.span_id) @@ -494,32 +496,35 @@ def test_response_headers(self): self.memory_exporter.clear() set_global_response_propagator(orig) - # todo(srikanthccv): fix this test - # this test is making request to real httpbin.org/status/200 which - # is not a good idea as it can fail due to availability of the - # service. - # def test_credential_removal(self): - # response = self.fetch( - # "http://username:password@httpbin.org/status/200" - # ) - # self.assertEqual(response.code, 200) - - # spans = self.sorted_spans(self.memory_exporter.get_finished_spans()) - # self.assertEqual(len(spans), 1) - # client = spans[0] - - # self.assertEqual(client.name, "GET") - # self.assertEqual(client.kind, SpanKind.CLIENT) - # self.assertSpanHasAttributes( - # client, - # { - # SpanAttributes.HTTP_URL: "http://httpbin.org/status/200", - # SpanAttributes.HTTP_METHOD: "GET", - # SpanAttributes.HTTP_STATUS_CODE: 200, - # }, - # ) - - # self.memory_exporter.clear() + def test_credential_removal(self): + app = HttpServerMock("test_credential_removal") + + @app.route("/status/200") + def index(): + return "hello" + + with app.run("localhost", 5000): + response = self.fetch( + "http://username:password@localhost:5000/status/200" + ) + self.assertEqual(response.code, 200) + + spans = self.sorted_spans(self.memory_exporter.get_finished_spans()) + self.assertEqual(len(spans), 1) + client = spans[0] + + self.assertEqual(client.name, "GET") + self.assertEqual(client.kind, SpanKind.CLIENT) + self.assertSpanHasAttributes( + client, + { + SpanAttributes.HTTP_URL: "http://localhost:5000/status/200", + SpanAttributes.HTTP_METHOD: "GET", + SpanAttributes.HTTP_STATUS_CODE: 200, + }, + ) + + self.memory_exporter.clear() class TestTornadoInstrumentationWithXHeaders(TornadoTest): @@ -531,7 +536,7 @@ def test_xheaders(self): self.assertEqual(response.code, 201) spans = self.get_finished_spans() self.assertSpanHasAttributes( - spans.by_name("MainHandler.get"), + spans.by_name("GET /"), { SpanAttributes.HTTP_METHOD: "GET", SpanAttributes.HTTP_SCHEME: "http", @@ -605,7 +610,7 @@ def test_uninstrument(self): self.assertEqual(len(spans), 3) manual, server, client = self.sorted_spans(spans) self.assertEqual(manual.name, "manual") - self.assertEqual(server.name, "MainHandler.get") + self.assertEqual(server.name, "GET /") self.assertEqual(client.name, "GET") self.memory_exporter.clear() diff --git a/instrumentation/opentelemetry-instrumentation-tortoiseorm/pyproject.toml b/instrumentation/opentelemetry-instrumentation-tortoiseorm/pyproject.toml index 8f8ffb5995..42e3573806 100644 --- a/instrumentation/opentelemetry-instrumentation-tortoiseorm/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-tortoiseorm/pyproject.toml @@ -26,8 +26,8 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", ] [project.optional-dependencies] @@ -37,7 +37,7 @@ instruments = [ ] test = [ "opentelemetry-instrumentation-tortoiseorm[instruments]", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-tortoiseorm/src/opentelemetry/instrumentation/tortoiseorm/version.py b/instrumentation/opentelemetry-instrumentation-tortoiseorm/src/opentelemetry/instrumentation/tortoiseorm/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-tortoiseorm/src/opentelemetry/instrumentation/tortoiseorm/version.py +++ b/instrumentation/opentelemetry-instrumentation-tortoiseorm/src/opentelemetry/instrumentation/tortoiseorm/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-urllib/pyproject.toml b/instrumentation/opentelemetry-instrumentation-urllib/pyproject.toml index 1c85bf13d1..56bfb5e869 100644 --- a/instrumentation/opentelemetry-instrumentation-urllib/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-urllib/pyproject.toml @@ -26,16 +26,16 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", - "opentelemetry-util-http == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", + "opentelemetry-util-http == 0.42b0.dev", ] [project.optional-dependencies] instruments = [] test = [ "httpretty ~= 1.0", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-urllib/src/opentelemetry/instrumentation/urllib/__init__.py b/instrumentation/opentelemetry-instrumentation-urllib/src/opentelemetry/instrumentation/urllib/__init__.py index 091ccf99b1..cdd35a0bad 100644 --- a/instrumentation/opentelemetry-instrumentation-urllib/src/opentelemetry/instrumentation/urllib/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-urllib/src/opentelemetry/instrumentation/urllib/__init__.py @@ -207,7 +207,7 @@ def _instrumented_open_call( method = request.get_method().upper() - span_name = f"HTTP {method}".strip() + span_name = method.strip() url = remove_url_credentials(url) diff --git a/instrumentation/opentelemetry-instrumentation-urllib/src/opentelemetry/instrumentation/urllib/version.py b/instrumentation/opentelemetry-instrumentation-urllib/src/opentelemetry/instrumentation/urllib/version.py index 16c754f1d3..084725a38e 100644 --- a/instrumentation/opentelemetry-instrumentation-urllib/src/opentelemetry/instrumentation/urllib/version.py +++ b/instrumentation/opentelemetry-instrumentation-urllib/src/opentelemetry/instrumentation/urllib/version.py @@ -12,6 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" _instruments = tuple() diff --git a/instrumentation/opentelemetry-instrumentation-urllib/tests/test_metrics_instrumentation.py b/instrumentation/opentelemetry-instrumentation-urllib/tests/test_metrics_instrumentation.py index c9417fc67b..f56aa4f97d 100644 --- a/instrumentation/opentelemetry-instrumentation-urllib/tests/test_metrics_instrumentation.py +++ b/instrumentation/opentelemetry-instrumentation-urllib/tests/test_metrics_instrumentation.py @@ -27,8 +27,8 @@ class TestUrllibMetricsInstrumentation(TestBase): - URL = "http://httpbin.org/status/200" - URL_POST = "http://httpbin.org/post" + URL = "http://mock/status/200" + URL_POST = "http://mock/post" def setUp(self): super().setUp() diff --git a/instrumentation/opentelemetry-instrumentation-urllib/tests/test_urllib_integration.py b/instrumentation/opentelemetry-instrumentation-urllib/tests/test_urllib_integration.py index 9937d42176..f27f594a30 100644 --- a/instrumentation/opentelemetry-instrumentation-urllib/tests/test_urllib_integration.py +++ b/instrumentation/opentelemetry-instrumentation-urllib/tests/test_urllib_integration.py @@ -46,9 +46,9 @@ class RequestsIntegrationTestBase(abc.ABC): # pylint: disable=no-member - URL = "http://httpbin.org/status/200" - URL_TIMEOUT = "http://httpbin.org/timeout/0" - URL_EXCEPTION = "http://httpbin.org/exception/0" + URL = "http://mock/status/200" + URL_TIMEOUT = "http://mock/timeout/0" + URL_EXCEPTION = "http://mock/exception/0" # pylint: disable=invalid-name def setUp(self): @@ -83,7 +83,7 @@ def setUp(self): ) httpretty.register_uri( httpretty.GET, - "http://httpbin.org/status/500", + "http://mock/status/500", status=500, ) @@ -124,7 +124,7 @@ def test_basic(self): span = self.assert_span() self.assertIs(span.kind, trace.SpanKind.CLIENT) - self.assertEqual(span.name, "HTTP GET") + self.assertEqual(span.name, "GET") self.assertEqual( span.attributes, @@ -142,7 +142,7 @@ def test_basic(self): ) def test_excluded_urls_explicit(self): - url_201 = "http://httpbin.org/status/201" + url_201 = "http://mock/status/201" httpretty.register_uri( httpretty.GET, url_201, @@ -172,7 +172,7 @@ def test_excluded_urls_from_env(self): self.assert_span(num_spans=1) def test_not_foundbasic(self): - url_404 = "http://httpbin.org/status/404/" + url_404 = "http://mock/status/404/" httpretty.register_uri( httpretty.GET, url_404, @@ -209,7 +209,7 @@ def test_response_code_none(self): span = self.assert_span() self.assertIs(span.kind, trace.SpanKind.CLIENT) - self.assertEqual(span.name, "HTTP GET") + self.assertEqual(span.name, "GET") self.assertEqual( span.attributes, @@ -336,14 +336,14 @@ def test_custom_tracer_provider(self): def test_requests_exception_with_response(self, *_, **__): with self.assertRaises(HTTPError): - self.perform_request("http://httpbin.org/status/500") + self.perform_request("http://mock/status/500") span = self.assert_span() self.assertEqual( dict(span.attributes), { SpanAttributes.HTTP_METHOD: "GET", - SpanAttributes.HTTP_URL: "http://httpbin.org/status/500", + SpanAttributes.HTTP_URL: "http://mock/status/500", SpanAttributes.HTTP_STATUS_CODE: 500, }, ) @@ -365,7 +365,7 @@ def test_requests_timeout_exception(self, *_, **__): self.assertEqual(span.status.status_code, StatusCode.ERROR) def test_credential_removal(self): - url = "http://username:password@httpbin.org/status/200" + url = "http://username:password@mock/status/200" with self.assertRaises(Exception): self.perform_request(url) diff --git a/instrumentation/opentelemetry-instrumentation-urllib3/README.rst b/instrumentation/opentelemetry-instrumentation-urllib3/README.rst index 0c53c299a0..4e0089e21e 100644 --- a/instrumentation/opentelemetry-instrumentation-urllib3/README.rst +++ b/instrumentation/opentelemetry-instrumentation-urllib3/README.rst @@ -38,8 +38,8 @@ The hooks can be configured as follows: def response_hook(span, request, response): pass - URLLib3Instrumentor.instrument( - request_hook=request_hook, response_hook=response_hook) + URLLib3Instrumentor().instrument( + request_hook=request_hook, response_hook=response_hook ) Exclude lists diff --git a/instrumentation/opentelemetry-instrumentation-urllib3/pyproject.toml b/instrumentation/opentelemetry-instrumentation-urllib3/pyproject.toml index 4757ed7fde..db4d485d8e 100644 --- a/instrumentation/opentelemetry-instrumentation-urllib3/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-urllib3/pyproject.toml @@ -26,20 +26,20 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", - "opentelemetry-util-http == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", + "opentelemetry-util-http == 0.42b0.dev", "wrapt >= 1.0.0, < 2.0.0", ] [project.optional-dependencies] instruments = [ - "urllib3 >= 1.0.0, < 2.0.0", + "urllib3 >= 1.0.0, < 3.0.0", ] test = [ "opentelemetry-instrumentation-urllib3[instruments]", "httpretty ~= 1.0", - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-urllib3/src/opentelemetry/instrumentation/urllib3/__init__.py b/instrumentation/opentelemetry-instrumentation-urllib3/src/opentelemetry/instrumentation/urllib3/__init__.py index 91d5576fc0..d3016ea5ee 100644 --- a/instrumentation/opentelemetry-instrumentation-urllib3/src/opentelemetry/instrumentation/urllib3/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-urllib3/src/opentelemetry/instrumentation/urllib3/__init__.py @@ -56,8 +56,8 @@ def request_hook(span, request): def response_hook(span, request, response): pass - URLLib3Instrumentor.instrument( - request_hook=request_hook, response_hook=response_hook) + URLLib3Instrumentor().instrument( + request_hook=request_hook, response_hook=response_hook ) Exclude lists @@ -225,7 +225,7 @@ def instrumented_urlopen(wrapped, instance, args, kwargs): headers = _prepare_headers(kwargs) body = _get_url_open_arg("body", args, kwargs) - span_name = f"HTTP {method.strip()}" + span_name = method.strip() span_attributes = { SpanAttributes.HTTP_METHOD: method, SpanAttributes.HTTP_URL: url, diff --git a/instrumentation/opentelemetry-instrumentation-urllib3/src/opentelemetry/instrumentation/urllib3/package.py b/instrumentation/opentelemetry-instrumentation-urllib3/src/opentelemetry/instrumentation/urllib3/package.py index 2f5df62de8..9d52db0a1f 100644 --- a/instrumentation/opentelemetry-instrumentation-urllib3/src/opentelemetry/instrumentation/urllib3/package.py +++ b/instrumentation/opentelemetry-instrumentation-urllib3/src/opentelemetry/instrumentation/urllib3/package.py @@ -13,6 +13,6 @@ # limitations under the License. -_instruments = ("urllib3 >= 1.0.0, < 2.0.0",) +_instruments = ("urllib3 >= 1.0.0, < 3.0.0",) _supports_metrics = True diff --git a/instrumentation/opentelemetry-instrumentation-urllib3/src/opentelemetry/instrumentation/urllib3/version.py b/instrumentation/opentelemetry-instrumentation-urllib3/src/opentelemetry/instrumentation/urllib3/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-urllib3/src/opentelemetry/instrumentation/urllib3/version.py +++ b/instrumentation/opentelemetry-instrumentation-urllib3/src/opentelemetry/instrumentation/urllib3/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-urllib3/tests/test_urllib3_integration.py b/instrumentation/opentelemetry-instrumentation-urllib3/tests/test_urllib3_integration.py index ae59d57c51..7ba7e2731b 100644 --- a/instrumentation/opentelemetry-instrumentation-urllib3/tests/test_urllib3_integration.py +++ b/instrumentation/opentelemetry-instrumentation-urllib3/tests/test_urllib3_integration.py @@ -35,8 +35,8 @@ class TestURLLib3Instrumentor(TestBase): - HTTP_URL = "http://httpbin.org/status/200" - HTTPS_URL = "https://httpbin.org/status/200" + HTTP_URL = "http://mock/status/200" + HTTPS_URL = "https://mock/status/200" def setUp(self): super().setUp() @@ -87,7 +87,7 @@ def assert_success_span( span = self.assert_span() self.assertIs(trace.SpanKind.CLIENT, span.kind) - self.assertEqual("HTTP GET", span.name) + self.assertEqual("GET", span.name) attributes = { SpanAttributes.HTTP_METHOD: "GET", @@ -123,7 +123,7 @@ def test_basic_http_success(self): self.assert_success_span(response, self.HTTP_URL) def test_basic_http_success_using_connection_pool(self): - pool = urllib3.HTTPConnectionPool("httpbin.org") + pool = urllib3.HTTPConnectionPool("mock") response = pool.request("GET", "/status/200") self.assert_success_span(response, self.HTTP_URL) @@ -133,13 +133,13 @@ def test_basic_https_success(self): self.assert_success_span(response, self.HTTPS_URL) def test_basic_https_success_using_connection_pool(self): - pool = urllib3.HTTPSConnectionPool("httpbin.org") + pool = urllib3.HTTPSConnectionPool("mock") response = pool.request("GET", "/status/200") self.assert_success_span(response, self.HTTPS_URL) def test_basic_not_found(self): - url_404 = "http://httpbin.org/status/404" + url_404 = "http://mock/status/404" httpretty.register_uri(httpretty.GET, url_404, status=404) response = self.perform_request(url_404) @@ -152,30 +152,30 @@ def test_basic_not_found(self): self.assertIs(trace.status.StatusCode.ERROR, span.status.status_code) def test_basic_http_non_default_port(self): - url = "http://httpbin.org:666/status/200" + url = "http://mock:666/status/200" httpretty.register_uri(httpretty.GET, url, body="Hello!") response = self.perform_request(url) self.assert_success_span(response, url) def test_basic_http_absolute_url(self): - url = "http://httpbin.org:666/status/200" + url = "http://mock:666/status/200" httpretty.register_uri(httpretty.GET, url, body="Hello!") - pool = urllib3.HTTPConnectionPool("httpbin.org", port=666) + pool = urllib3.HTTPConnectionPool("mock", port=666) response = pool.request("GET", url) self.assert_success_span(response, url) def test_url_open_explicit_arg_parameters(self): - url = "http://httpbin.org:666/status/200" + url = "http://mock:666/status/200" httpretty.register_uri(httpretty.GET, url, body="Hello!") - pool = urllib3.HTTPConnectionPool("httpbin.org", port=666) + pool = urllib3.HTTPConnectionPool("mock", port=666) response = pool.urlopen(method="GET", url="/status/200") self.assert_success_span(response, url) def test_excluded_urls_explicit(self): - url_201 = "http://httpbin.org/status/201" + url_201 = "http://mock/status/201" httpretty.register_uri( httpretty.GET, url_201, @@ -301,7 +301,7 @@ def url_filter(url): self.assert_success_span(response, self.HTTP_URL) def test_credential_removal(self): - url = "http://username:password@httpbin.org/status/200" + url = "http://username:password@mock/status/200" response = self.perform_request(url) self.assert_success_span(response, self.HTTP_URL) @@ -328,7 +328,9 @@ def response_hook(span, request, response): def test_request_hook_params(self): def request_hook(span, request, headers, body): - span.set_attribute("request_hook_headers", json.dumps(headers)) + span.set_attribute( + "request_hook_headers", json.dumps(dict(headers)) + ) span.set_attribute("request_hook_body", body) URLLib3Instrumentor().uninstrument() @@ -339,7 +341,7 @@ def request_hook(span, request, headers, body): headers = {"header1": "value1", "header2": "value2"} body = "param1=1¶m2=2" - pool = urllib3.HTTPConnectionPool("httpbin.org") + pool = urllib3.HTTPConnectionPool("mock") response = pool.request( "POST", "/status/200", body=body, headers=headers ) @@ -366,7 +368,7 @@ def request_hook(span, request, headers, body): body = "param1=1¶m2=2" - pool = urllib3.HTTPConnectionPool("httpbin.org") + pool = urllib3.HTTPConnectionPool("mock") response = pool.urlopen("POST", "/status/200", body) self.assertEqual(b"Hello!", response.data) diff --git a/instrumentation/opentelemetry-instrumentation-urllib3/tests/test_urllib3_ip_support.py b/instrumentation/opentelemetry-instrumentation-urllib3/tests/test_urllib3_ip_support.py index 5baddee516..1199ad3d5b 100644 --- a/instrumentation/opentelemetry-instrumentation-urllib3/tests/test_urllib3_ip_support.py +++ b/instrumentation/opentelemetry-instrumentation-urllib3/tests/test_urllib3_ip_support.py @@ -77,7 +77,7 @@ def assert_success_span( span = self.assert_span() self.assertIs(trace.SpanKind.CLIENT, span.kind) - self.assertEqual("HTTP GET", span.name) + self.assertEqual("GET", span.name) attributes = { "http.status_code": 200, diff --git a/instrumentation/opentelemetry-instrumentation-urllib3/tests/test_urllib3_metrics.py b/instrumentation/opentelemetry-instrumentation-urllib3/tests/test_urllib3_metrics.py index ca691ebd47..2fd4cb2c5c 100644 --- a/instrumentation/opentelemetry-instrumentation-urllib3/tests/test_urllib3_metrics.py +++ b/instrumentation/opentelemetry-instrumentation-urllib3/tests/test_urllib3_metrics.py @@ -18,7 +18,7 @@ import httpretty import urllib3 import urllib3.exceptions -from urllib3.request import encode_multipart_formdata +from urllib3 import encode_multipart_formdata from opentelemetry.instrumentation.urllib3 import URLLib3Instrumentor from opentelemetry.test.httptest import HttpTestBase @@ -26,7 +26,7 @@ class TestURLLib3InstrumentorMetric(HttpTestBase, TestBase): - HTTP_URL = "http://httpbin.org/status/200" + HTTP_URL = "http://mock/status/200" def setUp(self): super().setUp() @@ -68,11 +68,11 @@ def test_basic_metrics(self): min_data_point=client_duration_estimated, attributes={ "http.flavor": "1.1", - "http.host": "httpbin.org", + "http.host": "mock", "http.method": "GET", "http.scheme": "http", "http.status_code": 200, - "net.peer.name": "httpbin.org", + "net.peer.name": "mock", "net.peer.port": 80, }, ) @@ -91,11 +91,11 @@ def test_basic_metrics(self): min_data_point=0, attributes={ "http.flavor": "1.1", - "http.host": "httpbin.org", + "http.host": "mock", "http.method": "GET", "http.scheme": "http", "http.status_code": 200, - "net.peer.name": "httpbin.org", + "net.peer.name": "mock", "net.peer.port": 80, }, ) @@ -116,11 +116,11 @@ def test_basic_metrics(self): min_data_point=expected_size, attributes={ "http.flavor": "1.1", - "http.host": "httpbin.org", + "http.host": "mock", "http.method": "GET", "http.scheme": "http", "http.status_code": 200, - "net.peer.name": "httpbin.org", + "net.peer.name": "mock", "net.peer.port": 80, }, ) @@ -144,11 +144,11 @@ def test_str_request_body_size_metrics(self): min_data_point=6, attributes={ "http.flavor": "1.1", - "http.host": "httpbin.org", + "http.host": "mock", "http.method": "POST", "http.scheme": "http", "http.status_code": 200, - "net.peer.name": "httpbin.org", + "net.peer.name": "mock", "net.peer.port": 80, }, ) @@ -172,11 +172,11 @@ def test_bytes_request_body_size_metrics(self): min_data_point=6, attributes={ "http.flavor": "1.1", - "http.host": "httpbin.org", + "http.host": "mock", "http.method": "POST", "http.scheme": "http", "http.status_code": 200, - "net.peer.name": "httpbin.org", + "net.peer.name": "mock", "net.peer.port": 80, }, ) @@ -201,11 +201,11 @@ def test_fields_request_body_size_metrics(self): min_data_point=expected_value, attributes={ "http.flavor": "1.1", - "http.host": "httpbin.org", + "http.host": "mock", "http.method": "POST", "http.scheme": "http", "http.status_code": 200, - "net.peer.name": "httpbin.org", + "net.peer.name": "mock", "net.peer.port": 80, }, ) @@ -229,11 +229,11 @@ def test_bytesio_request_body_size_metrics(self): min_data_point=6, attributes={ "http.flavor": "1.1", - "http.host": "httpbin.org", + "http.host": "mock", "http.method": "POST", "http.scheme": "http", "http.status_code": 200, - "net.peer.name": "httpbin.org", + "net.peer.name": "mock", "net.peer.port": 80, }, ) diff --git a/instrumentation/opentelemetry-instrumentation-wsgi/pyproject.toml b/instrumentation/opentelemetry-instrumentation-wsgi/pyproject.toml index aa0c25c140..50860fb0d6 100644 --- a/instrumentation/opentelemetry-instrumentation-wsgi/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-wsgi/pyproject.toml @@ -26,15 +26,15 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", - "opentelemetry-semantic-conventions == 0.39b0.dev", - "opentelemetry-util-http == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", + "opentelemetry-semantic-conventions == 0.42b0.dev", + "opentelemetry-util-http == 0.42b0.dev", ] [project.optional-dependencies] instruments = [] test = [ - "opentelemetry-test-utils == 0.39b0.dev", + "opentelemetry-test-utils == 0.42b0.dev", ] [project.urls] diff --git a/instrumentation/opentelemetry-instrumentation-wsgi/src/opentelemetry/instrumentation/wsgi/__init__.py b/instrumentation/opentelemetry-instrumentation-wsgi/src/opentelemetry/instrumentation/wsgi/__init__.py index b4d53f9a8b..35e217264d 100644 --- a/instrumentation/opentelemetry-instrumentation-wsgi/src/opentelemetry/instrumentation/wsgi/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-wsgi/src/opentelemetry/instrumentation/wsgi/__init__.py @@ -197,6 +197,12 @@ def response_hook(span: Span, environ: WSGIEnvironment, status: str, response_he Note: The environment variable names used to capture HTTP headers are still experimental, and thus are subject to change. +Sanitizing methods +****************** +In order to prevent unbound cardinality for HTTP methods by default nonstandard ones are labeled as ``NONSTANDARD``. +To record all of the names set the environment variable ``OTEL_PYTHON_INSTRUMENTATION_HTTP_CAPTURE_ALL_METHODS`` +to a value that evaluates to true, e.g. ``1``. + API --- """ @@ -226,6 +232,7 @@ def response_hook(span: Span, environ: WSGIEnvironment, status: str, response_he normalise_request_header_name, normalise_response_header_name, remove_url_credentials, + sanitize_method, ) _HTTP_VERSION_PREFIX = "HTTP/" @@ -295,7 +302,7 @@ def collect_request_attributes(environ): """ result = { - SpanAttributes.HTTP_METHOD: environ.get("REQUEST_METHOD"), + SpanAttributes.HTTP_METHOD: sanitize_method(environ.get("REQUEST_METHOD")), SpanAttributes.HTTP_SERVER_NAME: environ.get("SERVER_NAME"), SpanAttributes.HTTP_SCHEME: environ.get("wsgi.url_scheme"), } @@ -440,8 +447,21 @@ def add_response_attributes( def get_default_span_name(environ): - """Default implementation for name_callback, returns HTTP {METHOD_NAME}.""" - return f"HTTP {environ.get('REQUEST_METHOD', '')}".strip() + """ + Default span name is the HTTP method and URL path, or just the method. + https://github.com/open-telemetry/opentelemetry-specification/pull/3165 + https://opentelemetry.io/docs/reference/specification/trace/semantic_conventions/http/#name + + Args: + environ: The WSGI environ object. + Returns: + The span name. + """ + method = sanitize_method(environ.get("REQUEST_METHOD", "").strip()) + path = environ.get("PATH_INFO", "").strip() + if method and path: + return f"{method} {path}" + return method class OpenTelemetryMiddleware: diff --git a/instrumentation/opentelemetry-instrumentation-wsgi/src/opentelemetry/instrumentation/wsgi/version.py b/instrumentation/opentelemetry-instrumentation-wsgi/src/opentelemetry/instrumentation/wsgi/version.py index eb62a67e28..c2996671d6 100644 --- a/instrumentation/opentelemetry-instrumentation-wsgi/src/opentelemetry/instrumentation/wsgi/version.py +++ b/instrumentation/opentelemetry-instrumentation-wsgi/src/opentelemetry/instrumentation/wsgi/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/instrumentation/opentelemetry-instrumentation-wsgi/tests/test_wsgi_middleware.py b/instrumentation/opentelemetry-instrumentation-wsgi/tests/test_wsgi_middleware.py index ffe2982052..6aef096218 100644 --- a/instrumentation/opentelemetry-instrumentation-wsgi/tests/test_wsgi_middleware.py +++ b/instrumentation/opentelemetry-instrumentation-wsgi/tests/test_wsgi_middleware.py @@ -33,6 +33,7 @@ OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS, OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST, OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE, + OTEL_PYTHON_INSTRUMENTATION_HTTP_CAPTURE_ALL_METHODS, ) @@ -128,7 +129,7 @@ def validate_response( self, response, error=None, - span_name="HTTP GET", + span_name="GET /", http_method="GET", span_attributes=None, response_headers=None, @@ -284,12 +285,31 @@ def test_wsgi_metrics(self): ) self.assertTrue(number_data_point_seen and histogram_data_point_seen) - def test_default_span_name_missing_request_method(self): - """Test that default span_names with missing request method.""" - self.environ.pop("REQUEST_METHOD") + def test_nonstandard_http_method(self): + self.environ["REQUEST_METHOD"]= "NONSTANDARD" app = otel_wsgi.OpenTelemetryMiddleware(simple_wsgi) response = app(self.environ, self.start_response) - self.validate_response(response, span_name="HTTP", http_method=None) + self.validate_response(response, span_name="UNKNOWN /", http_method="UNKNOWN") + + @mock.patch.dict( + "os.environ", + { + OTEL_PYTHON_INSTRUMENTATION_HTTP_CAPTURE_ALL_METHODS: "1", + }, + ) + def test_nonstandard_http_method_allowed(self): + self.environ["REQUEST_METHOD"]= "NONSTANDARD" + app = otel_wsgi.OpenTelemetryMiddleware(simple_wsgi) + response = app(self.environ, self.start_response) + self.validate_response(response, span_name="NONSTANDARD /", http_method="NONSTANDARD") + + def test_default_span_name_missing_path_info(self): + """Test that default span_names with missing path info.""" + self.environ.pop("PATH_INFO") + method = self.environ.get("REQUEST_METHOD", "").strip() + app = otel_wsgi.OpenTelemetryMiddleware(simple_wsgi) + response = app(self.environ, self.start_response) + self.validate_response(response, span_name=method) class TestWsgiAttributes(unittest.TestCase): @@ -437,10 +457,10 @@ def test_response_attributes(self): self.span.set_attribute.assert_has_calls(expected, any_order=True) def test_credential_removal(self): - self.environ["HTTP_HOST"] = "username:password@httpbin.com" + self.environ["HTTP_HOST"] = "username:password@mock" self.environ["PATH_INFO"] = "/status/200" expected = { - SpanAttributes.HTTP_URL: "http://httpbin.com/status/200", + SpanAttributes.HTTP_URL: "http://mock/status/200", SpanAttributes.NET_HOST_PORT: 80, } self.assertGreaterEqual( @@ -455,7 +475,7 @@ def validate_response( response, exporter, error=None, - span_name="HTTP GET", + span_name="GET /", http_method="GET", ): while True: diff --git a/opentelemetry-contrib-instrumentations/pyproject.toml b/opentelemetry-contrib-instrumentations/pyproject.toml index d05a7712ad..feb229e384 100644 --- a/opentelemetry-contrib-instrumentations/pyproject.toml +++ b/opentelemetry-contrib-instrumentations/pyproject.toml @@ -29,48 +29,50 @@ classifiers = [ "Programming Language :: Python :: 3.11", ] dependencies = [ - "opentelemetry-instrumentation-aio-pika==0.39b0.dev", - "opentelemetry-instrumentation-aiohttp-client==0.39b0.dev", - "opentelemetry-instrumentation-aiopg==0.39b0.dev", - "opentelemetry-instrumentation-asgi==0.39b0.dev", - "opentelemetry-instrumentation-asyncpg==0.39b0.dev", - "opentelemetry-instrumentation-aws-lambda==0.39b0.dev", - "opentelemetry-instrumentation-boto==0.39b0.dev", - "opentelemetry-instrumentation-boto3sqs==0.39b0.dev", - "opentelemetry-instrumentation-botocore==0.39b0.dev", - "opentelemetry-instrumentation-celery==0.39b0.dev", - "opentelemetry-instrumentation-confluent-kafka==0.39b0.dev", - "opentelemetry-instrumentation-dbapi==0.39b0.dev", - "opentelemetry-instrumentation-django==0.39b0.dev", - "opentelemetry-instrumentation-elasticsearch==0.39b0.dev", - "opentelemetry-instrumentation-falcon==0.39b0.dev", - "opentelemetry-instrumentation-fastapi==0.39b0.dev", - "opentelemetry-instrumentation-flask==0.39b0.dev", - "opentelemetry-instrumentation-grpc==0.39b0.dev", - "opentelemetry-instrumentation-httpx==0.39b0.dev", - "opentelemetry-instrumentation-jinja2==0.39b0.dev", - "opentelemetry-instrumentation-kafka-python==0.39b0.dev", - "opentelemetry-instrumentation-logging==0.39b0.dev", - "opentelemetry-instrumentation-mysql==0.39b0.dev", - "opentelemetry-instrumentation-pika==0.39b0.dev", - "opentelemetry-instrumentation-psycopg2==0.39b0.dev", - "opentelemetry-instrumentation-pymemcache==0.39b0.dev", - "opentelemetry-instrumentation-pymongo==0.39b0.dev", - "opentelemetry-instrumentation-pymysql==0.39b0.dev", - "opentelemetry-instrumentation-pyramid==0.39b0.dev", - "opentelemetry-instrumentation-redis==0.39b0.dev", - "opentelemetry-instrumentation-remoulade==0.39b0.dev", - "opentelemetry-instrumentation-requests==0.39b0.dev", - "opentelemetry-instrumentation-sklearn==0.39b0.dev", - "opentelemetry-instrumentation-sqlalchemy==0.39b0.dev", - "opentelemetry-instrumentation-sqlite3==0.39b0.dev", - "opentelemetry-instrumentation-starlette==0.39b0.dev", - "opentelemetry-instrumentation-system-metrics==0.39b0.dev", - "opentelemetry-instrumentation-tornado==0.39b0.dev", - "opentelemetry-instrumentation-tortoiseorm==0.39b0.dev", - "opentelemetry-instrumentation-urllib==0.39b0.dev", - "opentelemetry-instrumentation-urllib3==0.39b0.dev", - "opentelemetry-instrumentation-wsgi==0.39b0.dev", + "opentelemetry-instrumentation-aio-pika==0.42b0.dev", + "opentelemetry-instrumentation-aiohttp-client==0.42b0.dev", + "opentelemetry-instrumentation-aiopg==0.42b0.dev", + "opentelemetry-instrumentation-asgi==0.42b0.dev", + "opentelemetry-instrumentation-asyncpg==0.42b0.dev", + "opentelemetry-instrumentation-aws-lambda==0.42b0.dev", + "opentelemetry-instrumentation-boto==0.42b0.dev", + "opentelemetry-instrumentation-boto3sqs==0.42b0.dev", + "opentelemetry-instrumentation-botocore==0.42b0.dev", + "opentelemetry-instrumentation-cassandra==0.42b0.dev", + "opentelemetry-instrumentation-celery==0.42b0.dev", + "opentelemetry-instrumentation-confluent-kafka==0.42b0.dev", + "opentelemetry-instrumentation-dbapi==0.42b0.dev", + "opentelemetry-instrumentation-django==0.42b0.dev", + "opentelemetry-instrumentation-elasticsearch==0.42b0.dev", + "opentelemetry-instrumentation-falcon==0.42b0.dev", + "opentelemetry-instrumentation-fastapi==0.42b0.dev", + "opentelemetry-instrumentation-flask==0.42b0.dev", + "opentelemetry-instrumentation-grpc==0.42b0.dev", + "opentelemetry-instrumentation-httpx==0.42b0.dev", + "opentelemetry-instrumentation-jinja2==0.42b0.dev", + "opentelemetry-instrumentation-kafka-python==0.42b0.dev", + "opentelemetry-instrumentation-logging==0.42b0.dev", + "opentelemetry-instrumentation-mysql==0.42b0.dev", + "opentelemetry-instrumentation-mysqlclient==0.42b0.dev", + "opentelemetry-instrumentation-pika==0.42b0.dev", + "opentelemetry-instrumentation-psycopg2==0.42b0.dev", + "opentelemetry-instrumentation-pymemcache==0.42b0.dev", + "opentelemetry-instrumentation-pymongo==0.42b0.dev", + "opentelemetry-instrumentation-pymysql==0.42b0.dev", + "opentelemetry-instrumentation-pyramid==0.42b0.dev", + "opentelemetry-instrumentation-redis==0.42b0.dev", + "opentelemetry-instrumentation-remoulade==0.42b0.dev", + "opentelemetry-instrumentation-requests==0.42b0.dev", + "opentelemetry-instrumentation-sklearn==0.42b0.dev", + "opentelemetry-instrumentation-sqlalchemy==0.42b0.dev", + "opentelemetry-instrumentation-sqlite3==0.42b0.dev", + "opentelemetry-instrumentation-starlette==0.42b0.dev", + "opentelemetry-instrumentation-system-metrics==0.42b0.dev", + "opentelemetry-instrumentation-tornado==0.42b0.dev", + "opentelemetry-instrumentation-tortoiseorm==0.42b0.dev", + "opentelemetry-instrumentation-urllib==0.42b0.dev", + "opentelemetry-instrumentation-urllib3==0.42b0.dev", + "opentelemetry-instrumentation-wsgi==0.42b0.dev", ] [project.optional-dependencies] diff --git a/opentelemetry-contrib-instrumentations/src/opentelemetry/contrib-instrumentations/version.py b/opentelemetry-contrib-instrumentations/src/opentelemetry/contrib-instrumentations/version.py index eb62a67e28..c2996671d6 100644 --- a/opentelemetry-contrib-instrumentations/src/opentelemetry/contrib-instrumentations/version.py +++ b/opentelemetry-contrib-instrumentations/src/opentelemetry/contrib-instrumentations/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/opentelemetry-distro/pyproject.toml b/opentelemetry-distro/pyproject.toml index eafbba7130..9898884319 100644 --- a/opentelemetry-distro/pyproject.toml +++ b/opentelemetry-distro/pyproject.toml @@ -24,13 +24,13 @@ classifiers = [ ] dependencies = [ "opentelemetry-api ~= 1.12", - "opentelemetry-instrumentation == 0.39b0.dev", + "opentelemetry-instrumentation == 0.42b0.dev", "opentelemetry-sdk ~= 1.13", ] [project.optional-dependencies] otlp = [ - "opentelemetry-exporter-otlp == 1.18.0.dev", + "opentelemetry-exporter-otlp == 1.21.0.dev", ] test = [] diff --git a/opentelemetry-distro/src/opentelemetry/distro/version.py b/opentelemetry-distro/src/opentelemetry/distro/version.py index eb62a67e28..c2996671d6 100644 --- a/opentelemetry-distro/src/opentelemetry/distro/version.py +++ b/opentelemetry-distro/src/opentelemetry/distro/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/opentelemetry-instrumentation/README.rst b/opentelemetry-instrumentation/README.rst index 95b8fe582b..df21ce5b3d 100644 --- a/opentelemetry-instrumentation/README.rst +++ b/opentelemetry-instrumentation/README.rst @@ -18,12 +18,15 @@ This package provides a couple of commands that help automatically instruments a .. note:: You need to install a distro package to get auto instrumentation working. The ``opentelemetry-distro`` - package contains the default distro and automatically configures some of the common options for users. + package contains the default distro and configurator and automatically configures some of the common options for users. For more info about ``opentelemetry-distro`` check `here `__ :: pip install opentelemetry-distro[otlp] + When creating a custom distro and/or configurator, be sure to add entry points for each under `opentelemetry_distro` and `opentelemetry_configurator` respectfully. + If you have entry points for multiple distros or configurators present in your environment, you should specify the entry point name of the distro and configurator you want to be used via the `OTEL_PYTHON_DISTRO` and `OTEL_PYTHON_CONFIGURATOR` environment variables. + opentelemetry-bootstrap ----------------------- @@ -58,6 +61,8 @@ The command supports the following configuration options as CLI arguments and en * ``--traces_exporter`` or ``OTEL_TRACES_EXPORTER`` * ``--metrics_exporter`` or ``OTEL_METRICS_EXPORTER`` +* ``--distro`` or ``OTEL_PYTHON_DISTRO`` +* ``--configurator`` or ``OTEL_PYTHON_CONFIGURATOR`` Used to specify which trace exporter to use. Can be set to one or more of the well-known exporter names (see below). diff --git a/opentelemetry-instrumentation/src/opentelemetry/instrumentation/auto_instrumentation/_load.py b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/auto_instrumentation/_load.py new file mode 100644 index 0000000000..27b57da3ef --- /dev/null +++ b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/auto_instrumentation/_load.py @@ -0,0 +1,124 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from logging import getLogger +from os import environ + +from pkg_resources import iter_entry_points + +from opentelemetry.instrumentation.dependencies import ( + get_dist_dependency_conflicts, +) +from opentelemetry.instrumentation.distro import BaseDistro, DefaultDistro +from opentelemetry.instrumentation.environment_variables import ( + OTEL_PYTHON_CONFIGURATOR, + OTEL_PYTHON_DISABLED_INSTRUMENTATIONS, + OTEL_PYTHON_DISTRO, +) +from opentelemetry.instrumentation.version import __version__ + +_logger = getLogger(__name__) + + +def _load_distro() -> BaseDistro: + distro_name = environ.get(OTEL_PYTHON_DISTRO, None) + for entry_point in iter_entry_points("opentelemetry_distro"): + try: + # If no distro is specified, use first to come up. + if distro_name is None or distro_name == entry_point.name: + distro = entry_point.load()() + if not isinstance(distro, BaseDistro): + _logger.debug( + "%s is not an OpenTelemetry Distro. Skipping", + entry_point.name, + ) + continue + _logger.debug( + "Distribution %s will be configured", entry_point.name + ) + return distro + except Exception as exc: # pylint: disable=broad-except + _logger.exception( + "Distribution %s configuration failed", entry_point.name + ) + raise exc + return DefaultDistro() + + +def _load_instrumentors(distro): + package_to_exclude = environ.get(OTEL_PYTHON_DISABLED_INSTRUMENTATIONS, []) + if isinstance(package_to_exclude, str): + package_to_exclude = package_to_exclude.split(",") + # to handle users entering "requests , flask" or "requests, flask" with spaces + package_to_exclude = [x.strip() for x in package_to_exclude] + + for entry_point in iter_entry_points("opentelemetry_pre_instrument"): + entry_point.load()() + + for entry_point in iter_entry_points("opentelemetry_instrumentor"): + if entry_point.name in package_to_exclude: + _logger.debug( + "Instrumentation skipped for library %s", entry_point.name + ) + continue + + try: + conflict = get_dist_dependency_conflicts(entry_point.dist) + if conflict: + _logger.debug( + "Skipping instrumentation %s: %s", + entry_point.name, + conflict, + ) + continue + + # tell instrumentation to not run dep checks again as we already did it above + distro.load_instrumentor(entry_point, skip_dep_check=True) + _logger.debug("Instrumented %s", entry_point.name) + except Exception as exc: # pylint: disable=broad-except + _logger.exception("Instrumenting of %s failed", entry_point.name) + raise exc + + for entry_point in iter_entry_points("opentelemetry_post_instrument"): + entry_point.load()() + + +def _load_configurators(): + configurator_name = environ.get(OTEL_PYTHON_CONFIGURATOR, None) + configured = None + for entry_point in iter_entry_points("opentelemetry_configurator"): + if configured is not None: + _logger.warning( + "Configuration of %s not loaded, %s already loaded", + entry_point.name, + configured, + ) + continue + try: + if ( + configurator_name is None + or configurator_name == entry_point.name + ): + entry_point.load()().configure(auto_instrumentation_version=__version__) # type: ignore + configured = entry_point.name + else: + _logger.warning( + "Configuration of %s not loaded because %s is set by %s", + entry_point.name, + configurator_name, + OTEL_PYTHON_CONFIGURATOR, + ) + except Exception as exc: # pylint: disable=broad-except + _logger.exception("Configuration of %s failed", entry_point.name) + raise exc diff --git a/opentelemetry-instrumentation/src/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py index 9504e359af..912675f1b7 100644 --- a/opentelemetry-instrumentation/src/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py +++ b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py @@ -16,99 +16,16 @@ from os import environ from os.path import abspath, dirname, pathsep -from pkg_resources import iter_entry_points - -from opentelemetry.instrumentation.dependencies import ( - get_dist_dependency_conflicts, -) -from opentelemetry.instrumentation.distro import BaseDistro, DefaultDistro -from opentelemetry.instrumentation.environment_variables import ( - OTEL_PYTHON_DISABLED_INSTRUMENTATIONS, +from opentelemetry.instrumentation.auto_instrumentation._load import ( + _load_configurators, + _load_distro, + _load_instrumentors, ) from opentelemetry.instrumentation.utils import _python_path_without_directory -from opentelemetry.instrumentation.version import __version__ logger = getLogger(__name__) -def _load_distros() -> BaseDistro: - for entry_point in iter_entry_points("opentelemetry_distro"): - try: - distro = entry_point.load()() - if not isinstance(distro, BaseDistro): - logger.debug( - "%s is not an OpenTelemetry Distro. Skipping", - entry_point.name, - ) - continue - logger.debug( - "Distribution %s will be configured", entry_point.name - ) - return distro - except Exception as exc: # pylint: disable=broad-except - logger.exception( - "Distribution %s configuration failed", entry_point.name - ) - raise exc - return DefaultDistro() - - -def _load_instrumentors(distro): - package_to_exclude = environ.get(OTEL_PYTHON_DISABLED_INSTRUMENTATIONS, []) - if isinstance(package_to_exclude, str): - package_to_exclude = package_to_exclude.split(",") - # to handle users entering "requests , flask" or "requests, flask" with spaces - package_to_exclude = [x.strip() for x in package_to_exclude] - - for entry_point in iter_entry_points("opentelemetry_pre_instrument"): - entry_point.load()() - - for entry_point in iter_entry_points("opentelemetry_instrumentor"): - if entry_point.name in package_to_exclude: - logger.debug( - "Instrumentation skipped for library %s", entry_point.name - ) - continue - - try: - conflict = get_dist_dependency_conflicts(entry_point.dist) - if conflict: - logger.debug( - "Skipping instrumentation %s: %s", - entry_point.name, - conflict, - ) - continue - - # tell instrumentation to not run dep checks again as we already did it above - distro.load_instrumentor(entry_point, skip_dep_check=True) - logger.debug("Instrumented %s", entry_point.name) - except Exception as exc: # pylint: disable=broad-except - logger.exception("Instrumenting of %s failed", entry_point.name) - raise exc - - for entry_point in iter_entry_points("opentelemetry_post_instrument"): - entry_point.load()() - - -def _load_configurators(): - configured = None - for entry_point in iter_entry_points("opentelemetry_configurator"): - if configured is not None: - logger.warning( - "Configuration of %s not loaded, %s already loaded", - entry_point.name, - configured, - ) - continue - try: - entry_point.load()().configure(auto_instrumentation_version=__version__) # type: ignore - configured = entry_point.name - except Exception as exc: # pylint: disable=broad-except - logger.exception("Configuration of %s failed", entry_point.name) - raise exc - - def initialize(): # prevents auto-instrumentation of subprocesses if code execs another python process environ["PYTHONPATH"] = _python_path_without_directory( @@ -116,7 +33,7 @@ def initialize(): ) try: - distro = _load_distros() + distro = _load_distro() distro.configure() _load_configurators() _load_instrumentors(distro) diff --git a/opentelemetry-instrumentation/src/opentelemetry/instrumentation/bootstrap_gen.py b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/bootstrap_gen.py index 05c77b9fea..8d856abf65 100644 --- a/opentelemetry-instrumentation/src/opentelemetry/instrumentation/bootstrap_gen.py +++ b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/bootstrap_gen.py @@ -18,158 +18,170 @@ libraries = { "aio_pika": { "library": "aio_pika >= 7.2.0, < 10.0.0", - "instrumentation": "opentelemetry-instrumentation-aio-pika==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-aio-pika==0.42b0.dev", }, "aiohttp": { "library": "aiohttp ~= 3.0", - "instrumentation": "opentelemetry-instrumentation-aiohttp-client==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-aiohttp-client==0.42b0.dev", }, "aiopg": { "library": "aiopg >= 0.13.0, < 2.0.0", - "instrumentation": "opentelemetry-instrumentation-aiopg==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-aiopg==0.42b0.dev", }, "asgiref": { "library": "asgiref ~= 3.0", - "instrumentation": "opentelemetry-instrumentation-asgi==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-asgi==0.42b0.dev", }, "asyncpg": { "library": "asyncpg >= 0.12.0", - "instrumentation": "opentelemetry-instrumentation-asyncpg==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-asyncpg==0.42b0.dev", }, "boto": { "library": "boto~=2.0", - "instrumentation": "opentelemetry-instrumentation-boto==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-boto==0.42b0.dev", }, "boto3": { "library": "boto3 ~= 1.0", - "instrumentation": "opentelemetry-instrumentation-boto3sqs==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-boto3sqs==0.42b0.dev", }, "botocore": { "library": "botocore ~= 1.0", - "instrumentation": "opentelemetry-instrumentation-botocore==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-botocore==0.42b0.dev", + }, + "cassandra-driver": { + "library": "cassandra-driver ~= 3.25", + "instrumentation": "opentelemetry-instrumentation-cassandra==0.42b0.dev", + }, + "scylla-driver": { + "library": "scylla-driver ~= 3.25", + "instrumentation": "opentelemetry-instrumentation-cassandra==0.42b0.dev", }, "celery": { "library": "celery >= 4.0, < 6.0", - "instrumentation": "opentelemetry-instrumentation-celery==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-celery==0.42b0.dev", }, "confluent-kafka": { - "library": "confluent-kafka >= 1.8.2, < 2.0.0", - "instrumentation": "opentelemetry-instrumentation-confluent-kafka==0.39b0.dev", + "library": "confluent-kafka >= 1.8.2, <= 2.2.0", + "instrumentation": "opentelemetry-instrumentation-confluent-kafka==0.42b0.dev", }, "django": { "library": "django >= 1.10", - "instrumentation": "opentelemetry-instrumentation-django==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-django==0.42b0.dev", }, "elasticsearch": { "library": "elasticsearch >= 2.0", - "instrumentation": "opentelemetry-instrumentation-elasticsearch==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-elasticsearch==0.42b0.dev", }, "falcon": { "library": "falcon >= 1.4.1, < 4.0.0", - "instrumentation": "opentelemetry-instrumentation-falcon==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-falcon==0.42b0.dev", }, "fastapi": { "library": "fastapi ~= 0.58", - "instrumentation": "opentelemetry-instrumentation-fastapi==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-fastapi==0.42b0.dev", }, "flask": { "library": "flask >= 1.0, < 3.0", - "instrumentation": "opentelemetry-instrumentation-flask==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-flask==0.42b0.dev", }, "grpcio": { "library": "grpcio ~= 1.27", - "instrumentation": "opentelemetry-instrumentation-grpc==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-grpc==0.42b0.dev", }, "httpx": { - "library": "httpx >= 0.18.0, <= 0.23.0", - "instrumentation": "opentelemetry-instrumentation-httpx==0.39b0.dev", + "library": "httpx >= 0.18.0", + "instrumentation": "opentelemetry-instrumentation-httpx==0.42b0.dev", }, "jinja2": { "library": "jinja2 >= 2.7, < 4.0", - "instrumentation": "opentelemetry-instrumentation-jinja2==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-jinja2==0.42b0.dev", }, "kafka-python": { "library": "kafka-python >= 2.0", - "instrumentation": "opentelemetry-instrumentation-kafka-python==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-kafka-python==0.42b0.dev", }, "mysql-connector-python": { "library": "mysql-connector-python ~= 8.0", - "instrumentation": "opentelemetry-instrumentation-mysql==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-mysql==0.42b0.dev", + }, + "mysqlclient": { + "library": "mysqlclient < 3", + "instrumentation": "opentelemetry-instrumentation-mysqlclient==0.42b0.dev", }, "pika": { "library": "pika >= 0.12.0", - "instrumentation": "opentelemetry-instrumentation-pika==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-pika==0.42b0.dev", }, "psycopg2": { "library": "psycopg2 >= 2.7.3.1", - "instrumentation": "opentelemetry-instrumentation-psycopg2==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-psycopg2==0.42b0.dev", }, "pymemcache": { "library": "pymemcache >= 1.3.5, < 5", - "instrumentation": "opentelemetry-instrumentation-pymemcache==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-pymemcache==0.42b0.dev", }, "pymongo": { "library": "pymongo >= 3.1, < 5.0", - "instrumentation": "opentelemetry-instrumentation-pymongo==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-pymongo==0.42b0.dev", }, "PyMySQL": { "library": "PyMySQL < 2", - "instrumentation": "opentelemetry-instrumentation-pymysql==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-pymysql==0.42b0.dev", }, "pyramid": { "library": "pyramid >= 1.7", - "instrumentation": "opentelemetry-instrumentation-pyramid==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-pyramid==0.42b0.dev", }, "redis": { "library": "redis >= 2.6", - "instrumentation": "opentelemetry-instrumentation-redis==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-redis==0.42b0.dev", }, "remoulade": { "library": "remoulade >= 0.50", - "instrumentation": "opentelemetry-instrumentation-remoulade==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-remoulade==0.42b0.dev", }, "requests": { "library": "requests ~= 2.0", - "instrumentation": "opentelemetry-instrumentation-requests==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-requests==0.42b0.dev", }, "scikit-learn": { "library": "scikit-learn ~= 0.24.0", - "instrumentation": "opentelemetry-instrumentation-sklearn==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-sklearn==0.42b0.dev", }, "sqlalchemy": { "library": "sqlalchemy", - "instrumentation": "opentelemetry-instrumentation-sqlalchemy==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-sqlalchemy==0.42b0.dev", }, "starlette": { "library": "starlette ~= 0.13.0", - "instrumentation": "opentelemetry-instrumentation-starlette==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-starlette==0.42b0.dev", }, "psutil": { "library": "psutil >= 5", - "instrumentation": "opentelemetry-instrumentation-system-metrics==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-system-metrics==0.42b0.dev", }, "tornado": { "library": "tornado >= 5.1.1", - "instrumentation": "opentelemetry-instrumentation-tornado==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-tornado==0.42b0.dev", }, "tortoise-orm": { "library": "tortoise-orm >= 0.17.0", - "instrumentation": "opentelemetry-instrumentation-tortoiseorm==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-tortoiseorm==0.42b0.dev", }, "pydantic": { "library": "pydantic >= 1.10.2", - "instrumentation": "opentelemetry-instrumentation-tortoiseorm==0.39b0.dev", + "instrumentation": "opentelemetry-instrumentation-tortoiseorm==0.42b0.dev", }, "urllib3": { - "library": "urllib3 >= 1.0.0, < 2.0.0", - "instrumentation": "opentelemetry-instrumentation-urllib3==0.39b0.dev", + "library": "urllib3 >= 1.0.0, < 3.0.0", + "instrumentation": "opentelemetry-instrumentation-urllib3==0.42b0.dev", }, } default_instrumentations = [ - "opentelemetry-instrumentation-aws-lambda==0.39b0.dev", - "opentelemetry-instrumentation-dbapi==0.39b0.dev", - "opentelemetry-instrumentation-logging==0.39b0.dev", - "opentelemetry-instrumentation-sqlite3==0.39b0.dev", - "opentelemetry-instrumentation-urllib==0.39b0.dev", - "opentelemetry-instrumentation-wsgi==0.39b0.dev", + "opentelemetry-instrumentation-aws-lambda==0.42b0.dev", + "opentelemetry-instrumentation-dbapi==0.42b0.dev", + "opentelemetry-instrumentation-logging==0.42b0.dev", + "opentelemetry-instrumentation-sqlite3==0.42b0.dev", + "opentelemetry-instrumentation-urllib==0.42b0.dev", + "opentelemetry-instrumentation-wsgi==0.42b0.dev", ] diff --git a/opentelemetry-instrumentation/src/opentelemetry/instrumentation/environment_variables.py b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/environment_variables.py index ad28f06859..7886779632 100644 --- a/opentelemetry-instrumentation/src/opentelemetry/instrumentation/environment_variables.py +++ b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/environment_variables.py @@ -16,3 +16,13 @@ """ .. envvar:: OTEL_PYTHON_DISABLED_INSTRUMENTATIONS """ + +OTEL_PYTHON_DISTRO = "OTEL_PYTHON_DISTRO" +""" +.. envvar:: OTEL_PYTHON_DISTRO +""" + +OTEL_PYTHON_CONFIGURATOR = "OTEL_PYTHON_CONFIGURATOR" +""" +.. envvar:: OTEL_PYTHON_CONFIGURATOR +""" diff --git a/opentelemetry-instrumentation/src/opentelemetry/instrumentation/utils.py b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/utils.py index 3022e6ddd0..35a55a1279 100644 --- a/opentelemetry-instrumentation/src/opentelemetry/instrumentation/utils.py +++ b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/utils.py @@ -95,7 +95,7 @@ def _start_internal_or_server_span( Args: tracer : tracer in use by given instrumentation library - name (string): name of the span + span_name (string): name of the span start_time : start time of the span context_carrier : object which contains values that are used to construct a Context. This object diff --git a/opentelemetry-instrumentation/src/opentelemetry/instrumentation/version.py b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/version.py index eb62a67e28..c2996671d6 100644 --- a/opentelemetry-instrumentation/src/opentelemetry/instrumentation/version.py +++ b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/opentelemetry-instrumentation/tests/auto_instrumentation/test_load.py b/opentelemetry-instrumentation/tests/auto_instrumentation/test_load.py new file mode 100644 index 0000000000..1e2a851e48 --- /dev/null +++ b/opentelemetry-instrumentation/tests/auto_instrumentation/test_load.py @@ -0,0 +1,312 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# type: ignore + +from unittest import TestCase +from unittest.mock import Mock, call, patch + +from opentelemetry.instrumentation.auto_instrumentation import _load +from opentelemetry.instrumentation.environment_variables import ( + OTEL_PYTHON_CONFIGURATOR, + OTEL_PYTHON_DISABLED_INSTRUMENTATIONS, + OTEL_PYTHON_DISTRO, +) +from opentelemetry.instrumentation.version import __version__ + + +class TestLoad(TestCase): + @patch.dict( + "os.environ", {OTEL_PYTHON_CONFIGURATOR: "custom_configurator2"} + ) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.iter_entry_points" + ) + def test_load_configurators(self, iter_mock): + # Add multiple entry points but only specify the 2nd in the environment variable. + ep_mock1 = Mock() + ep_mock1.name = "custom_configurator1" + configurator_mock1 = Mock() + ep_mock1.load.return_value = configurator_mock1 + ep_mock2 = Mock() + ep_mock2.name = "custom_configurator2" + configurator_mock2 = Mock() + ep_mock2.load.return_value = configurator_mock2 + ep_mock3 = Mock() + ep_mock3.name = "custom_configurator3" + configurator_mock3 = Mock() + ep_mock3.load.return_value = configurator_mock3 + + iter_mock.return_value = (ep_mock1, ep_mock2, ep_mock3) + _load._load_configurators() + configurator_mock1.assert_not_called() + configurator_mock2().configure.assert_called_once_with( + auto_instrumentation_version=__version__ + ) + configurator_mock3.assert_not_called() + + @patch.dict( + "os.environ", {OTEL_PYTHON_CONFIGURATOR: "custom_configurator2"} + ) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.iter_entry_points" + ) + def test_load_configurators_no_ep( + self, + iter_mock, + ): + iter_mock.return_value = () + # Confirm method does not crash if not entry points exist. + _load._load_configurators() + + @patch.dict( + "os.environ", {OTEL_PYTHON_CONFIGURATOR: "custom_configurator2"} + ) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.iter_entry_points" + ) + def test_load_configurators_error(self, iter_mock): + # Add multiple entry points but only specify the 2nd in the environment variable. + ep_mock1 = Mock() + ep_mock1.name = "custom_configurator1" + configurator_mock1 = Mock() + ep_mock1.load.return_value = configurator_mock1 + ep_mock2 = Mock() + ep_mock2.name = "custom_configurator2" + configurator_mock2 = Mock() + configurator_mock2().configure.side_effect = Exception() + ep_mock2.load.return_value = configurator_mock2 + ep_mock3 = Mock() + ep_mock3.name = "custom_configurator3" + configurator_mock3 = Mock() + ep_mock3.load.return_value = configurator_mock3 + + iter_mock.return_value = (ep_mock1, ep_mock2, ep_mock3) + # Confirm failed configuration raises exception. + self.assertRaises(Exception, _load._load_configurators) + + @patch.dict("os.environ", {OTEL_PYTHON_DISTRO: "custom_distro2"}) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.isinstance" + ) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.iter_entry_points" + ) + def test_load_distro(self, iter_mock, isinstance_mock): + # Add multiple entry points but only specify the 2nd in the environment variable. + ep_mock1 = Mock() + ep_mock1.name = "custom_distro1" + distro_mock1 = Mock() + ep_mock1.load.return_value = distro_mock1 + ep_mock2 = Mock() + ep_mock2.name = "custom_distro2" + distro_mock2 = Mock() + ep_mock2.load.return_value = distro_mock2 + ep_mock3 = Mock() + ep_mock3.name = "custom_distro3" + distro_mock3 = Mock() + ep_mock3.load.return_value = distro_mock3 + + iter_mock.return_value = (ep_mock1, ep_mock2, ep_mock3) + # Mock entry points to be instances of BaseDistro. + isinstance_mock.return_value = True + self.assertEqual( + _load._load_distro(), + distro_mock2(), + ) + + @patch.dict("os.environ", {OTEL_PYTHON_DISTRO: "custom_distro2"}) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.isinstance" + ) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.DefaultDistro" + ) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.iter_entry_points" + ) + def test_load_distro_not_distro( + self, iter_mock, default_distro_mock, isinstance_mock + ): + # Add multiple entry points but only specify the 2nd in the environment variable. + ep_mock1 = Mock() + ep_mock1.name = "custom_distro1" + distro_mock1 = Mock() + ep_mock1.load.return_value = distro_mock1 + ep_mock2 = Mock() + ep_mock2.name = "custom_distro2" + distro_mock2 = Mock() + ep_mock2.load.return_value = distro_mock2 + ep_mock3 = Mock() + ep_mock3.name = "custom_distro3" + distro_mock3 = Mock() + ep_mock3.load.return_value = distro_mock3 + + iter_mock.return_value = (ep_mock1, ep_mock2, ep_mock3) + # Confirm default distro is used if specified entry point is not a BaseDistro + isinstance_mock.return_value = False + self.assertEqual( + _load._load_distro(), + default_distro_mock(), + ) + + @patch.dict("os.environ", {OTEL_PYTHON_DISTRO: "custom_distro2"}) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.DefaultDistro" + ) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.iter_entry_points" + ) + def test_load_distro_no_ep(self, iter_mock, default_distro_mock): + iter_mock.return_value = () + # Confirm default distro is used if there are no entry points. + self.assertEqual( + _load._load_distro(), + default_distro_mock(), + ) + + @patch.dict("os.environ", {OTEL_PYTHON_DISTRO: "custom_distro2"}) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.isinstance" + ) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.iter_entry_points" + ) + def test_load_distro_error(self, iter_mock, isinstance_mock): + ep_mock1 = Mock() + ep_mock1.name = "custom_distro1" + distro_mock1 = Mock() + ep_mock1.load.return_value = distro_mock1 + ep_mock2 = Mock() + ep_mock2.name = "custom_distro2" + distro_mock2 = Mock() + distro_mock2.side_effect = Exception() + ep_mock2.load.return_value = distro_mock2 + ep_mock3 = Mock() + ep_mock3.name = "custom_distro3" + distro_mock3 = Mock() + ep_mock3.load.return_value = distro_mock3 + + iter_mock.return_value = (ep_mock1, ep_mock2, ep_mock3) + isinstance_mock.return_value = True + # Confirm method raises exception if it fails to load a distro. + self.assertRaises(Exception, _load._load_distro) + + @patch.dict( + "os.environ", + {OTEL_PYTHON_DISABLED_INSTRUMENTATIONS: " instr1 , instr3 "}, + ) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.get_dist_dependency_conflicts" + ) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.iter_entry_points" + ) + def test_load_instrumentors(self, iter_mock, dep_mock): + # Mock opentelemetry_pre_instrument entry points + pre_ep_mock1 = Mock() + pre_ep_mock1.name = "pre1" + pre_mock1 = Mock() + pre_ep_mock1.load.return_value = pre_mock1 + + pre_ep_mock2 = Mock() + pre_ep_mock2.name = "pre2" + pre_mock2 = Mock() + pre_ep_mock2.load.return_value = pre_mock2 + + # Mock opentelemetry_instrumentor entry points + ep_mock1 = Mock() + ep_mock1.name = "instr1" + + ep_mock2 = Mock() + ep_mock2.name = "instr2" + + ep_mock3 = Mock() + ep_mock3.name = "instr3" + + ep_mock4 = Mock() + ep_mock4.name = "instr4" + + # Mock opentelemetry_instrumentor entry points + post_ep_mock1 = Mock() + post_ep_mock1.name = "post1" + post_mock1 = Mock() + post_ep_mock1.load.return_value = post_mock1 + + post_ep_mock2 = Mock() + post_ep_mock2.name = "post2" + post_mock2 = Mock() + post_ep_mock2.load.return_value = post_mock2 + + distro_mock = Mock() + + # Mock entry points in order + iter_mock.side_effect = [ + (pre_ep_mock1, pre_ep_mock2), + (ep_mock1, ep_mock2, ep_mock3, ep_mock4), + (post_ep_mock1, post_ep_mock2), + ] + # No dependency conflict + dep_mock.return_value = None + _load._load_instrumentors(distro_mock) + # All opentelemetry_pre_instrument entry points should be loaded + pre_mock1.assert_called_once() + pre_mock2.assert_called_once() + self.assertEqual(iter_mock.call_count, 3) + # Only non-disabled instrumentations should be loaded + distro_mock.load_instrumentor.assert_has_calls( + [ + call(ep_mock2, skip_dep_check=True), + call(ep_mock4, skip_dep_check=True), + ] + ) + self.assertEqual(distro_mock.load_instrumentor.call_count, 2) + # All opentelemetry_post_instrument entry points should be loaded + post_mock1.assert_called_once() + post_mock2.assert_called_once() + + @patch.dict( + "os.environ", + {OTEL_PYTHON_DISABLED_INSTRUMENTATIONS: " instr1 , instr3 "}, + ) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.get_dist_dependency_conflicts" + ) + @patch( + "opentelemetry.instrumentation.auto_instrumentation._load.iter_entry_points" + ) + def test_load_instrumentors_dep_conflict(self, iter_mock, dep_mock): + ep_mock1 = Mock() + ep_mock1.name = "instr1" + + ep_mock2 = Mock() + ep_mock2.name = "instr2" + + ep_mock3 = Mock() + ep_mock3.name = "instr3" + + ep_mock4 = Mock() + ep_mock4.name = "instr4" + + distro_mock = Mock() + + iter_mock.return_value = (ep_mock1, ep_mock2, ep_mock3, ep_mock4) + # If a dependency conflict is raised, that instrumentation should not be loaded, but others still should. + dep_mock.side_effect = [None, "DependencyConflict"] + _load._load_instrumentors(distro_mock) + distro_mock.load_instrumentor.assert_has_calls( + [ + call(ep_mock2, skip_dep_check=True), + ] + ) + distro_mock.load_instrumentor.assert_called_once() diff --git a/opentelemetry-instrumentation/tests/test_run.py b/opentelemetry-instrumentation/tests/auto_instrumentation/test_run.py similarity index 100% rename from opentelemetry-instrumentation/tests/test_run.py rename to opentelemetry-instrumentation/tests/auto_instrumentation/test_run.py diff --git a/propagator/opentelemetry-propagator-ot-trace/src/opentelemetry/propagators/ot_trace/version.py b/propagator/opentelemetry-propagator-ot-trace/src/opentelemetry/propagators/ot_trace/version.py index eb62a67e28..c2996671d6 100644 --- a/propagator/opentelemetry-propagator-ot-trace/src/opentelemetry/propagators/ot_trace/version.py +++ b/propagator/opentelemetry-propagator-ot-trace/src/opentelemetry/propagators/ot_trace/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/resource/opentelemetry-resource-detector-azure/LICENSE b/resource/opentelemetry-resource-detector-azure/LICENSE new file mode 100644 index 0000000000..1ef7dad2c5 --- /dev/null +++ b/resource/opentelemetry-resource-detector-azure/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright The OpenTelemetry Authors + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/resource/opentelemetry-resource-detector-azure/MANIFEST.rst b/resource/opentelemetry-resource-detector-azure/MANIFEST.rst new file mode 100644 index 0000000000..2906eeef0f --- /dev/null +++ b/resource/opentelemetry-resource-detector-azure/MANIFEST.rst @@ -0,0 +1,9 @@ +graft src +graft tests +global-exclude *.pyc +global-exclude *.pyo +global-exclude __pycache__/* +include CHANGELOG.md +include MANIFEST.in +include README.rst +include LICENSE \ No newline at end of file diff --git a/resource/opentelemetry-resource-detector-azure/README.rst b/resource/opentelemetry-resource-detector-azure/README.rst new file mode 100644 index 0000000000..6a376534ad --- /dev/null +++ b/resource/opentelemetry-resource-detector-azure/README.rst @@ -0,0 +1,82 @@ +OpenTelemetry Resource detectors for Azure +========================================== + +|pypi| + +.. |pypi| image:: https://badge.fury.io/py/opentelemetry-resource-detector-azure.svg + :target: https://pypi.org/project/opentelemetry-resource-detector-azure/ + +This library contains OpenTelemetry `Resource Detectors `_ for the following Azure resources: + * `Azure App Service `_ + * `Azure Virtual Machines `_ + +Installation +------------ + +:: + + pip install opentelemetry-resource-detector-azure + +--------------------------- + +Usage example for ``opentelemetry-resource-detector-azure`` + +.. code-block:: python + + from opentelemetry import trace + from opentelemetry.sdk.trace import TracerProvider + from opentelemetry.resource.detector.azure.app_service import ( + AzureAppServiceResourceDetector, + AzureVMResourceDetector, + ) + from opentelemetry.resource.detector.azure.vm import ( + AzureVMResourceDetector, + ) + from opentelemetry.sdk.resources import get_aggregated_resources + + + trace.set_tracer_provider( + TracerProvider( + resource=get_aggregated_resources( + [ + AzureAppServiceResourceDetector(), + AzureVMResourceDetector(), + ] + ), + ) + ) + +Mappings +-------- + +The Azure App Service Resource Detector sets the following Resource Attributes: + * ``service.name`` set to the value of the ``WEBSITE_SITE_NAME`` environment variable. + * ``cloud.platform`` set to ``azure_app_service``. + * ``cloud.provider`` set to ``azure``. + * ``cloud.resource_id`` set using the ``WEBSITE_RESOURCE_GROUP``, ``WEBSITE_OWNER_NAME``, and ``WEBSITE_SITE_NAME`` environment variables. + * ``cloud.region`` set to the value of the ``REGION_NAME`` environment variable. + * ``deployment.environment`` set to the value of the ``WEBSITE_SLOT_NAME`` environment variable. + * ``host.id`` set to the value of the ``WEBSITE_HOSTNAME`` environment variable. + * ``service.instance.id`` set to the value of the ``WEBSITE_INSTANCE_ID`` environment variable. + * ``azure.app.service.stamp`` set to the value of the ``WEBSITE_HOME_STAMPNAME`` environment variable. + +The Azure VM Resource Detector sets the following Resource Attributes according to the response from the `Azure Metadata Service `_: + * ``azure.vm.scaleset.name`` set to the value of the ``vmScaleSetName`` field. + * ``azure.vm.sku`` set to the value of the ``sku`` field. + * ``cloud.platform`` set to the value of the ``azure_vm``. + * ``cloud.provider`` set to the value of the ``azure``. + * ``cloud.region`` set to the value of the ``location`` field. + * ``cloud.resource_id`` set to the value of the ``resourceId`` field. + * ``host.id`` set to the value of the ``vmId`` field. + * ``host.name`` set to the value of the ``name`` field. + * ``host.type`` set to the value of the ``vmSize`` field. + * ``os.type`` set to the value of the ``osType`` field. + * ``os.version`` set to the value of the ``version`` field. + * ``service.instance.id`` set to the value of the ``vmId`` field. + +For more information, see the `Semantic Conventions for Cloud Resource Attributes `_. + +References +---------- + +* `OpenTelemetry Project `_ \ No newline at end of file diff --git a/resource/opentelemetry-resource-detector-azure/pyproject.toml b/resource/opentelemetry-resource-detector-azure/pyproject.toml new file mode 100644 index 0000000000..c4ae6fc191 --- /dev/null +++ b/resource/opentelemetry-resource-detector-azure/pyproject.toml @@ -0,0 +1,51 @@ +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[project] +name = "opentelemetry-resource-detector-azure" +dynamic = ["version"] +description = "Azure Resource Detector for OpenTelemetry" +readme = "README.rst" +license = "Apache-2.0" +requires-python = ">=3.7" +authors = [ + { name = "OpenTelemetry Authors", email = "cncf-opentelemetry-contributors@lists.cncf.io" }, +] +classifiers = [ + "Development Status :: 4 - Beta", + "Intended Audience :: Developers", + "License :: OSI Approved :: Apache Software License", + "Programming Language :: Python", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.7", + "Programming Language :: Python :: 3.8", + "Programming Language :: Python :: 3.9", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", +] +dependencies = [ + "opentelemetry-sdk ~= 1.19", +] + +[project.optional-dependencies] +test = [] + +[project.entry-points.opentelemetry_resource_detector] +azure_app_service = "opentelemetry.resource.detector.azure.app_service:AzureAppServiceResourceDetector" +azure_vm = "opentelemetry.resource.detector.azure.vm:AzureVMResourceDetector" + +[project.urls] +Homepage = "https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/resource/opentelemetry-resource-detector-azure" + +[tool.hatch.version] +path = "src/opentelemetry/resource/detector/azure/version.py" + +[tool.hatch.build.targets.sdist] +include = [ + "/src", + "/tests", +] + +[tool.hatch.build.targets.wheel] +packages = ["src/opentelemetry"] diff --git a/resource/opentelemetry-resource-detector-azure/src/opentelemetry/resource/detector/azure/app_service.py b/resource/opentelemetry-resource-detector-azure/src/opentelemetry/resource/detector/azure/app_service.py new file mode 100644 index 0000000000..ea0959cb93 --- /dev/null +++ b/resource/opentelemetry-resource-detector-azure/src/opentelemetry/resource/detector/azure/app_service.py @@ -0,0 +1,75 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from os import environ + +from opentelemetry.sdk.resources import ResourceDetector, Resource +from opentelemetry.semconv.resource import ResourceAttributes, CloudPlatformValues, CloudProviderValues + +_AZURE_APP_SERVICE_STAMP_RESOURCE_ATTRIBUTE = "azure.app.service.stamp" +# TODO: Remove once this resource attribute is no longer missing from SDK +_CLOUD_RESOURCE_ID_RESOURCE_ATTRIBUTE = "cloud.resource_id" +_REGION_NAME = "REGION_NAME" +_WEBSITE_HOME_STAMPNAME = "WEBSITE_HOME_STAMPNAME" +_WEBSITE_HOSTNAME = "WEBSITE_HOSTNAME" +_WEBSITE_INSTANCE_ID = "WEBSITE_INSTANCE_ID" +_WEBSITE_OWNER_NAME = "WEBSITE_OWNER_NAME" +_WEBSITE_RESOURCE_GROUP = "WEBSITE_RESOURCE_GROUP" +_WEBSITE_SITE_NAME = "WEBSITE_SITE_NAME" +_WEBSITE_SLOT_NAME = "WEBSITE_SLOT_NAME" + + +_APP_SERVICE_ATTRIBUTE_ENV_VARS = { + ResourceAttributes.CLOUD_REGION: _REGION_NAME, + ResourceAttributes.DEPLOYMENT_ENVIRONMENT: _WEBSITE_SLOT_NAME, + ResourceAttributes.HOST_ID: _WEBSITE_HOSTNAME, + ResourceAttributes.SERVICE_INSTANCE_ID: _WEBSITE_INSTANCE_ID, + _AZURE_APP_SERVICE_STAMP_RESOURCE_ATTRIBUTE: _WEBSITE_HOME_STAMPNAME, +} + +class AzureAppServiceResourceDetector(ResourceDetector): + def detect(self) -> Resource: + attributes = {} + website_site_name = environ.get(_WEBSITE_SITE_NAME) + if website_site_name: + attributes[ResourceAttributes.SERVICE_NAME] = website_site_name + attributes[ResourceAttributes.CLOUD_PROVIDER] = CloudProviderValues.AZURE.value + attributes[ResourceAttributes.CLOUD_PLATFORM] = CloudPlatformValues.AZURE_APP_SERVICE.value + + azure_resource_uri = _get_azure_resource_uri(website_site_name) + if azure_resource_uri: + attributes[_CLOUD_RESOURCE_ID_RESOURCE_ATTRIBUTE] = azure_resource_uri + for (key, env_var) in _APP_SERVICE_ATTRIBUTE_ENV_VARS.items(): + value = environ.get(env_var) + if value: + attributes[key] = value + + return Resource(attributes) + +def _get_azure_resource_uri(website_site_name): + website_resource_group = environ.get(_WEBSITE_RESOURCE_GROUP) + website_owner_name = environ.get(_WEBSITE_OWNER_NAME) + + subscription_id = website_owner_name + if website_owner_name and '+' in website_owner_name: + subscription_id = website_owner_name[0:website_owner_name.index('+')] + + if not (website_resource_group and subscription_id): + return None + + return "/subscriptions/%s/resourceGroups/%s/providers/Microsoft.Web/sites/%s" % ( + subscription_id, + website_resource_group, + website_site_name, + ) diff --git a/resource/opentelemetry-resource-detector-azure/src/opentelemetry/resource/detector/azure/version.py b/resource/opentelemetry-resource-detector-azure/src/opentelemetry/resource/detector/azure/version.py new file mode 100644 index 0000000000..5fd301e2e6 --- /dev/null +++ b/resource/opentelemetry-resource-detector-azure/src/opentelemetry/resource/detector/azure/version.py @@ -0,0 +1,15 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +__version__ = "0.1.0" diff --git a/resource/opentelemetry-resource-detector-azure/src/opentelemetry/resource/detector/azure/vm.py b/resource/opentelemetry-resource-detector-azure/src/opentelemetry/resource/detector/azure/vm.py new file mode 100644 index 0000000000..02f8ea537f --- /dev/null +++ b/resource/opentelemetry-resource-detector-azure/src/opentelemetry/resource/detector/azure/vm.py @@ -0,0 +1,97 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from json import loads +from logging import getLogger +from os import environ +from urllib.request import Request, urlopen +from urllib.error import URLError + +from opentelemetry.sdk.resources import ResourceDetector, Resource +from opentelemetry.semconv.resource import ResourceAttributes, CloudPlatformValues, CloudProviderValues + + +# TODO: Remove when cloud resource id is no longer missing in Resource Attributes +_CLOUD_RESOURCE_ID_RESOURCE_ATTRIBUTE = "cloud.resource_id" +_AZURE_VM_METADATA_ENDPOINT = "http://169.254.169.254/metadata/instance/compute?api-version=2021-12-13&format=json" +_AZURE_VM_SCALE_SET_NAME_ATTRIBUTE = "azure.vm.scaleset.name" +_AZURE_VM_SKU_ATTRIBUTE = "azure.vm.sku" +_logger = getLogger(__name__) + +EXPECTED_AZURE_AMS_ATTRIBUTES = [ + _AZURE_VM_SCALE_SET_NAME_ATTRIBUTE, + _AZURE_VM_SKU_ATTRIBUTE, + ResourceAttributes.CLOUD_PLATFORM, + ResourceAttributes.CLOUD_PROVIDER, + ResourceAttributes.CLOUD_REGION, + _CLOUD_RESOURCE_ID_RESOURCE_ATTRIBUTE, + ResourceAttributes.HOST_ID, + ResourceAttributes.HOST_NAME, + ResourceAttributes.HOST_TYPE, + ResourceAttributes.OS_TYPE, + ResourceAttributes.OS_VERSION, + ResourceAttributes.SERVICE_INSTANCE_ID, +] + +class AzureVMResourceDetector(ResourceDetector): + # pylint: disable=no-self-use + def detect(self) -> "Resource": + attributes = {} + metadata_json = _AzureVMMetadataServiceRequestor().get_azure_vm_metadata() + if not metadata_json: + return Resource(attributes) + for attribute_key in EXPECTED_AZURE_AMS_ATTRIBUTES: + attributes[attribute_key] = _AzureVMMetadataServiceRequestor().get_attribute_from_metadata(metadata_json, attribute_key) + return Resource(attributes) + +class _AzureVMMetadataServiceRequestor: + def get_azure_vm_metadata(self): + request = Request(_AZURE_VM_METADATA_ENDPOINT) + request.add_header("Metadata", "True") + try: + response = urlopen(request).read() + return loads(response) + except URLError: + # Not on Azure VM + return None + except Exception as e: + _logger.exception("Failed to receive Azure VM metadata: %s", e) + return None + + def get_attribute_from_metadata(self, metadata_json, attribute_key): + ams_value = "" + if attribute_key == _AZURE_VM_SCALE_SET_NAME_ATTRIBUTE: + ams_value = metadata_json["vmScaleSetName"] + elif attribute_key == _AZURE_VM_SKU_ATTRIBUTE: + ams_value = metadata_json["sku"] + elif attribute_key == ResourceAttributes.CLOUD_PLATFORM: + ams_value = CloudPlatformValues.AZURE_VM.value + elif attribute_key == ResourceAttributes.CLOUD_PROVIDER: + ams_value = CloudProviderValues.AZURE.value + elif attribute_key == ResourceAttributes.CLOUD_REGION: + ams_value = metadata_json["location"] + elif attribute_key == _CLOUD_RESOURCE_ID_RESOURCE_ATTRIBUTE: + ams_value = metadata_json["resourceId"] + elif attribute_key == ResourceAttributes.HOST_ID or \ + attribute_key == ResourceAttributes.SERVICE_INSTANCE_ID: + ams_value = metadata_json["vmId"] + elif attribute_key == ResourceAttributes.HOST_NAME: + ams_value = metadata_json["name"] + elif attribute_key == ResourceAttributes.HOST_TYPE: + ams_value = metadata_json["vmSize"] + elif attribute_key == ResourceAttributes.OS_TYPE: + ams_value = metadata_json["osType"] + elif attribute_key == ResourceAttributes.OS_VERSION: + ams_value = metadata_json["version"] + return ams_value diff --git a/resource/opentelemetry-resource-detector-azure/tests/__init__.py b/resource/opentelemetry-resource-detector-azure/tests/__init__.py new file mode 100644 index 0000000000..b0a6f42841 --- /dev/null +++ b/resource/opentelemetry-resource-detector-azure/tests/__init__.py @@ -0,0 +1,13 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. diff --git a/resource/opentelemetry-resource-detector-azure/tests/test_app_service.py b/resource/opentelemetry-resource-detector-azure/tests/test_app_service.py new file mode 100644 index 0000000000..f5d6a0dd3d --- /dev/null +++ b/resource/opentelemetry-resource-detector-azure/tests/test_app_service.py @@ -0,0 +1,116 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +import unittest +from unittest.mock import patch + +from opentelemetry.resource.detector.azure.app_service import ( + AzureAppServiceResourceDetector, +) + +TEST_WEBSITE_SITE_NAME = "TEST_WEBSITE_SITE_NAME" +TEST_REGION_NAME = "TEST_REGION_NAME" +TEST_WEBSITE_SLOT_NAME = "TEST_WEBSITE_SLOT_NAME" +TEST_WEBSITE_HOSTNAME = "TEST_WEBSITE_HOSTNAME" +TEST_WEBSITE_INSTANCE_ID = "TEST_WEBSITE_INSTANCE_ID" +TEST_WEBSITE_HOME_STAMPNAME = "TEST_WEBSITE_HOME_STAMPNAME" + +TEST_WEBSITE_RESOURCE_GROUP = "TEST_WEBSITE_RESOURCE_GROUP" +TEST_WEBSITE_OWNER_NAME = "TEST_WEBSITE_OWNER_NAME" + +class TestAzureAppServiceResourceDetector(unittest.TestCase): + @patch.dict("os.environ", { + "WEBSITE_SITE_NAME": TEST_WEBSITE_SITE_NAME, + "REGION_NAME": TEST_REGION_NAME, + "WEBSITE_SLOT_NAME": TEST_WEBSITE_SLOT_NAME, + "WEBSITE_HOSTNAME": TEST_WEBSITE_HOSTNAME, + "WEBSITE_INSTANCE_ID": TEST_WEBSITE_INSTANCE_ID, + "WEBSITE_HOME_STAMPNAME": TEST_WEBSITE_HOME_STAMPNAME, + "WEBSITE_RESOURCE_GROUP": TEST_WEBSITE_RESOURCE_GROUP, + "WEBSITE_OWNER_NAME": TEST_WEBSITE_OWNER_NAME, + }, clear=True) + def test_on_app_service(self): + resource = AzureAppServiceResourceDetector().detect() + attributes = resource.attributes + self.assertEqual(attributes["service.name"], TEST_WEBSITE_SITE_NAME) + self.assertEqual(attributes["cloud.provider"], "azure") + self.assertEqual(attributes["cloud.platform"], "azure_app_service") + + self.assertEqual(attributes["cloud.resource_id"], \ + f"/subscriptions/{TEST_WEBSITE_OWNER_NAME}/resourceGroups/{TEST_WEBSITE_RESOURCE_GROUP}/providers/Microsoft.Web/sites/{TEST_WEBSITE_SITE_NAME}") + + self.assertEqual(attributes["cloud.region"], TEST_REGION_NAME) + self.assertEqual(attributes["deployment.environment"], TEST_WEBSITE_SLOT_NAME) + self.assertEqual(attributes["host.id"], TEST_WEBSITE_HOSTNAME) + self.assertEqual(attributes["service.instance.id"], TEST_WEBSITE_INSTANCE_ID) + self.assertEqual(attributes["azure.app.service.stamp"], TEST_WEBSITE_HOME_STAMPNAME) + + @patch.dict("os.environ", { + "WEBSITE_SITE_NAME": TEST_WEBSITE_SITE_NAME, + "REGION_NAME": TEST_REGION_NAME, + "WEBSITE_SLOT_NAME": TEST_WEBSITE_SLOT_NAME, + "WEBSITE_HOSTNAME": TEST_WEBSITE_HOSTNAME, + "WEBSITE_INSTANCE_ID": TEST_WEBSITE_INSTANCE_ID, + "WEBSITE_HOME_STAMPNAME": TEST_WEBSITE_HOME_STAMPNAME, + "WEBSITE_OWNER_NAME": TEST_WEBSITE_OWNER_NAME, + }, clear=True) + def test_on_app_service_no_resource_group(self): + resource = AzureAppServiceResourceDetector().detect() + attributes = resource.attributes + self.assertEqual(attributes["service.name"], TEST_WEBSITE_SITE_NAME) + self.assertEqual(attributes["cloud.provider"], "azure") + self.assertEqual(attributes["cloud.platform"], "azure_app_service") + + self.assertTrue("cloud.resource_id" not in attributes) + + self.assertEqual(attributes["cloud.region"], TEST_REGION_NAME) + self.assertEqual(attributes["deployment.environment"], TEST_WEBSITE_SLOT_NAME) + self.assertEqual(attributes["host.id"], TEST_WEBSITE_HOSTNAME) + self.assertEqual(attributes["service.instance.id"], TEST_WEBSITE_INSTANCE_ID) + self.assertEqual(attributes["azure.app.service.stamp"], TEST_WEBSITE_HOME_STAMPNAME) + + @patch.dict("os.environ", { + "WEBSITE_SITE_NAME": TEST_WEBSITE_SITE_NAME, + "REGION_NAME": TEST_REGION_NAME, + "WEBSITE_SLOT_NAME": TEST_WEBSITE_SLOT_NAME, + "WEBSITE_HOSTNAME": TEST_WEBSITE_HOSTNAME, + "WEBSITE_INSTANCE_ID": TEST_WEBSITE_INSTANCE_ID, + "WEBSITE_HOME_STAMPNAME": TEST_WEBSITE_HOME_STAMPNAME, + "WEBSITE_OWNER_NAME": TEST_WEBSITE_OWNER_NAME, + }, clear=True) + def test_on_app_service_no_owner(self): + resource = AzureAppServiceResourceDetector().detect() + attributes = resource.attributes + self.assertEqual(attributes["service.name"], TEST_WEBSITE_SITE_NAME) + self.assertEqual(attributes["cloud.provider"], "azure") + self.assertEqual(attributes["cloud.platform"], "azure_app_service") + + self.assertTrue("cloud.resource_id" not in attributes) + + self.assertEqual(attributes["cloud.region"], TEST_REGION_NAME) + self.assertEqual(attributes["deployment.environment"], TEST_WEBSITE_SLOT_NAME) + self.assertEqual(attributes["host.id"], TEST_WEBSITE_HOSTNAME) + self.assertEqual(attributes["service.instance.id"], TEST_WEBSITE_INSTANCE_ID) + self.assertEqual(attributes["azure.app.service.stamp"], TEST_WEBSITE_HOME_STAMPNAME) + + @patch.dict("os.environ", { + "REGION_NAME": TEST_REGION_NAME, + "WEBSITE_SLOT_NAME": TEST_WEBSITE_SLOT_NAME, + "WEBSITE_HOSTNAME": TEST_WEBSITE_HOSTNAME, + "WEBSITE_INSTANCE_ID": TEST_WEBSITE_INSTANCE_ID, + "WEBSITE_HOME_STAMPNAME": TEST_WEBSITE_HOME_STAMPNAME, + "WEBSITE_OWNER_NAME": TEST_WEBSITE_OWNER_NAME, + }, clear=True) + def test_off_app_service(self): + resource = AzureAppServiceResourceDetector().detect() + self.assertEqual(resource.attributes, {}) diff --git a/resource/opentelemetry-resource-detector-azure/tests/test_vm.py b/resource/opentelemetry-resource-detector-azure/tests/test_vm.py new file mode 100644 index 0000000000..0531fa02b1 --- /dev/null +++ b/resource/opentelemetry-resource-detector-azure/tests/test_vm.py @@ -0,0 +1,382 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +import unittest +from unittest.mock import patch, Mock + +from opentelemetry.semconv.resource import ResourceAttributes +from opentelemetry.resource.detector.azure.vm import ( + AzureVMResourceDetector, +) + +LINUX_JSON = """ +{ + "additionalCapabilities": { + "hibernationEnabled": "false" + }, + "azEnvironment": "AzurePublicCloud", + "customData": "", + "evictionPolicy": "", + "extendedLocation": { + "name": "", + "type": "" + }, + "host": { + "id": "" + }, + "hostGroup": { + "id": "" + }, + "isHostCompatibilityLayerVm": "true", + "licenseType": "", + "location": "westus", + "name": "examplevmname", + "offer": "0001-com-ubuntu-server-focal", + "osProfile": { + "adminUsername": "azureuser", + "computerName": "examplevmname", + "disablePasswordAuthentication": "true" + }, + "osType": "Linux", + "placementGroupId": "", + "plan": { + "name": "", + "product": "", + "publisher": "" + }, + "platformFaultDomain": "0", + "platformSubFaultDomain": "", + "platformUpdateDomain": "0", + "priority": "", + "provider": "Microsoft.Compute", + "publicKeys": [ + { + "keyData": "ssh-rsa 0", + "path": "/home/user/.ssh/authorized_keys0" + }, + { + "keyData": "ssh-rsa 1", + "path": "/home/user/.ssh/authorized_keys1" + } + ], + "publisher": "canonical", + "resourceGroupName": "macikgo-test-may-23", + "resourceId": "/subscriptions/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/macikgo-test-may-23/providers/Microsoft.Compute/virtualMachines/examplevmname", + "securityProfile": { + "encryptionAtHost": "false", + "secureBootEnabled": "true", + "securityType": "TrustedLaunch", + "virtualTpmEnabled": "true" + }, + "sku": "20_04-lts-gen2", + "storageProfile": { + "dataDisks": [ + { + "bytesPerSecondThrottle": "979202048", + "caching": "None", + "createOption": "Empty", + "diskCapacityBytes": "274877906944", + "diskSizeGB": "1024", + "image": { + "uri": "" + }, + "isSharedDisk": "false", + "isUltraDisk": "true", + "lun": "0", + "managedDisk": { + "id": "/subscriptions/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/macikgo-test-may-23/providers/Microsoft.Compute/disks/exampledatadiskname", + "storageAccountType": "StandardSSD_LRS" + }, + "name": "exampledatadiskname", + "opsPerSecondThrottle": "65280", + "vhd": { + "uri": "" + }, + "writeAcceleratorEnabled": "false" + } + ], + "imageReference": { + "id": "", + "offer": "0001-com-ubuntu-server-focal", + "publisher": "canonical", + "sku": "20_04-lts-gen2", + "version": "latest" + }, + "osDisk": { + "caching": "ReadWrite", + "createOption": "FromImage", + "diffDiskSettings": { + "option": "" + }, + "diskSizeGB": "30", + "encryptionSettings": { + "enabled": "false", + "diskEncryptionKey": { + "sourceVault": { + "id": "/subscriptions/test-source-guid/resourceGroups/testrg/providers/Microsoft.KeyVault/vaults/test-kv" + }, + "secretUrl": "https://test-disk.vault.azure.net/secrets/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx" + }, + "keyEncryptionKey": { + "sourceVault": { + "id": "/subscriptions/test-key-guid/resourceGroups/testrg/providers/Microsoft.KeyVault/vaults/test-kv" + }, + "keyUrl": "https://test-key.vault.azure.net/secrets/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx" + } + }, + "image": { + "uri": "" + }, + "managedDisk": { + "id": "/subscriptions/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/macikgo-test-may-23/providers/Microsoft.Compute/disks/exampleosdiskname", + "storageAccountType": "StandardSSD_LRS" + }, + "name": "exampledatadiskname", + "osType": "Linux", + "vhd": { + "uri": "" + }, + "writeAcceleratorEnabled": "false" + }, + "resourceDisk": { + "size": "16384" + } + }, + "subscriptionId": "xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx", + "tags": "azsecpack:nonprod;platformsettings.host_environment.service.platform_optedin_for_rootcerts:true", + "tagsList": [ + { + "name": "azsecpack", + "value": "nonprod" + }, + { + "name": "platformsettings.host_environment.service.platform_optedin_for_rootcerts", + "value": "true" + } + ], + "userData": "", + "version": "20.04.202307240", + "virtualMachineScaleSet": { + "id": "/subscriptions/xxxxxxxx-xxxxx-xxx-xxx-xxxx/resourceGroups/resource-group-name/providers/Microsoft.Compute/virtualMachineScaleSets/virtual-machine-scale-set-name" + }, + "vmId": "02aab8a4-74ef-476e-8182-f6d2ba4166a6", + "vmScaleSetName": "crpteste9vflji9", + "vmSize": "Standard_A3", + "zone": "1" +} +""" +WINDOWS_JSON =""" +{ + "additionalCapabilities": { + "hibernationEnabled": "false" + }, + "azEnvironment": "AzurePublicCloud", + "customData": "", + "evictionPolicy": "", + "extendedLocation": { + "name": "", + "type": "" + }, + "host": { + "id": "" + }, + "hostGroup": { + "id": "" + }, + "isHostCompatibilityLayerVm": "true", + "licenseType": "Windows_Client", + "location": "westus", + "name": "examplevmname", + "offer": "WindowsServer", + "osProfile": { + "adminUsername": "azureuser", + "computerName": "examplevmname", + "disablePasswordAuthentication": "true" + }, + "osType": "Windows", + "placementGroupId": "", + "plan": { + "name": "", + "product": "", + "publisher": "" + }, + "platformFaultDomain": "0", + "platformSubFaultDomain": "", + "platformUpdateDomain": "0", + "priority": "", + "provider": "Microsoft.Compute", + "publicKeys": [ + { + "keyData": "ssh-rsa 0", + "path": "/home/user/.ssh/authorized_keys0" + }, + { + "keyData": "ssh-rsa 1", + "path": "/home/user/.ssh/authorized_keys1" + } + ], + "publisher": "RDFE-Test-Microsoft-Windows-Server-Group", + "resourceGroupName": "macikgo-test-may-23", + "resourceId": "/subscriptions/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/macikgo-test-may-23/providers/Microsoft.Compute/virtualMachines/examplevmname", + "securityProfile": { + "encryptionAtHost": "false", + "secureBootEnabled": "true", + "securityType": "TrustedLaunch", + "virtualTpmEnabled": "true" + }, + "sku": "2019-Datacenter", + "storageProfile": { + "dataDisks": [ + { + "bytesPerSecondThrottle": "979202048", + "caching": "None", + "createOption": "Empty", + "diskCapacityBytes": "274877906944", + "diskSizeGB": "1024", + "image": { + "uri": "" + }, + "isSharedDisk": "false", + "isUltraDisk": "true", + "lun": "0", + "managedDisk": { + "id": "/subscriptions/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/macikgo-test-may-23/providers/Microsoft.Compute/disks/exampledatadiskname", + "storageAccountType": "StandardSSD_LRS" + }, + "name": "exampledatadiskname", + "opsPerSecondThrottle": "65280", + "vhd": { + "uri": "" + }, + "writeAcceleratorEnabled": "false" + } + ], + "imageReference": { + "id": "", + "offer": "WindowsServer", + "publisher": "MicrosoftWindowsServer", + "sku": "2019-Datacenter", + "version": "latest" + }, + "osDisk": { + "caching": "ReadWrite", + "createOption": "FromImage", + "diffDiskSettings": { + "option": "" + }, + "diskSizeGB": "30", + "encryptionSettings": { + "enabled": "false", + "diskEncryptionKey": { + "sourceVault": { + "id": "/subscriptions/test-source-guid/resourceGroups/testrg/providers/Microsoft.KeyVault/vaults/test-kv" + }, + "secretUrl": "https://test-disk.vault.azure.net/secrets/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx" + }, + "keyEncryptionKey": { + "sourceVault": { + "id": "/subscriptions/test-key-guid/resourceGroups/testrg/providers/Microsoft.KeyVault/vaults/test-kv" + }, + "keyUrl": "https://test-key.vault.azure.net/secrets/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx" + } + }, + "image": { + "uri": "" + }, + "managedDisk": { + "id": "/subscriptions/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/macikgo-test-may-23/providers/Microsoft.Compute/disks/exampleosdiskname", + "storageAccountType": "StandardSSD_LRS" + }, + "name": "exampledatadiskname", + "osType": "Windows", + "vhd": { + "uri": "" + }, + "writeAcceleratorEnabled": "false" + }, + "resourceDisk": { + "size": "16384" + } + }, + "subscriptionId": "xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx", + "tags": "azsecpack:nonprod;platformsettings.host_environment.service.platform_optedin_for_rootcerts:true", + "userData": "Zm9vYmFy", + "tagsList": [ + { + "name": "azsecpack", + "value": "nonprod" + }, + { + "name": "platformsettings.host_environment.service.platform_optedin_for_rootcerts", + "value": "true" + } + ], + "userData": "", + "version": "20.04.202307240", + "virtualMachineScaleSet": { + "id": "/subscriptions/xxxxxxxx-xxxxx-xxx-xxx-xxxx/resourceGroups/resource-group-name/providers/Microsoft.Compute/virtualMachineScaleSets/virtual-machine-scale-set-name" + }, + "vmId": "02aab8a4-74ef-476e-8182-f6d2ba4166a6", + "vmScaleSetName": "crpteste9vflji9", + "vmSize": "Standard_A3", + "zone": "1" +} +""" +LINUX_ATTRIBUTES = { + "azure.vm.scaleset.name": "crpteste9vflji9", + "azure.vm.sku": "20_04-lts-gen2", + "cloud.platform": "azure_vm", + "cloud.provider": "azure", + "cloud.region": "westus", + "cloud.resource_id": "/subscriptions/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/macikgo-test-may-23/providers/Microsoft.Compute/virtualMachines/examplevmname", + "host.id": "02aab8a4-74ef-476e-8182-f6d2ba4166a6", + "host.name": "examplevmname", + "host.type": "Standard_A3", + "os.type": "Linux", + "os.version": "20.04.202307240", + "service.instance.id": "02aab8a4-74ef-476e-8182-f6d2ba4166a6", +} +WINDOWS_ATTRIBUTES = { + "azure.vm.scaleset.name": "crpteste9vflji9", + "azure.vm.sku": "2019-Datacenter", + "cloud.platform": "azure_vm", + "cloud.provider": "azure", + "cloud.region": "westus", + "cloud.resource_id": "/subscriptions/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/macikgo-test-may-23/providers/Microsoft.Compute/virtualMachines/examplevmname", + "host.id": "02aab8a4-74ef-476e-8182-f6d2ba4166a6", + "host.name": "examplevmname", + "host.type": "Standard_A3", + "os.type": "Windows", + "os.version": "20.04.202307240", + "service.instance.id": "02aab8a4-74ef-476e-8182-f6d2ba4166a6", +} + + +class TestAzureVMResourceDetector(unittest.TestCase): + @patch("opentelemetry.resource.detector.azure.vm.urlopen") + def test_linux(self, mock_urlopen): + mock_open = Mock() + mock_urlopen.return_value = mock_open + mock_open.read.return_value = LINUX_JSON + attributes = AzureVMResourceDetector().detect().attributes + for attribute_key in LINUX_ATTRIBUTES: + self.assertEqual(attributes[attribute_key], LINUX_ATTRIBUTES[attribute_key]) + + @patch("opentelemetry.resource.detector.azure.vm.urlopen") + def test_windows(self, mock_urlopen): + mock_open = Mock() + mock_urlopen.return_value = mock_open + mock_open.read.return_value = WINDOWS_JSON + attributes = AzureVMResourceDetector().detect().attributes + for attribute_key in WINDOWS_ATTRIBUTES: + self.assertEqual(attributes[attribute_key], WINDOWS_ATTRIBUTES[attribute_key]) diff --git a/resource/opentelemetry-resource-detector-container/LICENSE b/resource/opentelemetry-resource-detector-container/LICENSE new file mode 100644 index 0000000000..1ef7dad2c5 --- /dev/null +++ b/resource/opentelemetry-resource-detector-container/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright The OpenTelemetry Authors + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/resource/opentelemetry-resource-detector-container/MANIFEST.rst b/resource/opentelemetry-resource-detector-container/MANIFEST.rst new file mode 100644 index 0000000000..2906eeef0f --- /dev/null +++ b/resource/opentelemetry-resource-detector-container/MANIFEST.rst @@ -0,0 +1,9 @@ +graft src +graft tests +global-exclude *.pyc +global-exclude *.pyo +global-exclude __pycache__/* +include CHANGELOG.md +include MANIFEST.in +include README.rst +include LICENSE \ No newline at end of file diff --git a/resource/opentelemetry-resource-detector-container/README.rst b/resource/opentelemetry-resource-detector-container/README.rst new file mode 100644 index 0000000000..8fadd67951 --- /dev/null +++ b/resource/opentelemetry-resource-detector-container/README.rst @@ -0,0 +1,46 @@ +OpenTelemetry Resource detectors for containers +========================================================== + +|pypi| + +.. |pypi| image:: TODO + :target: TODO + + +This library provides custom resource detector for container platforms + +Installation +------------ + +:: + + pip install opentelemetry-resource-detector-container + +--------------------------- + +Usage example for `opentelemetry-resource-detector-container` + +.. code-block:: python + + from opentelemetry import trace + from opentelemetry.sdk.trace import TracerProvider + from opentelemetry.resource.detector.container import ( + ContainerResourceDetector, + ) + from opentelemetry.sdk.resources import get_aggregated_resources + + + trace.set_tracer_provider( + TracerProvider( + resource=get_aggregated_resources( + [ + ContainerResourceDetector(), + ] + ), + ) + ) + +References +---------- + +* `OpenTelemetry Project `_ diff --git a/resource/opentelemetry-resource-detector-container/pyproject.toml b/resource/opentelemetry-resource-detector-container/pyproject.toml new file mode 100644 index 0000000000..87fa4c83a8 --- /dev/null +++ b/resource/opentelemetry-resource-detector-container/pyproject.toml @@ -0,0 +1,50 @@ +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[project] +name = "opentelemetry-resource-detector-container" +dynamic = ["version"] +description = "Container Resource Detector for OpenTelemetry" +readme = "README.rst" +license = "Apache-2.0" +requires-python = ">=3.7" +authors = [ + { name = "OpenTelemetry Authors", email = "cncf-opentelemetry-contributors@lists.cncf.io" }, +] +classifiers = [ + "Development Status :: 5 - Production/Stable", + "Intended Audience :: Developers", + "License :: OSI Approved :: Apache Software License", + "Programming Language :: Python", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.7", + "Programming Language :: Python :: 3.8", + "Programming Language :: Python :: 3.9", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", +] +dependencies = [ + "opentelemetry-sdk ~= 1.12", +] + +[project.optional-dependencies] +test = [] + +[project.entry-points.opentelemetry_resource_detector] +container = "opentelemetry.resource.detector.container:ContainerResourceDetector" + +[project.urls] +Homepage = "https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/resource/opentelemetry-resource-detector-container" + +[tool.hatch.version] +path = "src/opentelemetry/resource/detector/container/version.py" + +[tool.hatch.build.targets.sdist] +include = [ + "/src", + "/tests", +] + +[tool.hatch.build.targets.wheel] +packages = ["src/opentelemetry"] diff --git a/resource/opentelemetry-resource-detector-container/src/opentelemetry/resource/detector/container/__init__.py b/resource/opentelemetry-resource-detector-container/src/opentelemetry/resource/detector/container/__init__.py new file mode 100644 index 0000000000..8e7db6a7a8 --- /dev/null +++ b/resource/opentelemetry-resource-detector-container/src/opentelemetry/resource/detector/container/__init__.py @@ -0,0 +1,95 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from logging import getLogger + +from opentelemetry.sdk.resources import Resource, ResourceDetector +from opentelemetry.semconv.resource import ResourceAttributes + +logger = getLogger(__name__) +_DEFAULT_CGROUP_V1_PATH = "/proc/self/cgroup" +_DEFAULT_CGROUP_V2_PATH = "/proc/self/mountinfo" +_CONTAINER_ID_LENGTH = 64 + + +def _get_container_id_v1(): + container_id = None + try: + with open( + _DEFAULT_CGROUP_V1_PATH, encoding="utf8" + ) as container_info_file: + for raw_line in container_info_file.readlines(): + line = raw_line.strip() + if len(line) > _CONTAINER_ID_LENGTH: + container_id = line[-_CONTAINER_ID_LENGTH:] + break + except FileNotFoundError as exception: + logger.warning("Failed to get container id. Exception: %s", exception) + return container_id + + +def _get_container_id_v2(): + container_id = None + try: + with open( + _DEFAULT_CGROUP_V2_PATH, encoding="utf8" + ) as container_info_file: + for raw_line in container_info_file.readlines(): + line = raw_line.strip() + if any( + key_word in line for key_word in ["containers", "hostname"] + ): + container_id_list = [ + id_ + for id_ in line.split("/") + if len(id_) == _CONTAINER_ID_LENGTH + ] + if len(container_id_list) > 0: + container_id = container_id_list[0] + break + + except FileNotFoundError as exception: + logger.warning("Failed to get container id. Exception: %s", exception) + return container_id + + +def _get_container_id(): + return _get_container_id_v1() or _get_container_id_v2() + + +class ContainerResourceDetector(ResourceDetector): + """Detects container.id only available when app is running inside the + docker container and return it in a Resource + """ + + def detect(self) -> "Resource": + try: + container_id = _get_container_id() + resource = Resource.get_empty() + if container_id: + resource = resource.merge( + Resource({ResourceAttributes.CONTAINER_ID: container_id}) + ) + return resource + + # pylint: disable=broad-except + except Exception as exception: + logger.warning( + "%s Resource Detection failed silently: %s", + self.__class__.__name__, + exception, + ) + if self.raise_on_error: + raise exception + return Resource.get_empty() diff --git a/resource/opentelemetry-resource-detector-container/src/opentelemetry/resource/detector/container/version.py b/resource/opentelemetry-resource-detector-container/src/opentelemetry/resource/detector/container/version.py new file mode 100644 index 0000000000..c2996671d6 --- /dev/null +++ b/resource/opentelemetry-resource-detector-container/src/opentelemetry/resource/detector/container/version.py @@ -0,0 +1,15 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +__version__ = "0.42b0.dev" diff --git a/resource/opentelemetry-resource-detector-container/tests/__init__.py b/resource/opentelemetry-resource-detector-container/tests/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/resource/opentelemetry-resource-detector-container/tests/test_container.py b/resource/opentelemetry-resource-detector-container/tests/test_container.py new file mode 100644 index 0000000000..ac55afa291 --- /dev/null +++ b/resource/opentelemetry-resource-detector-container/tests/test_container.py @@ -0,0 +1,146 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from unittest.mock import mock_open, patch + +from opentelemetry import trace as trace_api +from opentelemetry.resource.detector.container import ContainerResourceDetector +from opentelemetry.sdk.resources import get_aggregated_resources +from opentelemetry.semconv.resource import ResourceAttributes +from opentelemetry.test.test_base import TestBase + +MockContainerResourceAttributes = { + ResourceAttributes.CONTAINER_ID: "7be92808767a667f35c8505cbf40d14e931ef6db5b0210329cf193b15ba9d605", +} + + +class ContainerResourceDetectorTest(TestBase): + @patch( + "builtins.open", + new_callable=mock_open, + read_data=f"""14:name=systemd:/docker/{MockContainerResourceAttributes[ResourceAttributes.CONTAINER_ID]} + 13:rdma:/ + 12:pids:/docker/bogusContainerIdThatShouldNotBeOneSetBecauseTheFirstOneWasPicked + 11:hugetlb:/docker/bogusContainerIdThatShouldNotBeOneSetBecauseTheFirstOneWasPicked + 10:net_prio:/docker/bogusContainerIdThatShouldNotBeOneSetBecauseTheFirstOneWasPicked + 9:perf_event:/docker/bogusContainerIdThatShouldNotBeOneSetBecauseTheFirstOneWasPicked + 8:net_cls:/docker/bogusContainerIdThatShouldNotBeOneSetBecauseTheFirstOneWasPicked + 7:freezer:/docker/ + 6:devices:/docker/bogusContainerIdThatShouldNotBeOneSetBecauseTheFirstOneWasPicked + 5:memory:/docker/bogusContainerIdThatShouldNotBeOneSetBecauseTheFirstOneWasPicked + 4:blkio:/docker/bogusContainerIdThatShouldNotBeOneSetBecauseTheFirstOneWasPicked + 3:cpuacct:/docker/bogusContainerIdThatShouldNotBeOneSetBecauseTheFirstOneWasPicked + 2:cpu:/docker/bogusContainerIdThatShouldNotBeOneSetBecauseTheFirstOneWasPicked + 1:cpuset:/docker/bogusContainerIdThatShouldNotBeOneSetBecauseTheFirstOneWasPicked + """, + ) + def test_container_id_detect_from_cgroup_file(self, mock_cgroup_file): + actual = ContainerResourceDetector().detect() + self.assertDictEqual( + actual.attributes.copy(), MockContainerResourceAttributes + ) + + @patch( + "opentelemetry.resource.detector.container._get_container_id_v1", + return_value=None, + ) + @patch( + "builtins.open", + new_callable=mock_open, + read_data=f""" + 608 607 0:183 / /proc rw,nosuid,nodev,noexec,relatime - proc proc rw + 609 607 0:184 / /dev rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755 + 610 609 0:185 / /dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666 + 611 607 0:186 / /sys ro,nosuid,nodev,noexec,relatime - sysfs sysfs ro + 612 611 0:29 / /sys/fs/cgroup ro,nosuid,nodev,noexec,relatime - cgroup2 cgroup rw + 613 609 0:182 / /dev/mqueue rw,nosuid,nodev,noexec,relatime - mqueue mqueue rw + 614 609 0:187 / /dev/shm rw,nosuid,nodev,noexec,relatime - tmpfs shm rw,size=65536k + 615 607 254:1 /docker/containers/{MockContainerResourceAttributes[ResourceAttributes.CONTAINER_ID]}/resolv.conf /etc/resolv.conf rw,relatime - ext4 /dev/vda1 rw + 616 607 254:1 /docker/containers/{MockContainerResourceAttributes[ResourceAttributes.CONTAINER_ID]}/hostname /etc/hostname rw,relatime - ext4 /dev/vda1 rw + 617 607 254:1 /docker/containers/bogusContainerIdThatShouldNotBeOneSetBecauseTheFirstOneWasPicked/hosts /etc/hosts rw,relatime - ext4 /dev/vda1 rw + 618 607 0:131 /Users/sankmeht/development/otel/opentelemetry-python /development/otel/opentelemetry-python rw,nosuid,nodev,relatime - fuse.grpcfuse grpcfuse rw,user_id=0,group_id=0,allow_other,max_read=1048576 + 619 607 0:131 /Users/sankmeht/development/otel/opentelemetry-python-contrib /development/otel/opentelemetry-python-contrib rw,nosuid,nodev,relatime - fuse.grpcfuse grpcfuse rw,user_id=0,group_id=0,allow_other,max_read=1048576 + 519 609 0:185 /0 /dev/console rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666 + 520 608 0:183 /bus /proc/bus ro,nosuid,nodev,noexec,relatime - proc proc rw + 521 608 0:183 /fs /proc/fs ro,nosuid,nodev,noexec,relatime - proc proc rw + 522 608 0:183 /irq /proc/irq ro,nosuid,nodev,noexec,relatime - proc proc rw + 523 608 0:183 /sys /proc/sys ro,nosuid,nodev,noexec,relatime - proc proc rw + 524 608 0:183 /sysrq-trigger /proc/sysrq-trigger ro,nosuid,nodev,noexec,relatime - proc proc rw + 525 608 0:212 / /proc/acpi ro,relatime - tmpfs tmpfs ro + 526 608 0:184 /null /proc/kcore rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755 + 527 608 0:184 /null /proc/keys rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755 + 528 608 0:184 /null /proc/timer_list rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755 + 529 611 0:213 / /sys/firmware ro,relatime - tmpfs tmpfs ro + """, + ) + def test_container_id_detect_from_mountinfo_file( + self, mock_get_container_id_v1, mock_cgroup_file + ): + actual = ContainerResourceDetector().detect() + self.assertDictEqual( + actual.attributes.copy(), MockContainerResourceAttributes + ) + + @patch( + "opentelemetry.resource.detector.container._get_container_id", + return_value=MockContainerResourceAttributes[ + ResourceAttributes.CONTAINER_ID + ], + ) + def test_container_id_as_span_attribute(self, mock_cgroup_file): + tracer_provider, exporter = self.create_tracer_provider( + resource=get_aggregated_resources([ContainerResourceDetector()]) + ) + tracer = tracer_provider.get_tracer(__name__) + + with tracer.start_as_current_span( + "test", kind=trace_api.SpanKind.SERVER + ) as _: + pass + + span_list = exporter.get_finished_spans() + self.assertEqual( + span_list[0].resource.attributes["container.id"], + MockContainerResourceAttributes[ResourceAttributes.CONTAINER_ID], + ) + + @patch( + "opentelemetry.resource.detector.container._get_container_id", + return_value=MockContainerResourceAttributes[ + ResourceAttributes.CONTAINER_ID + ], + ) + def test_container_id_detect_from_cgroup(self, mock_get_container_id): + actual = ContainerResourceDetector().detect() + self.assertDictEqual( + actual.attributes.copy(), MockContainerResourceAttributes + ) + + @patch( + "opentelemetry.resource.detector.container._get_container_id_v1", + return_value=None, + ) + @patch( + "opentelemetry.resource.detector.container._get_container_id_v2", + return_value=MockContainerResourceAttributes[ + ResourceAttributes.CONTAINER_ID + ], + ) + def test_container_id_detect_from_mount_info( + self, mock_get_container_id_v1, mock_get_container_id_v2 + ): + actual = ContainerResourceDetector().detect() + self.assertDictEqual( + actual.attributes.copy(), MockContainerResourceAttributes + ) diff --git a/scripts/build.sh b/scripts/build.sh index 56b350257a..dc3e237946 100755 --- a/scripts/build.sh +++ b/scripts/build.sh @@ -16,7 +16,7 @@ DISTDIR=dist mkdir -p $DISTDIR rm -rf $DISTDIR/* - for d in exporter/*/ opentelemetry-instrumentation/ opentelemetry-contrib-instrumentations/ opentelemetry-distro/ instrumentation/*/ propagator/*/ sdk-extension/*/ util/*/ ; do + for d in exporter/*/ opentelemetry-instrumentation/ opentelemetry-contrib-instrumentations/ opentelemetry-distro/ instrumentation/*/ propagator/*/ resource/*/ sdk-extension/*/ util/*/ ; do ( echo "building $d" cd "$d" diff --git a/tests/opentelemetry-docker-tests/tests/celery/test_celery_functional.py b/tests/opentelemetry-docker-tests/tests/celery/test_celery_functional.py index 1a86154ffc..0b9cb3dac5 100644 --- a/tests/opentelemetry-docker-tests/tests/celery/test_celery_functional.py +++ b/tests/opentelemetry-docker-tests/tests/celery/test_celery_functional.py @@ -279,7 +279,7 @@ def fn_exception(): assert len(span.events) == 1 event = span.events[0] assert event.name == "exception" - assert event.attributes[SpanAttributes.EXCEPTION_TYPE] == "ExceptionInfo" + assert event.attributes[SpanAttributes.EXCEPTION_TYPE] == "Exception" assert SpanAttributes.EXCEPTION_MESSAGE in event.attributes assert ( span.attributes.get(SpanAttributes.MESSAGING_MESSAGE_ID) @@ -303,8 +303,8 @@ def fn_exception(): span = spans[0] - assert span.status.is_ok is True - assert span.status.status_code == StatusCode.UNSET + assert span.status.is_ok is False + assert span.status.status_code == StatusCode.ERROR assert span.name == "run/test_celery_functional.fn_exception" assert span.attributes.get("celery.action") == "run" assert span.attributes.get("celery.state") == "FAILURE" @@ -420,7 +420,7 @@ def run(self): assert "Task class is failing" in span.status.description -def test_class_task_exception_excepted(celery_app, memory_exporter): +def test_class_task_exception_expected(celery_app, memory_exporter): class BaseTask(celery_app.Task): throws = (MyException,) @@ -443,8 +443,8 @@ def run(self): span = spans[0] - assert span.status.is_ok is True - assert span.status.status_code == StatusCode.UNSET + assert span.status.is_ok is False + assert span.status.status_code == StatusCode.ERROR assert span.name == "run/test_celery_functional.BaseTask" assert span.attributes.get("celery.action") == "run" assert span.attributes.get("celery.state") == "FAILURE" diff --git a/tests/opentelemetry-docker-tests/tests/redis/test_redis_functional.py b/tests/opentelemetry-docker-tests/tests/redis/test_redis_functional.py index 675a37fa9f..481b8d21c8 100644 --- a/tests/opentelemetry-docker-tests/tests/redis/test_redis_functional.py +++ b/tests/opentelemetry-docker-tests/tests/redis/test_redis_functional.py @@ -13,6 +13,7 @@ # limitations under the License. import asyncio +from time import time_ns import redis import redis.asyncio @@ -47,9 +48,7 @@ def _check_span(self, span, name): def test_long_command_sanitized(self): RedisInstrumentor().uninstrument() - RedisInstrumentor().instrument( - tracer_provider=self.tracer_provider, sanitize_query=True - ) + RedisInstrumentor().instrument(tracer_provider=self.tracer_provider) self.redis_client.mget(*range(2000)) @@ -75,7 +74,7 @@ def test_long_command(self): self._check_span(span, "MGET") self.assertTrue( span.attributes.get(SpanAttributes.DB_STATEMENT).startswith( - "MGET 0 1 2 3" + "MGET ? ? ? ?" ) ) self.assertTrue( @@ -84,9 +83,7 @@ def test_long_command(self): def test_basics_sanitized(self): RedisInstrumentor().uninstrument() - RedisInstrumentor().instrument( - tracer_provider=self.tracer_provider, sanitize_query=True - ) + RedisInstrumentor().instrument(tracer_provider=self.tracer_provider) self.assertIsNone(self.redis_client.get("cheese")) spans = self.memory_exporter.get_finished_spans() @@ -105,15 +102,13 @@ def test_basics(self): span = spans[0] self._check_span(span, "GET") self.assertEqual( - span.attributes.get(SpanAttributes.DB_STATEMENT), "GET cheese" + span.attributes.get(SpanAttributes.DB_STATEMENT), "GET ?" ) self.assertEqual(span.attributes.get("db.redis.args_length"), 2) def test_pipeline_traced_sanitized(self): RedisInstrumentor().uninstrument() - RedisInstrumentor().instrument( - tracer_provider=self.tracer_provider, sanitize_query=True - ) + RedisInstrumentor().instrument(tracer_provider=self.tracer_provider) with self.redis_client.pipeline(transaction=False) as pipeline: pipeline.set("blah", 32) @@ -144,15 +139,13 @@ def test_pipeline_traced(self): self._check_span(span, "SET RPUSH HGETALL") self.assertEqual( span.attributes.get(SpanAttributes.DB_STATEMENT), - "SET blah 32\nRPUSH foo éé\nHGETALL xxx", + "SET ? ?\nRPUSH ? ?\nHGETALL ?", ) self.assertEqual(span.attributes.get("db.redis.pipeline_length"), 3) def test_pipeline_immediate_sanitized(self): RedisInstrumentor().uninstrument() - RedisInstrumentor().instrument( - tracer_provider=self.tracer_provider, sanitize_query=True - ) + RedisInstrumentor().instrument(tracer_provider=self.tracer_provider) with self.redis_client.pipeline() as pipeline: pipeline.set("a", 1) @@ -182,7 +175,7 @@ def test_pipeline_immediate(self): span = spans[0] self._check_span(span, "SET") self.assertEqual( - span.attributes.get(SpanAttributes.DB_STATEMENT), "SET b 2" + span.attributes.get(SpanAttributes.DB_STATEMENT), "SET ? ?" ) def test_parent(self): @@ -230,7 +223,7 @@ def test_basics(self): span = spans[0] self._check_span(span, "GET") self.assertEqual( - span.attributes.get(SpanAttributes.DB_STATEMENT), "GET cheese" + span.attributes.get(SpanAttributes.DB_STATEMENT), "GET ?" ) self.assertEqual(span.attributes.get("db.redis.args_length"), 2) @@ -247,7 +240,7 @@ def test_pipeline_traced(self): self._check_span(span, "SET RPUSH HGETALL") self.assertEqual( span.attributes.get(SpanAttributes.DB_STATEMENT), - "SET blah 32\nRPUSH foo éé\nHGETALL xxx", + "SET ? ?\nRPUSH ? ?\nHGETALL ?", ) self.assertEqual(span.attributes.get("db.redis.pipeline_length"), 3) @@ -308,7 +301,7 @@ def test_long_command(self): self._check_span(span, "MGET") self.assertTrue( span.attributes.get(SpanAttributes.DB_STATEMENT).startswith( - "MGET 0 1 2 3" + "MGET ? ? ? ?" ) ) self.assertTrue( @@ -322,10 +315,33 @@ def test_basics(self): span = spans[0] self._check_span(span, "GET") self.assertEqual( - span.attributes.get(SpanAttributes.DB_STATEMENT), "GET cheese" + span.attributes.get(SpanAttributes.DB_STATEMENT), "GET ?" ) self.assertEqual(span.attributes.get("db.redis.args_length"), 2) + def test_execute_command_traced_full_time(self): + """Command should be traced for coroutine execution time, not creation time.""" + coro_created_time = None + finish_time = None + + async def pipeline_simple(): + nonlocal coro_created_time + nonlocal finish_time + + # delay coroutine creation from coroutine execution + coro = self.redis_client.get("foo") + coro_created_time = time_ns() + await coro + finish_time = time_ns() + + async_call(pipeline_simple()) + + spans = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans), 1) + span = spans[0] + self.assertTrue(span.start_time > coro_created_time) + self.assertTrue(span.end_time < finish_time) + def test_pipeline_traced(self): async def pipeline_simple(): async with self.redis_client.pipeline( @@ -344,10 +360,39 @@ async def pipeline_simple(): self._check_span(span, "SET RPUSH HGETALL") self.assertEqual( span.attributes.get(SpanAttributes.DB_STATEMENT), - "SET blah 32\nRPUSH foo éé\nHGETALL xxx", + "SET ? ?\nRPUSH ? ?\nHGETALL ?", ) self.assertEqual(span.attributes.get("db.redis.pipeline_length"), 3) + def test_pipeline_traced_full_time(self): + """Command should be traced for coroutine execution time, not creation time.""" + coro_created_time = None + finish_time = None + + async def pipeline_simple(): + async with self.redis_client.pipeline( + transaction=False + ) as pipeline: + nonlocal coro_created_time + nonlocal finish_time + pipeline.set("blah", 32) + pipeline.rpush("foo", "éé") + pipeline.hgetall("xxx") + + # delay coroutine creation from coroutine execution + coro = pipeline.execute() + coro_created_time = time_ns() + await coro + finish_time = time_ns() + + async_call(pipeline_simple()) + + spans = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans), 1) + span = spans[0] + self.assertTrue(span.start_time > coro_created_time) + self.assertTrue(span.end_time < finish_time) + def test_pipeline_immediate(self): async def pipeline_immediate(): async with self.redis_client.pipeline() as pipeline: @@ -364,9 +409,36 @@ async def pipeline_immediate(): span = spans[0] self._check_span(span, "SET") self.assertEqual( - span.attributes.get(SpanAttributes.DB_STATEMENT), "SET b 2" + span.attributes.get(SpanAttributes.DB_STATEMENT), "SET ? ?" ) + def test_pipeline_immediate_traced_full_time(self): + """Command should be traced for coroutine execution time, not creation time.""" + coro_created_time = None + finish_time = None + + async def pipeline_simple(): + async with self.redis_client.pipeline( + transaction=False + ) as pipeline: + nonlocal coro_created_time + nonlocal finish_time + pipeline.set("a", 1) + + # delay coroutine creation from coroutine execution + coro = pipeline.immediate_execute_command("SET", "b", 2) + coro_created_time = time_ns() + await coro + finish_time = time_ns() + + async_call(pipeline_simple()) + + spans = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans), 1) + span = spans[0] + self.assertTrue(span.start_time > coro_created_time) + self.assertTrue(span.end_time < finish_time) + def test_parent(self): """Ensure OpenTelemetry works with redis.""" ot_tracer = trace.get_tracer("redis_svc") @@ -412,10 +484,33 @@ def test_basics(self): span = spans[0] self._check_span(span, "GET") self.assertEqual( - span.attributes.get(SpanAttributes.DB_STATEMENT), "GET cheese" + span.attributes.get(SpanAttributes.DB_STATEMENT), "GET ?" ) self.assertEqual(span.attributes.get("db.redis.args_length"), 2) + def test_execute_command_traced_full_time(self): + """Command should be traced for coroutine execution time, not creation time.""" + coro_created_time = None + finish_time = None + + async def pipeline_simple(): + nonlocal coro_created_time + nonlocal finish_time + + # delay coroutine creation from coroutine execution + coro = self.redis_client.get("foo") + coro_created_time = time_ns() + await coro + finish_time = time_ns() + + async_call(pipeline_simple()) + + spans = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans), 1) + span = spans[0] + self.assertTrue(span.start_time > coro_created_time) + self.assertTrue(span.end_time < finish_time) + def test_pipeline_traced(self): async def pipeline_simple(): async with self.redis_client.pipeline( @@ -434,10 +529,39 @@ async def pipeline_simple(): self._check_span(span, "SET RPUSH HGETALL") self.assertEqual( span.attributes.get(SpanAttributes.DB_STATEMENT), - "SET blah 32\nRPUSH foo éé\nHGETALL xxx", + "SET ? ?\nRPUSH ? ?\nHGETALL ?", ) self.assertEqual(span.attributes.get("db.redis.pipeline_length"), 3) + def test_pipeline_traced_full_time(self): + """Command should be traced for coroutine execution time, not creation time.""" + coro_created_time = None + finish_time = None + + async def pipeline_simple(): + async with self.redis_client.pipeline( + transaction=False + ) as pipeline: + nonlocal coro_created_time + nonlocal finish_time + pipeline.set("blah", 32) + pipeline.rpush("foo", "éé") + pipeline.hgetall("xxx") + + # delay coroutine creation from coroutine execution + coro = pipeline.execute() + coro_created_time = time_ns() + await coro + finish_time = time_ns() + + async_call(pipeline_simple()) + + spans = self.memory_exporter.get_finished_spans() + self.assertEqual(len(spans), 1) + span = spans[0] + self.assertTrue(span.start_time > coro_created_time) + self.assertTrue(span.end_time < finish_time) + def test_parent(self): """Ensure OpenTelemetry works with redis.""" ot_tracer = trace.get_tracer("redis_svc") @@ -488,5 +612,5 @@ def test_get(self): span = spans[0] self._check_span(span, "GET") self.assertEqual( - span.attributes.get(SpanAttributes.DB_STATEMENT), "GET foo" + span.attributes.get(SpanAttributes.DB_STATEMENT), "GET ?" ) diff --git a/tests/opentelemetry-docker-tests/tests/sqlalchemy_tests/test_postgres.py b/tests/opentelemetry-docker-tests/tests/sqlalchemy_tests/test_postgres.py index bbc62bfbbf..ff664091c8 100644 --- a/tests/opentelemetry-docker-tests/tests/sqlalchemy_tests/test_postgres.py +++ b/tests/opentelemetry-docker-tests/tests/sqlalchemy_tests/test_postgres.py @@ -95,3 +95,40 @@ class PostgresCreatorTestCase(PostgresTestCase): "url": "postgresql://", "creator": lambda: psycopg2.connect(**POSTGRES_CONFIG), } + + +class PostgresMetricsTestCase(PostgresTestCase): + __test__ = True + + VENDOR = "postgresql" + SQL_DB = "opentelemetry-tests" + ENGINE_ARGS = { + "url": "postgresql://%(user)s:%(password)s@%(host)s:%(port)s/%(dbname)s" + % POSTGRES_CONFIG + } + + def test_metrics_pool_name(self): + with self.connection() as conn: + conn.execute("SELECT 1 + 1").fetchall() + + pool_name = "{}://{}:{}/{}".format( + self.VENDOR, + POSTGRES_CONFIG["host"], + POSTGRES_CONFIG["port"], + self.SQL_DB, + ) + metrics = self.get_sorted_metrics() + self.assertEqual(len(metrics), 1) + self.assert_metric_expected( + metrics[0], + [ + self.create_number_data_point( + value=0, + attributes={"pool.name": pool_name, "state": "idle"}, + ), + self.create_number_data_point( + value=0, + attributes={"pool.name": pool_name, "state": "used"}, + ), + ], + ) diff --git a/tox.ini b/tox.ini index 1603dfb745..aea0b49d4a 100644 --- a/tox.ini +++ b/tox.ini @@ -6,6 +6,10 @@ envlist = ; Environments are organized by individual package, allowing ; for specifying supported Python versions per package. + ; opentelemetry-resource-detector-container + py3{7,8,9,10,11}-test-resource-detector-container + pypy3-test-resource-detector-container + ; opentelemetry-sdk-extension-aws py3{7,8,9,10,11}-test-sdkextension-aws pypy3-test-sdkextension-aws @@ -80,20 +84,20 @@ envlist = pypy3-test-instrumentation-fastapi ; opentelemetry-instrumentation-flask - py3{7,8,9,10,11}-test-instrumentation-flask - pypy3-test-instrumentation-flask + py3{7,8,9,10,11}-test-instrumentation-flask{213,220} + pypy3-test-instrumentation-flask{213,220} ; opentelemetry-instrumentation-urllib py3{7,8,9,10,11}-test-instrumentation-urllib pypy3-test-instrumentation-urllib ; opentelemetry-instrumentation-urllib3 - py3{7,8,9,10,11}-test-instrumentation-urllib3 - pypy3-test-instrumentation-urllib3 + py3{7,8,9,10,11}-test-instrumentation-urllib3v{1,2} + ;pypy3-test-instrumentation-urllib3v{1,2} ; opentelemetry-instrumentation-requests py3{7,8,9,10,11}-test-instrumentation-requests - pypy3-test-instrumentation-requests + ;pypy3-test-instrumentation-requests ; opentelemetry-instrumentation-starlette. py3{7,8,9,10,11}-test-instrumentation-starlette @@ -117,6 +121,10 @@ envlist = py3{7,8,9,10,11}-test-instrumentation-mysql pypy3-test-instrumentation-mysql + ; opentelemetry-instrumentation-mysqlclient + py3{7,8,9,10,11}-test-instrumentation-mysqlclient + pypy3-test-instrumentation-mysqlclient + ; opentelemetry-instrumentation-psycopg2 py3{7,8,9,10,11}-test-instrumentation-psycopg2 ; ext-psycopg2 intentionally excluded from pypy3 @@ -186,7 +194,7 @@ envlist = ; opentelemetry-instrumentation-tornado py3{7,8,9,10,11}-test-instrumentation-tornado - pypy3-test-instrumentation-tornado + pypy3-test-instrumentation-tornado ; opentelemetry-instrumentation-tortoiseorm py3{7,8,9,10,11}-test-instrumentation-tortoiseorm @@ -224,6 +232,10 @@ envlist = ; // FIXME: Enable support for python 3.11 when https://github.com/confluentinc/confluent-kafka-python/issues/1452 is fixed py3{7,8,9,10}-test-instrumentation-confluent-kafka + ; opentelemetry-instrumentation-cassandra + py3{7,8,9,10,11}-test-instrumentation-cassandra + pypy3-test-instrumentation-cassandra + lint spellcheck docker-tests @@ -251,9 +263,13 @@ deps = ; FIXME: Elasticsearch >=7 causes CI workflow tests to hang, see open-telemetry/opentelemetry-python-contrib#620 ; elasticsearch7: elasticsearch-dsl>=7.0,<8.0 ; elasticsearch7: elasticsearch>=7.0,<8.0 + ; elasticsearch8: elasticsearch-dsl>=8.0,<9.0 + ; elasticsearch8: elasticsearch>=8.0,<9.0 falcon1: falcon ==1.4.1 falcon2: falcon >=2.0.0,<3.0.0 falcon3: falcon >=3.0.0,<4.0.0 + flask213: Flask ==2.1.3 + flask220: Flask >=2.2.0 grpc: pytest-asyncio sqlalchemy11: sqlalchemy>=1.1,<1.2 sqlalchemy14: aiosqlite @@ -271,7 +287,9 @@ deps = httpx18: httpx>=0.18.0,<0.19.0 httpx18: respx~=0.17.0 httpx21: httpx>=0.19.0 - httpx21: respx~=0.19.0 + httpx21: respx~=0.20.1 + urllib3v1: urllib3 >=1.0.0,<2.0.0 + urllib3v2: urllib3 >=2.0.0,<3.0.0 ; FIXME: add coverage testing ; FIXME: add mypy testing @@ -294,21 +312,23 @@ changedir = test-instrumentation-boto: instrumentation/opentelemetry-instrumentation-boto/tests test-instrumentation-botocore: instrumentation/opentelemetry-instrumentation-botocore/tests test-instrumentation-boto3sqs: instrumentation/opentelemetry-instrumentation-boto3sqs/tests + test-instrumentation-cassandra: instrumentation/opentelemetry-instrumentation-cassandra/tests test-instrumentation-celery: instrumentation/opentelemetry-instrumentation-celery/tests test-instrumentation-dbapi: instrumentation/opentelemetry-instrumentation-dbapi/tests test-instrumentation-django{1,2,3,4}: instrumentation/opentelemetry-instrumentation-django/tests test-instrumentation-elasticsearch{2,5,6}: instrumentation/opentelemetry-instrumentation-elasticsearch/tests test-instrumentation-falcon{1,2,3}: instrumentation/opentelemetry-instrumentation-falcon/tests test-instrumentation-fastapi: instrumentation/opentelemetry-instrumentation-fastapi/tests - test-instrumentation-flask: instrumentation/opentelemetry-instrumentation-flask/tests + test-instrumentation-flask{213,220}: instrumentation/opentelemetry-instrumentation-flask/tests test-instrumentation-urllib: instrumentation/opentelemetry-instrumentation-urllib/tests - test-instrumentation-urllib3: instrumentation/opentelemetry-instrumentation-urllib3/tests + test-instrumentation-urllib3v{1,2}: instrumentation/opentelemetry-instrumentation-urllib3/tests test-instrumentation-grpc: instrumentation/opentelemetry-instrumentation-grpc/tests test-instrumentation-jinja2: instrumentation/opentelemetry-instrumentation-jinja2/tests test-instrumentation-kafka-python: instrumentation/opentelemetry-instrumentation-kafka-python/tests test-instrumentation-confluent-kafka: instrumentation/opentelemetry-instrumentation-confluent-kafka/tests test-instrumentation-logging: instrumentation/opentelemetry-instrumentation-logging/tests test-instrumentation-mysql: instrumentation/opentelemetry-instrumentation-mysql/tests + test-instrumentation-mysqlclient: instrumentation/opentelemetry-instrumentation-mysqlclient/tests test-instrumentation-pika{0,1}: instrumentation/opentelemetry-instrumentation-pika/tests test-instrumentation-aio-pika{7,8,9}: instrumentation/opentelemetry-instrumentation-aio-pika/tests test-instrumentation-psycopg2: instrumentation/opentelemetry-instrumentation-psycopg2/tests @@ -330,6 +350,7 @@ changedir = test-instrumentation-httpx{18,21}: instrumentation/opentelemetry-instrumentation-httpx/tests test-util-http: util/opentelemetry-util-http/tests test-sdkextension-aws: sdk-extension/opentelemetry-sdk-extension-aws/tests + test-resource-detector-container: resource/opentelemetry-resource-detector-container/tests test-propagator-aws: propagator/opentelemetry-propagator-aws-xray/tests test-propagator-ot-trace: propagator/opentelemetry-propagator-ot-trace/tests test-exporter-richconsole: exporter/opentelemetry-exporter-richconsole/tests @@ -360,8 +381,8 @@ commands_pre = grpc: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-grpc[test] - falcon{1,2,3},flask,django{1,2,3,4},pyramid,tornado,starlette,fastapi,aiohttp,asgi,requests,urllib,urllib3,wsgi: pip install {toxinidir}/util/opentelemetry-util-http[test] - wsgi,falcon{1,2,3},flask,django{1,2,3,4},pyramid: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-wsgi[test] + falcon{1,2,3},flask{213,220},django{1,2,3,4},pyramid,tornado,starlette,fastapi,aiohttp,asgi,requests,urllib,urllib3v{1,2},wsgi: pip install {toxinidir}/util/opentelemetry-util-http[test] + wsgi,falcon{1,2,3},flask{213,220},django{1,2,3,4},pyramid: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-wsgi[test] asgi,django{3,4},starlette,fastapi: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-asgi[test] asyncpg: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-asyncpg[test] @@ -375,14 +396,16 @@ commands_pre = falcon{1,2,3}: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-falcon[test] - flask: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-flask[test] + flask{213,220}: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-flask[test] urllib: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-urllib[test] - urllib3: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-urllib3[test] + urllib3v{1,2}: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-urllib3[test] botocore: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-botocore[test] + cassandra: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-cassandra[test] + dbapi: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-dbapi[test] django{1,2,3,4}: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-django[test] @@ -391,6 +414,8 @@ commands_pre = mysql: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-dbapi {toxinidir}/instrumentation/opentelemetry-instrumentation-mysql[test] + mysqlclient: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-dbapi {toxinidir}/instrumentation/opentelemetry-instrumentation-mysqlclient[test] + pymemcache{135,200,300,342,400}: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-pymemcache[test] pymongo: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-pymongo[test] @@ -441,6 +466,8 @@ commands_pre = sdkextension-aws: pip install {toxinidir}/sdk-extension/opentelemetry-sdk-extension-aws[test] + resource-detector-container: pip install {toxinidir}/resource/opentelemetry-resource-detector-container[test] + http: pip install {toxinidir}/util/opentelemetry-util-http[test] ; In order to get a health coverage report, propagator-ot-trace: pip install {toxinidir}/propagator/opentelemetry-propagator-ot-trace[test] @@ -482,7 +509,7 @@ commands = codespell [testenv:lint] -basepython: python3.10 +basepython: python3.9 recreate = False deps = -c dev-requirements.txt @@ -513,6 +540,7 @@ commands_pre = python -m pip install -e {toxinidir}/instrumentation/opentelemetry-instrumentation-boto[test] python -m pip install -e {toxinidir}/instrumentation/opentelemetry-instrumentation-flask[test] python -m pip install -e {toxinidir}/instrumentation/opentelemetry-instrumentation-sqlalchemy[test] + python -m pip install -e {toxinidir}/instrumentation/opentelemetry-instrumentation-cassandra[test] python -m pip install -e {toxinidir}/instrumentation/opentelemetry-instrumentation-celery[test] python -m pip install -e {toxinidir}/instrumentation/opentelemetry-instrumentation-pika[test] python -m pip install -e {toxinidir}/instrumentation/opentelemetry-instrumentation-aio-pika[test] @@ -534,6 +562,7 @@ commands_pre = python -m pip install -e {toxinidir}/instrumentation/opentelemetry-instrumentation-urllib[test] python -m pip install -e {toxinidir}/instrumentation/opentelemetry-instrumentation-urllib3[test] python -m pip install -e {toxinidir}/instrumentation/opentelemetry-instrumentation-pymysql[test] + python -m pip install -e {toxinidir}/instrumentation/opentelemetry-instrumentation-mysqlclient[test] python -m pip install -e {toxinidir}/instrumentation/opentelemetry-instrumentation-pymongo[test] python -m pip install -e {toxinidir}/instrumentation/opentelemetry-instrumentation-elasticsearch[test] python -m pip install -e {toxinidir}/instrumentation/opentelemetry-instrumentation-asyncpg[test] @@ -546,6 +575,7 @@ commands_pre = python -m pip install -e {toxinidir}/exporter/opentelemetry-exporter-richconsole[test] python -m pip install -e {toxinidir}/exporter/opentelemetry-exporter-prometheus-remote-write[test] python -m pip install -e {toxinidir}/sdk-extension/opentelemetry-sdk-extension-aws[test] + python -m pip install -e {toxinidir}/resource/opentelemetry-resource-detector-container[test] python -m pip install -e {toxinidir}/propagator/opentelemetry-propagator-aws-xray[test] python -m pip install -e {toxinidir}/propagator/opentelemetry-propagator-ot-trace[test] python -m pip install -e {toxinidir}/opentelemetry-distro[test] @@ -573,6 +603,8 @@ deps = pyodbc~=4.0.30 flaky==3.7.0 remoulade>=0.50 + mysqlclient~=2.1.1 + pyyaml==5.3.1 changedir = tests/opentelemetry-docker-tests/tests @@ -590,6 +622,7 @@ commands_pre = -e {toxinidir}/instrumentation/opentelemetry-instrumentation-confluent-kafka \ -e {toxinidir}/instrumentation/opentelemetry-instrumentation-dbapi \ -e {toxinidir}/instrumentation/opentelemetry-instrumentation-mysql \ + -e {toxinidir}/instrumentation/opentelemetry-instrumentation-mysqlclient \ -e {toxinidir}/instrumentation/opentelemetry-instrumentation-psycopg2 \ -e {toxinidir}/instrumentation/opentelemetry-instrumentation-pymongo \ -e {toxinidir}/instrumentation/opentelemetry-instrumentation-pymysql \ diff --git a/util/opentelemetry-util-http/src/opentelemetry/util/http/__init__.py b/util/opentelemetry-util-http/src/opentelemetry/util/http/__init__.py index f3d39ab02f..4f4a5d0353 100644 --- a/util/opentelemetry-util-http/src/opentelemetry/util/http/__init__.py +++ b/util/opentelemetry-util-http/src/opentelemetry/util/http/__init__.py @@ -16,7 +16,7 @@ from re import IGNORECASE as RE_IGNORECASE from re import compile as re_compile from re import search -from typing import Iterable, List +from typing import Iterable, List, Optional from urllib.parse import urlparse, urlunparse from opentelemetry.semconv.trace import SpanAttributes @@ -31,6 +31,10 @@ "OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE" ) +OTEL_PYTHON_INSTRUMENTATION_HTTP_CAPTURE_ALL_METHODS = ( + "OTEL_PYTHON_INSTRUMENTATION_HTTP_CAPTURE_ALL_METHODS" +) + # List of recommended metrics attributes _duration_attrs = { SpanAttributes.HTTP_METHOD, @@ -186,6 +190,15 @@ def normalise_response_header_name(header: str) -> str: key = header.lower().replace("-", "_") return f"http.response.header.{key}" +def sanitize_method(method: Optional[str]) -> Optional[str]: + if method is None: + return None + method = method.upper() + if (environ.get(OTEL_PYTHON_INSTRUMENTATION_HTTP_CAPTURE_ALL_METHODS) or + # Based on https://www.rfc-editor.org/rfc/rfc7231#section-4.1 and https://www.rfc-editor.org/rfc/rfc5789#section-2. + method in ["GET", "HEAD", "POST", "PUT", "DELETE", "CONNECT", "OPTIONS", "TRACE", "PATCH"]): + return method + return "UNKNOWN" def get_custom_headers(env_var: str) -> List[str]: custom_headers = environ.get(env_var, []) diff --git a/util/opentelemetry-util-http/src/opentelemetry/util/http/version.py b/util/opentelemetry-util-http/src/opentelemetry/util/http/version.py index eb62a67e28..c2996671d6 100644 --- a/util/opentelemetry-util-http/src/opentelemetry/util/http/version.py +++ b/util/opentelemetry-util-http/src/opentelemetry/util/http/version.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.39b0.dev" +__version__ = "0.42b0.dev" diff --git a/util/opentelemetry-util-http/tests/test_sanitize_method.py b/util/opentelemetry-util-http/tests/test_sanitize_method.py new file mode 100644 index 0000000000..a488ef589e --- /dev/null +++ b/util/opentelemetry-util-http/tests/test_sanitize_method.py @@ -0,0 +1,44 @@ +# Copyright The OpenTelemetry Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import unittest +from unittest.mock import patch + +from opentelemetry.util.http import ( + OTEL_PYTHON_INSTRUMENTATION_HTTP_CAPTURE_ALL_METHODS, + sanitize_method, +) + +class TestSanitizeMethod(unittest.TestCase): + def test_standard_method_uppercase(self): + method = sanitize_method("GET") + self.assertEqual(method, "GET") + + def test_standard_method_lowercase(self): + method = sanitize_method("get") + self.assertEqual(method, "GET") + + def test_nonstandard_method(self): + method = sanitize_method("UNKNOWN") + self.assertEqual(method, "NONSTANDARD") + + @patch.dict( + "os.environ", + { + OTEL_PYTHON_INSTRUMENTATION_HTTP_CAPTURE_ALL_METHODS: "1", + }, + ) + def test_nonstandard_method_allowed(self): + method = sanitize_method("UNKNOWN") + self.assertEqual(method, "NONSTANDARD")