Better setting of in-app in stack frames #1894

antonpirker · 2023-02-10T08:00:16Z

How the in_app flag is set in stack trace frames (in set_in_app_in_frames()):

If there is already in_app set, it is left untouched.
If there is a module in the frame and it is in the in_app_includes -> in_app=True
If there is a module in the frame and it is in the in_app_excludes -> in_app=False
If there is an abs_path in the frame and the path is in /side-packages/ or /dist-packages/ -> in_app=False
If there is an abs_path in the frame and it starts with the current working directory of the process -> in_app=True
If nothing of the above is true, there will be no in_app set.

Fixes #1754
Fixes #320

…something else

sentry_sdk/utils.py

…tryue

sentry_sdk/client.py

Previously, in case packages added in `in_app_include` were installed into a location outside of the project root directory, stack frames from those packages were not marked as `in_app`. Cases include running Python from virtualenv, created outside of the project root directory, or Python packages installed into the system using package managers. This resulted in inconsistency: traces from the same project would have different `in_app` flags depending on the deployment method. Steps to reproduce (virtualenv outside of project root): ``` $ docker run --replace --rm --name sentry-postgres -e POSTGRES_USER=sentry -e POSTGRES_PASSWORD=sentry -d -p 5432:5432 postgres $ distrobox create -i ubuntu:24.04 -n sentry-test-in_app_include-venv $ distrobox enter sentry-test-in_app_include-venv $ python3 -m venv /tmp/.venv-test-in_app_include $ source /tmp/.venv-test-in_app_include/bin/activate $ pytest tests/integrations/django/test_db_query_data.py::test_query_source_with_in_app_include # FAIL ``` Steps to reproduce (system packages): ``` $ docker run --replace --rm --name sentry-postgres -e POSTGRES_USER=sentry -e POSTGRES_PASSWORD=sentry -d -p 5432:5432 postgres $ distrobox create -i ubuntu:24.04 -n sentry-test-in_app_include-os $ distrobox enter sentry-test-in_app_include-os $ sudo apt install python3-django python3-pytest python3-pytest-cov python3-pytest-django python3-jsonschema python3-urllib3 python3-certifi python3-werkzeug python3-psycopg2 $ pytest tests/integrations/django/test_db_query_data.py::test_query_source_with_in_app_include # FAIL ``` In this change, the logic was slightly changed to avoid these discrepancies and conform to the requirements, described in the PR with `in_app` flag introduction: getsentry#1894 (comment). Note that the `_module_in_list` function returns `False` if `name` is `None`, hence extra check before function call can be omitted to simplify code.

Previously, in case packages added in `in_app_include` were installed into a location outside of the project root directory, stack frames from those packages were not extended with OTel compatible source code information. Cases include running Python from virtualenv, created outside of the project root directory, or Python packages installed into the system using package managers. This resulted in inconsistency: traces from the same project would have different `in_app` flags depending on the deployment method. Steps to reproduce (virtualenv outside of project root): ``` $ docker run --replace --rm --name sentry-postgres -e POSTGRES_USER=sentry -e POSTGRES_PASSWORD=sentry -d -p 5432:5432 postgres $ distrobox create -i ubuntu:24.04 -n sentry-test-in_app_include-venv $ distrobox enter sentry-test-in_app_include-venv $ python3 -m venv /tmp/.venv-test-in_app_include $ source /tmp/.venv-test-in_app_include/bin/activate $ pytest tests/integrations/django/test_db_query_data.py::test_query_source_with_in_app_include # FAIL ``` Steps to reproduce (system packages): ``` $ docker run --replace --rm --name sentry-postgres -e POSTGRES_USER=sentry -e POSTGRES_PASSWORD=sentry -d -p 5432:5432 postgres $ distrobox create -i ubuntu:24.04 -n sentry-test-in_app_include-os $ distrobox enter sentry-test-in_app_include-os $ sudo apt install python3-django python3-pytest python3-pytest-cov python3-pytest-django python3-jsonschema python3-urllib3 python3-certifi python3-werkzeug python3-psycopg2 $ pytest tests/integrations/django/test_db_query_data.py::test_query_source_with_in_app_include # FAIL ``` In this change, the logic was slightly changed to avoid these discrepancies and conform to the requirements, described in the PR with better setting of in-app in stack frames: getsentry#1894 (comment). Note that the `_module_in_list` function returns `False` if `name` is `None` or `items` are falsy, hence extra check before function call can be omitted to simplify code.

…root Fix: getsentry#3312 Previously, when packages added in `in_app_include` were installed to a location outside of the project root directory, span from those packages were not extended with OTel compatible source code information. Cases include running Python from virtualenv created outside of the project root directory or Python packages installed into the system using package managers. This resulted in an inconsistency: spans from the same project would be different, depending on the deployment method. In this change, the logic was slightly changed to avoid these discrepancies and conform to the requirements, described in the PR with better setting of in-app in stack frames: getsentry#1894 (comment). Note that the `_module_in_list` function returns `False` if `name` is `None` or `items` are falsy, hence extra check before function call can be omitted to simplify code.

…root Fix: getsentry#3312 Previously, when packages added in `in_app_include` were installed to a location outside of the project root directory, span from those packages were not extended with OTel compatible source code information. Cases include running Python from virtualenv created outside of the project root directory or Python packages installed into the system using package managers. This resulted in an inconsistency: spans from the same project would be different, depending on the deployment method. In this change, the logic was slightly changed to avoid these discrepancies and conform to the requirements, described in the PR with better setting of in-app in stack frames: getsentry#1894 (comment).

…root (#3313) Fix: #3312 Previously, when packages added in `in_app_include` were installed to a location outside of the project root directory, span from those packages were not extended with OTel compatible source code information. Cases include running Python from virtualenv created outside of the project root directory or Python packages installed into the system using package managers. This resulted in an inconsistency: spans from the same project would be different, depending on the deployment method. In this change, the logic was slightly changed to avoid these discrepancies and conform to the requirements, described in the PR with better setting of in-app in stack frames: #1894 (comment). --------- Co-authored-by: Daniel Szoke <daniel.szoke@sentry.io> Co-authored-by: Daniel Szoke <7881302+szokeasaurusrex@users.noreply.github.com>

antonpirker added 5 commits February 10, 2023 08:52

Better setting of in-app

809b29d

Fixed tests

b8263b2

Handling if abs_path is not present in frame

ad07e1a

Added debug output

d97a478

Removed debug output and check for in_app because this test is about …

341c349

…something else

antonpirker marked this pull request as ready for review February 10, 2023 11:41

antonpirker requested review from sl0thentr0py and Zylphrex February 10, 2023 11:42

Zylphrex reviewed Feb 10, 2023

View reviewed changes

sentry_sdk/utils.py Outdated Show resolved Hide resolved

antonpirker added 6 commits February 10, 2023 15:21

Just set in_app if we are sure it is an external source

5bbc1bb

Fixed tests

4ae95f6

Check if frame is in project root directory and if yes set in_app to …

dc3872d

…tryue

Merge branch 'master' into antonpirker/1754-in-app

7c7355a

Made function more readable

7a2e3b5

Better name of function

ecb1fd7

Zylphrex approved these changes Feb 13, 2023

View reviewed changes

sl0thentr0py requested changes Feb 13, 2023

View reviewed changes

sentry_sdk/client.py Outdated Show resolved Hide resolved

antonpirker added 2 commits February 15, 2023 10:06

Merge branch 'master' into antonpirker/1754-in-app

707b440

Moved project_root configuration into options

905320e

sl0thentr0py reviewed Feb 15, 2023

View reviewed changes

sentry_sdk/client.py Outdated Show resolved Hide resolved

antonpirker added 3 commits February 15, 2023 11:14

Removed check because in_app is now not always set

2b80b5d

Only fetching working directory when not already set.

34b8e3b

Added type annotations

4a7c31a

sl0thentr0py approved these changes Feb 15, 2023

View reviewed changes

Added project_root to the default options

10e57a5

antonpirker merged commit bb20fc6 into master Feb 15, 2023

antonpirker deleted the antonpirker/1754-in-app branch February 15, 2023 10:51

rominf mentioned this pull request Jul 18, 2024

fix(tracing): fix logic for setting in_app flag rominf/sentry-python#1

Closed

rominf mentioned this pull request Jul 19, 2024

fix(tracing): Fix add_query_source with modules outside of project root #3313

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Better setting of in-app in stack frames #1894

Better setting of in-app in stack frames #1894

antonpirker commented Feb 10, 2023 •

edited

Loading

Better setting of in-app in stack frames #1894

Better setting of in-app in stack frames #1894

Conversation

antonpirker commented Feb 10, 2023 • edited Loading

antonpirker commented Feb 10, 2023 •

edited

Loading