Do not mark every frame as in-app

[armenzg]

Problem Statement

Most errors on Sentry with stacktraces have every frame marked as in_app even though many of the files are part of the 3rd party libraries. This causes issues for the commit context and issue details page.

Solution Brainstorm

This can be avoided if we look at the files that belong to the virtualenv. If a file is under os.environ.get("VIRTUAL_ENV") then it should be marked as in_app: False. Or even check that the abs path includes site-packages (e.g. /some/path/repo/.venv/lib/python3.10/site-packages/django/views/decorators/csrf.py).

This would be a huge win for suspect commits (since we always look for the first in-app frame) and issue details page (since we load the first in-app frame).

CC @NisanthanNanthakumar @scttcper @asottile-sentry

[asottile-sentry]

the way classify-imports does this is it treats anything in [\/](?:dist|site)-packages[\/] as third party -- could be an easy filter

[antonpirker]

This is also related to this issue #320

[armenzg]

I added some naive logic to mark certain requests as in-app for the BE:
https://github.com/getsentry/sentry/blob/75a049207fb9ab51ce422c8f18bb025226c91ed8/src/sentry/api/endpoints/project_stacktrace_link.py#L84-L86

There's only one false positive because the org that produces the 3rd party library has connected the code to their own code (I can't share a screenshot because of PII).

Here's a query wrt to accuracy broken down if I marked it as in-app (I also separate between derived code mappings vs not):
query: transaction:/api/0/projects/{organization_slug}/{project_slug}/stacktrace-link/ stacktrace_link.found:[True,False] stacktrace_link.platform:python stacktrace_link.in_app:[True,False] stacktrace_link.auto_derived:[False,True] stacktrace_link.code_mappings:True

When users define the code-mapping, we get an 8x increase if we discard paths that include site-packages.

We could start with the naive approach and then give customers documentation on how to adjust things if we mark them incorrectly.

[Zylphrex]

I took a very similar approach in classifying python frames as in app/not in the profiling service getsentry/vroom#111. The main difference here is that I had to compile a list of standard library module names from the python docs and source and explicitly marked those as not in app.