Skip to content

feat(replays): Add Breadcrumb AI Summaries #93256

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 28 commits into
base: master
Choose a base branch
from
Open

feat(replays): Add Breadcrumb AI Summaries #93256

wants to merge 28 commits into from
+719 −7

Conversation

cmanallen
Copy link
Member

TODO

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Jun 10, 2025
@cmanallen cmanallen marked this pull request as ready for review June 10, 2025 20:06
@cmanallen cmanallen requested review from a team as code owners June 10, 2025 20:06
@codecov Codecov
Copy link

codecov bot commented Jun 10, 2025 •
edited
Loading

❌ 12 Tests Failed:

Tests completed Failed Passed Skipped
112 12 100 2
View the top 3 failed test(s) by shortest run time 
::tests.sentry.api.endpoints.test_system_options
Stack Traces | 0s run time 
#x1B[1m#x1B[.../api/endpoints/test_system_options.py#x1B[0m:9: in <module>
    class SystemOptionsTest(APITestCase):
#x1B[1m#x1B[.../api/endpoints/test_system_options.py#x1B[0m:10: in SystemOptionsTest
    url = reverse("sentry-api-0-system-options")
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/urls/base.py#x1B[0m:98: in reverse
    resolved_url = resolver._reverse_with_prefix(view, prefix, *args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/urls/resolvers.py#x1B[0m:749: in _reverse_with_prefix
    self._populate()
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/urls/resolvers.py#x1B[0m:548: in _populate
    for url_pattern in reversed(self.url_patterns):
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/utils/functional.py#x1B[0m:47: in __get__
    res = instance.__dict__[self.name] = self.func(instance)
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/urls/resolvers.py#x1B[0m:718: in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/utils/functional.py#x1B[0m:47: in __get__
    res = instance.__dict__[self.name] = self.func(instance)
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/urls/resolvers.py#x1B[0m:711: in urlconf_module
    return import_module(self.urlconf_name)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.13.1....../x64/lib/python3.13/importlib/__init__.py#x1B[0m:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
#x1B[1m#x1B[31m<frozen importlib._bootstrap>#x1B[0m:1387: in _gcd_import
    ???
#x1B[1m#x1B[31m<frozen importlib._bootstrap>#x1B[0m:1360: in _find_and_load
    ???
#x1B[1m#x1B[31m<frozen importlib._bootstrap>#x1B[0m:1331: in _find_and_load_unlocked
    ???
#x1B[1m#x1B[31m<frozen importlib._bootstrap>#x1B[0m:935: in _load_unlocked
    ???
#x1B[1m#x1B[31m<frozen importlib._bootstrap_external>#x1B[0m:1026: in exec_module
    ???
#x1B[1m#x1B[31m<frozen importlib._bootstrap>#x1B[0m:488: in _call_with_frames_removed
    ???
#x1B[1m#x1B[.../sentry/conf/urls.py#x1B[0m:3: in <module>
    from sentry.web.urls import urlpatterns
#x1B[1m#x1B[.../sentry/web/urls.py#x1B[0m:168: in <module>
    include("sentry.api.urls"),
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/urls/conf.py#x1B[0m:39: in include
    urlconf_module = import_module(urlconf_module)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.13.1....../x64/lib/python3.13/importlib/__init__.py#x1B[0m:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
#x1B[1m#x1B[.../sentry/api/urls.py#x1B[0m:326: in <module>
    from sentry.replays.endpoints.project_replay_summarize_breadcrumbs import (
#x1B[1m#x1B[.../replays/endpoints/project_replay_summarize_breadcrumbs.py#x1B[0m:61: in <module>
    def analyze_recording_segments(segments: list[RecordingSegmentStorageMeta]) -> dict[str, Any]:
#x1B[1m#x1B[31mE   NameError: name 'Any' is not defined#x1B[0m
::tests.sentry.hybridcloud.apigateway.test_apigateway
Stack Traces | 0s run time 
#x1B[1m#x1B[.../hybridcloud/apigateway/test_apigateway.py#x1B[0m:11: in <module>
    from sentry.testutils.helpers.apigateway import ApiGatewayTestCase, verify_request_params
#x1B[1m#x1B[31m<frozen importlib._bootstrap>#x1B[0m:1360: in _find_and_load
    ???
#x1B[1m#x1B[31m<frozen importlib._bootstrap>#x1B[0m:1331: in _find_and_load_unlocked
    ???
#x1B[1m#x1B[31m<frozen importlib._bootstrap>#x1B[0m:935: in _load_unlocked
    ???
#x1B[1m#x1B[31m.venv/lib/python3.13.../_pytest/assertion/rewrite.py#x1B[0m:178: in exec_module
    exec(co, module.__dict__)
#x1B[1m#x1B[.../testutils/helpers/apigateway.py#x1B[0m:13: in <module>
    import sentry.api.urls as api_urls
#x1B[1m#x1B[.../sentry/api/urls.py#x1B[0m:326: in <module>
    from sentry.replays.endpoints.project_replay_summarize_breadcrumbs import (
#x1B[1m#x1B[.../replays/endpoints/project_replay_summarize_breadcrumbs.py#x1B[0m:61: in <module>
    def analyze_recording_segments(segments: list[RecordingSegmentStorageMeta]) -> dict[str, Any]:
#x1B[1m#x1B[31mE   NameError: name 'Any' is not defined#x1B[0m
::tests.sentry.replays.test_project_replay_summarize_breadcrumbs
Stack Traces | 0s run time 
#x1B[1m#x1B[.../sentry/replays/test_project_replay_summarize_breadcrumbs.py#x1B[0m:8: in <module>
    from sentry.replays.endpoints.project_replay_summarize_breadcrumbs import get_request_data
#x1B[1m#x1B[.../replays/endpoints/project_replay_summarize_breadcrumbs.py#x1B[0m:61: in <module>
    def analyze_recording_segments(segments: list[RecordingSegmentStorageMeta]) -> dict[str, Any]:
#x1B[1m#x1B[31mE   NameError: name 'Any' is not defined#x1B[0m

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

michellewzhang
michellewzhang reviewed Jun 10, 2025
View reviewed changes
Copy link
Member

@michellewzhang michellewzhang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we link the relevant linear tickets in the description from https://linear.app/getsentry/project/replay-breadcrumb-summary-7897b9280afa/overview? (i know it says TODO rn)

this one is related:

src/sentry/replays/blueprints/api.md
Comment on lines +593 to +594
| title | str | The main title of the user journey summary. |
| summary | str | A concise summary featuring the highlights of the user's journey while using the application. |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the difference between title and summary? do we need both?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its a condensed summary. We could drop it if its not useful. I want to add more fields before we take away though (at least during the early periods while we're experimenting).

billyvg
billyvg reviewed Jun 11, 2025
View reviewed changes
aliu39
aliu39 reviewed Jun 11, 2025
View reviewed changes
src/sentry/replays/blueprints/api.md
| ------------------------ | --------------- | --------------------------------------------------------------------------------------------- |
| title | str | The main title of the user journey summary. |
| summary | str | A concise summary featuring the highlights of the user's journey while using the application. |
| time_ranges | list[TimeRange] | A list of TimeRange objects. |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| time_ranges | list[TimeRange] | A list of TimeRange objects. |
| time_ranges | list[TimeRange] | An ordered list of TimeRange objects. |

aliu39
aliu39 reviewed Jun 11, 2025
View reviewed changes
Copy link
Member

@aliu39 aliu39 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, I don't know specifics for the event formats though. Do we have any docs for replay event/breadcrumb types?

src/sentry/replays/blueprints/api.md
"period_end": 1749584992.912,
"period_title": "Second Replay Load Failure"
}
]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could add a second example to show the ordering and relationship of start/end (will all ranges be disjoint? no gaps in between?)

src/sentry/replays/endpoints/project_replay_summarize_breadcrumbs.py
return response.content


def get_request_data(iterator: Iterator[tuple[int, memoryview]]) -> list[str]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def get_request_data(iterator: Iterator[tuple[int, memoryview]]) -> list[str]:
def get_request_data(segment_iterator: Iterator[tuple[int, memoryview]]) -> list[str]:

@michellewzhang
Copy link
Member

looks good, I don't know specifics for the event formats though. Do we have any docs for replay event/breadcrumb types?

@aliu39 if you look at node_modules/.pnpm/@sentry-internal+replay@9.24.0/node_modules/@sentry-internal/replay/build/npm/types/types/replayFrame.d.ts (or search for ReplayBreadcrumbFrame in the codebase the file might be easier to find) there are some types in there

@cmanallen @aliu39
Update src/sentry/replays/endpoints/project_replay_summarize_breadcru…
1680fd4 
…mbs.py

Co-authored-by: Andrew Liu <159852527+aliu39@users.noreply.github.com>
michellewzhang
michellewzhang reviewed Jun 12, 2025
View reviewed changes
src/sentry/replays/endpoints/project_replay_summarize_breadcrumbs.py
},
)
if response.status_code != 200:
raise ParseError("A non 200 HTTP status code was returned.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we make this error more detailed? when i was getting errors testing in devserver it was tough to figure out that it was coming from here, and what error was happening

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(it was a google cloud auth error but i didn't know until i looked at my seer terminal)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, think it'd be better to raise a ValueError or Exception so the code is 500 instead of 400, since it's not a user triggered error right? And the error message can contain the specific Seer response code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@michellewzhang michellewzhang michellewzhang left review comments

@aliu39 aliu39 aliu39 approved these changes

@billyvg billyvg billyvg approved these changes

Assignees
No one assigned
Labels
Scope: Backend Automatically applied to PRs that change backend components
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@cmanallen @michellewzhang @billyvg @aliu39