Skip to content

feat(spans): Evict spans during insert - #92393

Merged
jan-auer merged 3 commits into
masterfrom
ref/spans-evict-oldest
Jun 2, 2025
Merged

jan-auer merged 3 commits into
masterfrom
ref/spans-evict-oldest

Conversation

@jan-auer

@jan-auer jan-auer commented May 28, 2025 •

Copy link
Copy Markdown
Member

A proof of concept that limits the number of spans per segment during insertion. Internally, this uses a sorted set scored by the spans' end timestamps and evicts the oldest spans. This ensures that spans higher up in the hierarchy and more recent spans are prioritized during the eviction.

Ref VIEPF-15

@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label May 28, 2025
* master: (249 commits)
  feat(source-maps): Do not show pagination together with empty state (#92287)
  ref(project-creation): Introduce useCreateProjectRules hook (#92186)
  feat(agent-insights): Handle new keys (#92613)
  feat(source-maps): Introduce new empty state copies and react-native callout (#92286)
  ref(issues): Remove project from group activity type (#92600)
  feat(ourlogs): Use /trace-logs endpoint (#92577)
  feat(issues): Only update group hasSeen when user is member (#92597)
  fix(workflow_engine): Graceful Data Condition Eval Handling (#92591)
  feat(uptime): Implement detector handler (#91107)
  chore(autofix): Remove logs from response payload (#92589)
  fix(search): Fix issue with tags name 'constructor' (#92586)
  fix(autofix): Fix condition for onboarding check (#92584)
  fix(ourlogs): Return the same format as /events & limit 1000 for trace-logs (#92580)
  fix(autofix): Fix automation onboarding condition (#92579)
  feat(explore): Remove group by timestamp from explore (#92546)
  feat(trace-items): Autocomplete for semver attributes (#92515)
  feat(processing) Define EventProcessingStore.__all__ (#92555)
  feat(autofix): Better errored state (#92571)
  chore(autofix): Seer beta banner copy changes (#92576)
  feat(crons): Add endpoint to return counts by status (#92574)
  ...
@codecov

codecov Bot commented Jun 2, 2025 •

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #92393      +/-   ##
==========================================
+ Coverage   87.89%   87.96%   +0.07%     
==========================================
  Files       10240    10217      -23     
  Lines      587447   586303    -1144     
  Branches    22816    22623     -193     
==========================================
- Hits       516321   515763     -558     
+ Misses      70696    70126     -570     
+ Partials      430      414      -16     

@jan-auer
jan-auer marked this pull request as ready for review June 2, 2025 09:52
@jan-auer
jan-auer requested a review from a team June 2, 2025 09:52
@jan-auer
jan-auer requested a review from a team as a code owner June 2, 2025 09:52
@jan-auer
jan-auer merged commit 7461867 into master Jun 2, 2025
@jan-auer
jan-auer deleted the ref/spans-evict-oldest branch June 2, 2025 10:06
jan-auer added a commit that referenced this pull request Jun 2, 2025
Fixes a regression in #92393, where we changed the type of data in keys
without renaming the keys. This PR changes the namespace, which is
equivalent to flushing the DB.
untitaker added a commit that referenced this pull request Jun 2, 2025
untitaker added a commit that referenced this pull request Jun 2, 2025
- **Revert "fix(spans): Rename span buf prefix (#92617)"**
- **Revert "feat(spans): Evict spans during insert (#92393)"**

This PR incurred 30% overhead on insert_spans. We only see this reported
from the consumer/client -- we don't see additional time spent in redis
commands as reported by the cluster itself.
andrewshie-sentry pushed a commit that referenced this pull request Jun 2, 2025
A proof of concept that limits the number of spans per segment during
insertion. Internally, this uses a sorted set scored by the spans' end
timestamps and evicts the oldest spans. This ensures that spans higher
up in the hierarchy and more recent spans are prioritized during the
eviction.
andrewshie-sentry pushed a commit that referenced this pull request Jun 2, 2025
Fixes a regression in #92393, where we changed the type of data in keys
without renaming the keys. This PR changes the namespace, which is
equivalent to flushing the DB.
andrewshie-sentry pushed a commit that referenced this pull request Jun 2, 2025
- **Revert "fix(spans): Rename span buf prefix (#92617)"**
- **Revert "feat(spans): Evict spans during insert (#92393)"**

This PR incurred 30% overhead on insert_spans. We only see this reported
from the consumer/client -- we don't see additional time spent in redis
commands as reported by the cluster itself.
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 17, 2025

This branch was successfully deployed

1 active deployment
Preview — eb9623b4 Deployed Jun 2, 2025 by vercel[bot]
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants