Skip to content

feat(spans): Add flush_id to flushed segment messages#113416

Merged
untitaker merged 3 commits into
masterfrom
untitaker/feat/span-buffer-flush-id
Apr 20, 2026
Merged

feat(spans): Add flush_id to flushed segment messages#113416
untitaker merged 3 commits into
masterfrom
untitaker/feat/span-buffer-flush-id

Conversation

@untitaker

@untitaker untitaker commented Apr 20, 2026

Copy link
Copy Markdown
Member

Add a unique UUID (flush_id) to each message produced by FlushedSegment.to_messages(). The UUID is generated at call time, right before JSON serialization.

This allows us to see whether the duplicate segments we observe "come from" redis, or whether they are created post-serialization.

ref INC-2116
ref STREAM-881

Add a unique UUID (flush_id) to each message produced by FlushedSegment.to_messages(). The UUID is generated at call time, ensuring that if a segment is flushed twice from Redis (e.g., due to race conditions), each flush produces distinct IDs that downstream consumers can use for deduplication.

Refs INC-2116
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Apr 20, 2026
@untitaker
untitaker marked this pull request as ready for review April 20, 2026 13:09
@untitaker
untitaker requested a review from a team as a code owner April 20, 2026 13:09
@github-actions

Copy link
Copy Markdown
Contributor

Backend Test Failures

Failures on 8f5f70c in this run:

tests/sentry/spans/consumers/process/test_consumer.py::test_basic[None]log
[gw1] linux -- Python 3.13.1 /home/runner/work/sentry/sentry/.venv/bin/python3
tests/sentry/spans/consumers/process/test_consumer.py:83: in test_basic
    assert orjson.loads(msg.value) == {
E   AssertionError: assert {'flush_id': ...d': 12, ...}]} == {'spans': [{'...d': 12, ...}]}
E     
E     Omitting 1 identical items, use -vv to show
E     Left contains 1 more item:
E     {'flush_id': '8f0f55dffba7434e84a8b1a12bdb8dee'}
E     
E     Full diff:
E       {
E     +     'flush_id': '8f0f55dffba7434e84a8b1a12bdb8dee',
E           'spans': [
E               {
E                   'attributes': {
E                       'sentry.segment.id': {
E                           'type': 'string',
E                           'value': 'aaaaaaaaaaaaaaaa',
E                       },
E                   },
E                   'end_timestamp': 1700000000.0,
E                   'is_segment': True,
E                   'project_id': 12,
E                   'span_id': 'aaaaaaaaaaaaaaaa',
E                   'start_timestamp': 1699999999.0,
E                   'trace_id': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb',
E               },
E           ],
E       }
tests/sentry/spans/consumers/process/test_consumer.py::test_basic[2]log
[gw1] linux -- Python 3.13.1 /home/runner/work/sentry/sentry/.venv/bin/python3
tests/sentry/spans/consumers/process/test_consumer.py:83: in test_basic
    assert orjson.loads(msg.value) == {
E   AssertionError: assert {'flush_id': ...d': 12, ...}]} == {'spans': [{'...d': 12, ...}]}
E     
E     Omitting 1 identical items, use -vv to show
E     Left contains 1 more item:
E     {'flush_id': '7a5d094aebb44cab9741d69ac9393508'}
E     
E     Full diff:
E       {
E     +     'flush_id': '7a5d094aebb44cab9741d69ac9393508',
E           'spans': [
E               {
E                   'attributes': {
E                       'sentry.segment.id': {
E                           'type': 'string',
E                           'value': 'aaaaaaaaaaaaaaaa',
E                       },
E                   },
E                   'end_timestamp': 1700000000.0,
E                   'is_segment': True,
E                   'project_id': 12,
E                   'span_id': 'aaaaaaaaaaaaaaaa',
E                   'start_timestamp': 1699999999.0,
E                   'trace_id': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb',
E               },
E           ],
E       }

@untitaker
untitaker enabled auto-merge (squash) April 20, 2026 13:47
@untitaker
untitaker merged commit 100bcc2 into master Apr 20, 2026
56 checks passed
@untitaker
untitaker deleted the untitaker/feat/span-buffer-flush-id branch April 20, 2026 13:53
@linear-code

linear-code Bot commented Apr 20, 2026

Copy link
Copy Markdown

joshuarli added a commit that referenced this pull request Apr 22, 2026
…ommits

spans/test_buffer.py:
- test_to_messages_under_limit: our commit reverted the flush_id
  assertions added in #113416; restore messages[0]["spans"] == spans
  with flush_id checks
- test_max_segment_spans_limit: our commit wrongly expected segment.spans
  == [] (dropped); master expects len == 5 (chunked at message level)
- test_dropped_spans_emit_outcomes: missing enforce-segment-size: True,
  so the Lua script never drops spans and track_outcome is never called

integrations/opsgenie/test_integration.py: our commit replaced
OpsgenieUpdateConfigTest(TestCase) with OpsgenieIntegrationTest
(IntegrationTestCase) which uses the old web pipeline. Opsgenie now
has get_pipeline_views() = [], so any GET immediately calls
finish_pipeline() without installation_data. Restore master's class.

core/endpoints/test_organization_details.py: our commit removed str()
from defaultCodingAgentIntegrationId assertions; the API returns string
IDs (Sentry convention), so str(integration.id) is correct.

incidents/endpoints/test_organization_alert_rule_details.py: skip
test_delete_trigger_dual_update_resolve — CrossTransactionAssertionError
in _pre_setup() when a prior test leaves an open silo DB transaction.
@github-actions github-actions Bot locked and limited conversation to collaborators May 6, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants