Skip to content

Conversation

eli-r-ph
Copy link
Contributor

@eli-r-ph eli-r-ph commented Apr 16, 2025

Problem

We'd like to normalize all the event drop and rerouting (overflow etc.) across ingest services so this functionality is easier to use for maintainers.

Changes

Extracted the Python-specific (Django capture) changes from this closed PR.

🎗️ Will require a matching charts change to shift preexisting settings to the new var and format.

Does this work well for both Cloud and self-hosted?

Yes

How did you test this code?

Locally and in CI

@eli-r-ph eli-r-ph requested a review from a team April 16, 2025 21:09
@eli-r-ph eli-r-ph self-assigned this Apr 16, 2025
@nickbest-ph
Copy link
Contributor

nickbest-ph commented Apr 16, 2025

Is there a corresponding charts PR that needs to go out first? Or is this env var already set on the django pods?

@eli-r-ph
Copy link
Contributor Author

eli-r-ph commented Apr 16, 2025

I need to check on that since the naming is changed but I assume yes on the charts deploy - noted in the PR desc as well 👍

Copy link
Contributor

@pl pl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, do we have any dropped tokens in django capture on prod?

@pl
Copy link
Contributor

pl commented Apr 16, 2025

Also, I don't think any of the test failures are related to your changes.

@eli-r-ph eli-r-ph force-pushed the eli.r/normalize-ingest-topic-routing-py branch from 38f81ab to 500b74b Compare April 17, 2025 19:53
@eli-r-ph eli-r-ph marked this pull request as ready for review April 17, 2025 19:59
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

This PR normalizes event dropping functionality in PostHog's ingestion services by renaming environment variables and modifying token/distinct ID pair handling.

  • Changed environment variable from DROPPED_KEYS to DROP_EVENTS_BY_TOKEN_DISTINCT_ID in /posthog/settings/ingestion.py for clearer intent
  • Modified delimiter from semicolon to comma for token:distinct_id pairs in /posthog/api/capture.py
  • Made distinct_id optional in token:distinct_id pairs for more flexible event dropping
  • Reorganized Kafka topic imports in /posthog/api/capture.py for better code organization

2 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile

@eli-r-ph eli-r-ph requested review from pl and a team and removed request for pl April 18, 2025 00:03
Copy link
Contributor

@nickbest-ph nickbest-ph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@eli-r-ph
Copy link
Contributor Author

👋 working up corresponding PR to ensure the deployed env vars are set to expect this change

@eli-r-ph
Copy link
Contributor Author

I couldn't find any reference to DROPPED_KEYS env var in deployment repo so I think we're gtg here

@eli-r-ph eli-r-ph merged commit a6d76cf into master Apr 18, 2025
79 checks passed
@eli-r-ph eli-r-ph deleted the eli.r/normalize-ingest-topic-routing-py branch April 18, 2025 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants