-
Notifications
You must be signed in to change notification settings - Fork 1.6k
feat(cdp): batch hogfun watcher #30603
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
…ogfun-watcher # Conflicts: # frontend/src/layout/panel-layout/OrganizationDropdownMenu.tsx
Hey @meikelmosby! 👋 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
This PR refactors the ingestion pipeline to support batch processing of Hog functions and their watcher states, reducing per-event overhead.
• Updated plugin-server/src/worker/ingestion/event-pipeline/runner.ts to accept a new teamHogFunctions parameter and aggregate watcher promises.
• Modified plugin-server/src/ingestion/ingestion-consumer.ts to prefetch and filter Hog functions in batches using hogWatcher states.
• Refactored plugin-server/src/cdp/hog-transformations/hog-transformer.service.ts to simplify transformation logic and accept team hog functions.
• Aligned plugin-server/src/worker/ingestion/event-pipeline/transformEventStep.ts to maintain consistent watcherPromises output.
4 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile
plugin-server/src/cdp/hog-transformations/hog-transformer.service.ts
Outdated
Show resolved
Hide resolved
plugin-server/src/cdp/hog-transformations/hog-transformer.service.ts
Outdated
Show resolved
Hide resolved
plugin-server/src/cdp/hog-transformations/hog-transformer.service.ts
Outdated
Show resolved
Hide resolved
plugin-server/src/cdp/hog-transformations/hog-transformer.service.ts
Outdated
Show resolved
Hide resolved
plugin-server/src/cdp/hog-transformations/hog-transformer.service.ts
Outdated
Show resolved
Hide resolved
plugin-server/src/cdp/hog-transformations/hog-transformer.service.ts
Outdated
Show resolved
Hide resolved
plugin-server/src/cdp/hog-transformations/hog-transformer.service.ts
Outdated
Show resolved
Hide resolved
plugin-server/src/cdp/hog-transformations/hog-transformer.service.ts
Outdated
Show resolved
Hide resolved
plugin-server/src/cdp/hog-transformations/hog-transformer.service.ts
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Small change request otherwise looks good
Problem
Because we do not want to run getStates per Event or just using a percentage of our traffic we perform the operation using hogwatcher per batch
Changes
👉 Stay up-to-date with PostHog coding conventions for a smoother review.
How did you test this code?