Skip to content

feat(replay-vision): show impacted users and cohort per scanner#70059

Merged
fasyy612 merged 9 commits into
masterfrom
feat/impacted-users-on-scanner
Jul 17, 2026
Merged

feat(replay-vision): show impacted users and cohort per scanner#70059
fasyy612 merged 9 commits into
masterfrom
feat/impacted-users-on-scanner

Conversation

@fasyy612

@fasyy612 fasyy612 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Problem

Scanners tell you what they found, but not who it affected. There's no way to see how many sessions and users a scanner's findings cover, or to act on those users (survey them, check their retention, exclude them from an experiment) without manually transferring context out of Vision.

Changes

Adds impact counts and one-click cohorts to scanners.

  • New GET /vision/scanners/:id/impact/ returns affected sessions and users over a trailing window, counted from observations (not estimated)
  • New POST /vision/scanners/:id/affected_cohort/ saves those users as a static cohort, usable in other parts of PostHog
  • "Affected" uses vary per scanner. Verdict-yes for monitors, a required tag for classifiers, scorer should come later when we can set some 'threshold' that user can set, and Summarizers are not supported (since it's not much about seeing users that were affected)
  • Scanner page now has an impact panel with a "Save as cohort" button for monitors and per-tag cohort buttons on the classifier tag panels
  • Cohort creation reports the real member count, fails loudly instead of leaving partial cohorts behind (new raise_on_error passthrough on insert_users_by_list), and is capped at 10k users per request (can be adjusted it's an arbitrary number for now)
  • Both endpoints are exposed as MCP tools

Monitor:
Screenshot 2026-07-15 at 15 59 15
Classifier:
Screenshot 2026-07-15 at 16 05 14

How did you test this code?

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Automatic notifications

  • Publish to changelog?
  • Alert Sales and Marketing teams?

Docs update

🤖 Agent context

Autonomy: Human-driven (agent-assisted) - or - Fully autonomous

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

🤖 CI report

⚠️ Bundle size — 🔺 +1.9 KiB (+0.0%)

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 64.79 MiB · 🔺 +1.9 KiB (+0.0%)

File Size Δ vs base
posthog-app/_parent/products/replay_vision/frontend/replay_scanners/ReplayScanner.js 85.0 KiB 🔺 +1.9 KiB (+2.3%)

Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report

Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root Eager (shipped) Δ vs base Budget
entry (logged-out pages, app bootstrap)
src/index.tsx
1.22 MiB · 22 files no change ███░░░░░░░ 28.4% of 4.29 MiB
authenticated shell (every logged-in page)
src/scenes/AuthenticatedShell.tsx
8.13 MiB · 2,979 files no change █████████░ 87.9% of 9.25 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx

Largest files eagerly shipped from src/index.tsx
Size File
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
24.6 KiB ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
6.3 KiB ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js
4.5 KiB ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js
3.9 KiB ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js
1.4 KiB ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js
1.3 KiB src/RootErrorBoundary.tsx
912 B ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js
789 B src/scenes/ChunkLoadErrorBoundary.tsx
762 B src/index.tsx
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
Size File
281.3 KiB ../node_modules/.pnpm/posthog-js@1.402.3/node_modules/posthog-js/dist/rrweb.js
267.7 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
235.5 KiB src/taxonomy/core-filter-definitions-by-group.json
222.9 KiB ../node_modules/.pnpm/posthog-js@1.402.3/node_modules/posthog-js/dist/module.js
164.0 KiB src/queries/validators.js
154.3 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
105.8 KiB src/lib/api.ts
93.3 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
92.7 KiB ../packages/quill/packages/quill/dist/index.js

Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479

Dist folder size — 🔺 +23.8 KiB (+0.0%)

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 1312.76 MiB · 🔺 +23.8 KiB (+0.0%)

ℹ️ MCP UI apps size — 31 app(s), 16561.5 KB JS

Built size of each MCP UI app (main.js + styles.css).

App JS CSS
debug 598.2 KB 179.2 KB
action 456.5 KB 179.2 KB
action-list 563.0 KB 179.2 KB
cohort 455.4 KB 179.2 KB
cohort-list 562.0 KB 179.2 KB
email-template 455.3 KB 179.2 KB
error-details 471.1 KB 179.2 KB
error-issue 456.1 KB 179.2 KB
error-issue-list 562.9 KB 179.2 KB
experiment 560.1 KB 179.2 KB
experiment-list 563.8 KB 179.2 KB
experiment-results 561.8 KB 179.2 KB
feature-flag 565.8 KB 179.2 KB
feature-flag-list 569.5 KB 179.2 KB
feature-flag-testing 459.4 KB 179.2 KB
insight-actors 560.8 KB 179.2 KB
invite-email-preview 454.7 KB 179.2 KB
llm-costs 558.1 KB 179.2 KB
session-recording 457.2 KB 179.2 KB
session-summary 462.5 KB 179.2 KB
survey 457.0 KB 179.2 KB
survey-global-stats 560.9 KB 179.2 KB
survey-list 563.7 KB 179.2 KB
survey-stats 560.8 KB 179.2 KB
trace-span 455.8 KB 179.2 KB
trace-span-list 562.9 KB 179.2 KB
workflow 455.8 KB 179.2 KB
workflow-list 562.4 KB 179.2 KB
query-results 743.7 KB 179.2 KB
render-ui 824.0 KB 179.2 KB
visual-review-snapshots 460.3 KB 179.2 KB
⚠️ Backend coverage — 98.0% of changed backend lines covered — 6 uncovered

🧪 Backend test coverage

Patch coverage — changed backend lines (products + core): ████████████████████ 98.0% (296 / 302)

File Patch Uncovered changed lines
products/replay_vision/backend/impact.py 92.6% 64, 75, 118, 120, 122, 124

🤖 Agents: add a test covering the lines above, or note why under "How did you test this code?". Machine-readable gap list: the patch-coverage artifact on this run (gh run download 29489672076 -n patch-coverage), or the coverage-data block at the end of this comment.

Per-product line coverage (touched products)
Product Coverage Lines
platform_features ██░░░░░░░░░░░░░░░░░░ 12.1% 7 / 58
batch_exports ████████░░░░░░░░░░░░ 39.7% 8,414 / 21,220
demo ███████████░░░░░░░░░ 56.2% 1,497 / 2,663
warehouse_sources_queue ████████████░░░░░░░░ 59.2% 148 / 250
tasks █████████████░░░░░░░ 67.4% 25,805 / 38,278
data_tools ██████████████░░░░░░ 70.0% 63 / 90
ai_gateway ███████████████░░░░░ 75.0% 9 / 12
signals ████████████████░░░░ 79.1% 19,035 / 24,073
data_modeling ████████████████░░░░ 80.0% 4,834 / 6,045
cdp ████████████████░░░░ 80.7% 3,118 / 3,864
wizard ████████████████░░░░ 82.5% 772 / 936
notebooks █████████████████░░░ 84.3% 6,343 / 7,520
agent_platform █████████████████░░░ 84.7% 3,273 / 3,862
cohorts █████████████████░░░ 86.1% 4,022 / 4,671
actions █████████████████░░░ 86.6% 717 / 828
product_tours █████████████████░░░ 87.5% 1,266 / 1,447
exports ██████████████████░░ 88.3% 6,891 / 7,800
visual_review ██████████████████░░ 88.5% 5,565 / 6,287
business_knowledge ██████████████████░░ 88.5% 4,400 / 4,969
conversations ██████████████████░░ 88.9% 16,129 / 18,133
mcp_analytics ██████████████████░░ 89.1% 2,502 / 2,807
dashboards ██████████████████░░ 89.2% 5,725 / 6,418
error_tracking ██████████████████░░ 89.6% 9,738 / 10,872
alerts ██████████████████░░ 89.9% 3,638 / 4,046
engineering_analytics ██████████████████░░ 90.1% 5,105 / 5,665
streamlit_apps ██████████████████░░ 90.4% 2,499 / 2,764
slack_app ██████████████████░░ 90.6% 9,511 / 10,503
links ██████████████████░░ 90.6% 183 / 202
marketing_analytics ██████████████████░░ 90.8% 11,514 / 12,684
product_analytics ██████████████████░░ 91.2% 5,603 / 6,147
mcp_store ██████████████████░░ 91.8% 3,668 / 3,995
managed_migrations ██████████████████░░ 91.9% 908 / 988
data_warehouse ██████████████████░░ 92.1% 18,133 / 19,683
workflows ██████████████████░░ 92.4% 5,148 / 5,574
notifications ███████████████████░ 92.7% 1,026 / 1,107
web_analytics ███████████████████░ 92.7% 13,624 / 14,691
ai_observability ███████████████████░ 92.8% 14,868 / 16,019
surveys ███████████████████░ 92.9% 5,687 / 6,120
posthog_ai ███████████████████░ 93.2% 1,322 / 1,418
tracing ███████████████████░ 93.3% 2,435 / 2,611
approvals ███████████████████░ 93.3% 3,395 / 3,640
reminders ███████████████████░ 93.4% 468 / 501
early_access_features ███████████████████░ 93.8% 848 / 904
legal_documents ███████████████████░ 94.1% 1,568 / 1,667
endpoints ███████████████████░ 94.1% 8,606 / 9,143
skills ███████████████████░ 94.4% 2,827 / 2,995
revenue_analytics ███████████████████░ 94.5% 3,598 / 3,809
messaging ███████████████████░ 94.5% 2,530 / 2,677
review_hog ███████████████████░ 94.6% 6,532 / 6,905
logs ███████████████████░ 95.3% 9,528 / 9,994
experiments ███████████████████░ 95.6% 24,179 / 25,296
replay_vision ███████████████████░ 95.8% 13,647 / 14,251
growth ███████████████████░ 95.8% 2,837 / 2,960
annotations ███████████████████░ 96.2% 732 / 761
feature_flags ███████████████████░ 96.3% 16,118 / 16,741
warehouse_sources ███████████████████░ 96.3% 238,227 / 247,395
user_interviews ███████████████████░ 96.4% 2,242 / 2,325
access_control ███████████████████░ 96.8% 849 / 877
data_catalog ███████████████████░ 97.1% 2,035 / 2,095
customer_analytics ███████████████████░ 97.2% 7,480 / 7,698
analytics_platform ████████████████████ 98.0% 2,102 / 2,145
metrics ████████████████████ 98.2% 2,404 / 2,448
live_debugger ████████████████████ 99.2% 613 / 618
field_notes ████████████████████ 99.4% 158 / 159

Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.

ClickHouse migration SQL — none

No ClickHouse migrations in the latest push.

@trunk-io

trunk-io Bot commented Jul 10, 2026

Copy link
Copy Markdown

Static BadgeStatic BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

@fasyy612
fasyy612 force-pushed the feat/impacted-users-on-scanner branch from 07e0434 to 7891c0b Compare July 14, 2026 15:41
@fasyy612
fasyy612 force-pushed the feat/impacted-users-on-scanner branch from 68cdb69 to 21ccae7 Compare July 15, 2026 14:38
@fasyy612
fasyy612 marked this pull request as ready for review July 15, 2026 15:06
@fasyy612
fasyy612 requested review from a team, TueHaulund, arnohillen and ksvat and removed request for a team July 15, 2026 15:06
@github-actions

github-actions Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

🦔 Hogbox preview · ❌ build failed

The preview didn't come up for commit 28f2459. See the build log for the failing step. It'll retry on the next push.

@assign-reviewers-posthog
assign-reviewers-posthog Bot requested a review from a team July 15, 2026 15:06
@assign-reviewers-posthog

Copy link
Copy Markdown

👀 Auto-assigned reviewers

These soft owners were skipped because they only have minor changes here. Nothing blocks merge, so self-assign if you'd like a look:

  • @PostHog/team-feature-flags (products/cohorts/product.yaml)
  • @PostHog/team-devex (owners.yaml)

Soft owners come from each directory's owners.yaml and each product's product.yaml (resolved nearest-file-wins). The locator after each owner is the file that decided it. Generated files and lockfiles are ignored when deciding ownership.

Comment thread products/replay_vision/backend/api/scanners.py Outdated
Comment thread products/replay_vision/backend/api/scanners.py Outdated
Comment thread products/replay_vision/backend/api/scanners.py Outdated
@veria-ai

veria-ai Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 2 · PR risk: 0/10

@fasyy612
fasyy612 merged commit 835ac50 into master Jul 17, 2026
271 of 272 checks passed
@fasyy612
fasyy612 deleted the feat/impacted-users-on-scanner branch July 17, 2026 09:25
@deployment-status-posthog

deployment-status-posthog Bot commented Jul 17, 2026

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-07-17 09:55 UTC Run
prod-us ✅ Deployed 2026-07-17 10:10 UTC Run
prod-eu ✅ Deployed 2026-07-17 10:12 UTC Run

vdekrijger added a commit that referenced this pull request Jul 17, 2026
The typecheck job's `schema:build:json && git diff --exit-code` step fails on kea-typegen drift: #70059 added scannerImpact/affectedCohort/savingCohortTag to the logic source but did not commit the regenerated type block. Regenerate it so the diff check is clean. Unrelated to this PR's alerts change, but the committed types are stale on master and block the required check.

Generated-By: PostHog Code
Task-Id: 6e9d367a-27a5-4f93-a0f0-e0b5947ececb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants