Skip to content

Conversation

@alexluong
Copy link
Collaborator

@alexluong alexluong commented Dec 11, 2025

implements #586

Destination Filter

Adds support for event filtering on destinations, enabling event routing based on event content—not just topic.

Features

  • Filter Field: New optional filter field on destinations allows filtering events based on JSON matching patterns
  • simplejsonmatch Library: Go port of hookdeck/simple-json-match with 1-to-1 mapping of test cases from the original JS implementation. Note: The $ref operator is not supported in this implementation.
  • Portal UI: JSON editor for managing filters in destination settings and create destination flow.

UI

CleanShot.2025-12-12.at.00.33.35.mp4

@vercel
Copy link

vercel bot commented Dec 11, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
outpost-docs Ready Ready Preview, Comment Dec 16, 2025 10:00am
outpost-website Ready Ready Preview, Comment Dec 16, 2025 10:00am

@alexbouchardd
Copy link
Contributor

Should this be opt-in the same way topics are? 🤔

@alexluong
Copy link
Collaborator Author

Should this be opt-in the same way topics are? 🤔

Not 100% sure what you mean, I think it is opt-in. As in destination.filter is optional, if nothing is provided then it's automatically matched based on topic only.

@alexbouchardd
Copy link
Contributor

alexbouchardd commented Dec 11, 2025

Some ENABLE_DESTINATION_FILTERS env variable is what I meant. Are we confident everyone would like that supported? 🤔

@alexluong
Copy link
Collaborator Author

ah I see. Hmmm not sure, I don't see a strong reason why someone would prefer not to have this feature. Maybe performance concern?

If someone is opinionated enough, maybe they already provide their own UI and can choose not to use this feature? Maybe we can support a flag to disable this in the portal only?

I just think disabling this feature as a whole may be a bit complicated because of the Destination schema (Destination API).

@alexbouchardd
Copy link
Contributor

Yeah agree really that's a portal config if anything since if you implement your own UI then you get to choose that way.

The problem if we don't give control over it is that a user may deploy the portal and later want to build their own UI or migrate to another system and be stuck with that backward compatibility. Given it does create some amount of "lock-in" with Outpost, it should be something you can decide to expose to your users or not.

PORTAL_DISABLE_FILTER_CONFIGURATION
PORTAL_DISABLE_WEBHOOK_DESTINATION_CUSTOM_HEADERS

Does that make sense?

@alexluong
Copy link
Collaborator Author

yup, all clear then. Please continue finalizing the UI changes the way you see fit. I'll take care of the portal flag for these 2 features. Will merge and release if you approve the PR

Copy link
Contributor

@alexbouchardd alexbouchardd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are missing validation that the filter have top level data or metadata and no other top level key since that would be invalid syntax. True both the for the API and UI validation

type: object
nullable: true
additionalProperties: true
description: |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The {} doesn't make sense to me, a user should pass null to unset

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey so quick note, i'll merge to a temp v0.10.0 branch now but we can discuss this further


good catch, I do agree that {} is not ideal. The reason tho is that the way we parse incoming request, in Golang null and undefined are the same -> nil, and for PATCH we treat nil as no update. We can still support null but it requires a bit more work which we can do, but I think this is an acceptable tradeoff for now. Do let me know tho. I agree, I don't love this DX and spent quite a bit of time thinking on this exact question

@alexluong alexluong changed the base branch from main to v0.10.0 December 16, 2025 08:06
alexluong added a commit that referenced this pull request Dec 16, 2025
Combines both features:
- Destination filter (PR #594)
- Custom headers (PR #590)

Resolved conflict in Destination.scss by keeping styles from both branches.
# Conflicts:
#	internal/portal/src/scenes/Destination/Destination.scss
@alexluong alexluong merged commit ef5f48e into v0.10.0 Dec 16, 2025
2 of 4 checks passed
@alexluong alexluong deleted the dest-filter branch December 16, 2025 09:59
alexluong added a commit that referenced this pull request Dec 17, 2025
* feat: port simple-json-match from js to golang

* feat: destination model filter field

* feat: destination filter api

* refactor: centralize destination event matching

* test: publishmq filter

* test: e2e filter test

* docs: openapi filter unset documentation

* test: e2e destination api with filter

* feat: portal filter ui

* chore: Filter UI display improvements and syntax guide

---------

Co-authored-by: Alexandre Bouchard <alexbouchardd@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants