Skip to content

fix(workflows): GitHub Sink + event-based workflows silently reject all entities - #28451

Closed
sonika-shah wants to merge 1 commit into
mainfrom
fix/workflow-filter-json-inversion
Closed

fix(workflows): GitHub Sink + event-based workflows silently reject all entities#28451
sonika-shah wants to merge 1 commit into
mainfrom
fix/workflow-filter-json-inversion

Conversation

@sonika-shah

@sonika-shah sonika-shah commented May 27, 2026

Copy link
Copy Markdown
Collaborator

Fix

Restore the ! in FilterEntityImpl.passesExcludedFilter() so non-JsonLogic filter input passes through correctly. Extract the JSON-filter evaluation into a small applyJsonFilter helper so the behavior can be unit-tested.

- passesJsonFilter =  Boolean.TRUE.equals(RuleEngine.apply(filterLogic, ...))
+ passesJsonFilter = !Boolean.TRUE.equals(RuleEngine.apply(filterLogic, ...))

Why the ! is required

RuleEngine.apply() returns false when filterLogic cannot be evaluated as JsonLogic — its catch branch is annotated "falls back to triggering workflow". Event-based workflow triggers pass Elasticsearch query DSL through this code path, so the consumer must treat any non-TRUE result as "pass through". The ! provides that semantic.

Impact

Without the !, every event-based workflow with an ES-style filter (Git Sink, EventBased approval, custom user workflows) is filtered out at the trigger gate:

  • Trigger workflow ends at endEvent in <100ms via the ${!passesFilter} branch
  • The CallActivity to the main workflow is skipped
  • ACT_HI_PROCINST shows only *Trigger rows; the main workflow never starts
  • No exception or error is surfaced

Regression source

The ! was originally added by #25272 ("Show Relevant Exclude Fields in EventBased Workflows") and inadvertently removed by #25894 ("Task redesign") — a 498-file refactor where this one-character change in FilterEntityImpl.java (+1/-1) was easy to miss in review.

Why this wasn't caught by existing tests

  • FilterEntityImplTest — 17 existing tests, all targeting passesFieldBasedFilter. The JSON-filter branch (passesJsonFilter / RuleEngine.apply) had no coverage.
  • Integration tests (*IT.java) — verify workflows deploy correctly; they don't assert that a matching event drives the workflow to its terminal side effect.
  • Playwright E2E — covers Workflow Builder UI flows; doesn't exercise event-driven dispatch end-to-end.

Tests added (same PR, existing reflection pattern)

Five new tests in FilterEntityImplTest exercise applyJsonFilter via reflection — same pattern used by the existing tests for passesFieldBasedFilter. Uses the real RuleEngine (no mocks).

  • testApplyJsonFilterPassesWhenFilterLogicIsNull
  • testApplyJsonFilterPassesWhenFilterLogicIsEmpty
  • testApplyJsonFilterPassesWhenFilterIsNotJsonLogic — regression test; fails if ! is removed again
  • testApplyJsonFilterExcludesWhenJsonLogicMatches — pins the "matching rule excludes" semantic
  • testApplyJsonFilterPassesWhenJsonLogicDoesNotMatch

Verification

Before After
Workflow ends in 50-100ms with no side effect Workflow runs in 2-6s and produces its side effect
Main workflow process never starts Main workflow runs through CallActivity → sinkTask → endEvent

Test run: mvn test -pl openmetadata-service -Dtest=FilterEntityImplTest22/22 green (17 existing + 5 new). Verified end-to-end against a Git Sink workflow with an ES filter on databaseService / table / glossaryTerm.

Copilot AI review requested due to automatic review settings May 27, 2026 06:20
@github-actions github-actions Bot added backend safe to test Add this label to run secure Github workflows on PRs labels May 27, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a regression in event-based workflow trigger filtering where Elasticsearch-style JSON filter strings were incorrectly treated as “exclude everything” due to a missing boolean inversion, breaking sinks/approval workflows that rely on ES-query JSON rather than JsonLogic.

Changes:

  • Restores the intended inverted interpretation of RuleEngine.apply() results for excluded JSON filters via applyJsonFilter(...).
  • Extracts JSON-filter evaluation into a package-private static helper to make the behavior directly unit-testable.
  • Adds focused unit tests to pin the ES-query (non-JsonLogic) fallback behavior and JsonLogic match/non-match cases.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
openmetadata-service/src/main/java/org/openmetadata/service/governance/workflows/elements/triggers/impl/FilterEntityImpl.java Restores/centralizes inverted JSON filter evaluation so non-JsonLogic inputs “pass through” excluded filtering.
openmetadata-service/src/test/java/org/openmetadata/service/governance/workflows/elements/triggers/impl/FilterEntityImplTest.java Adds regression/unit coverage for applyJsonFilter including the ES-query JSON fallback path and JsonLogic behaviors.

@sonika-shah
sonika-shah force-pushed the fix/workflow-filter-json-inversion branch from c91c9e5 to f27041b Compare May 27, 2026 06:22
FilterEntityImpl.passesExcludedFilter() rejected every entity for
event-based workflows whose trigger filter is Elasticsearch query JSON
(rather than JsonLogic). The Boolean inversion (!) on the JSON-filter
result was missing, so the Git Sink, EventBased approval workflows,
and any user-built workflow with an ES-style filter took the
${!passesFilter} branch in their trigger BPMN and never invoked the
main workflow's CallActivity.

  Before: passesJsonFilter =  Boolean.TRUE.equals(RuleEngine.apply(...))
  After:  passesJsonFilter = !Boolean.TRUE.equals(RuleEngine.apply(...))

RuleEngine.apply() is documented to return false ("falls back to
triggering workflow") when filterLogic is not parseable as JsonLogic.
Event-based workflow filters carry Elasticsearch query DSL through
this path, so the consumer must treat any non-TRUE result as a pass.

Extracts the JSON-filter evaluation into a small private static helper
(applyJsonFilter) and adds five focused tests using the existing
reflection pattern in FilterEntityImplTest. The regression test
testApplyJsonFilterPassesWhenFilterIsNotJsonLogic pins the
event-based-trigger path and fails deterministically if the inversion
is ever removed again.
Copilot AI review requested due to automatic review settings May 27, 2026 06:32
@sonika-shah
sonika-shah force-pushed the fix/workflow-filter-json-inversion branch from f27041b to bf6c603 Compare May 27, 2026 06:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@sonika-shah sonika-shah changed the title fix(workflows): restore inverted JSON filter logic in FilterEntityImpl fix(workflows): event-based workflow filters silently reject all entities May 27, 2026
@sonika-shah sonika-shah changed the title fix(workflows): event-based workflow filters silently reject all entities fix(workflows): GitHub Sink + event-based workflows silently reject all entities May 27, 2026
@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown
Contributor

🟡 Playwright Results — all passed (11 flaky)

✅ 4251 passed · ❌ 0 failed · 🟡 11 flaky · ⏭️ 88 skipped

Shard Passed Failed Flaky Skipped
🟡 Shard 1 298 0 1 4
✅ Shard 2 803 0 0 9
🟡 Shard 3 801 0 2 8
🟡 Shard 4 844 0 1 12
🟡 Shard 5 717 0 2 47
🟡 Shard 6 788 0 5 8
🟡 11 flaky test(s) (passed on retry)
  • Pages/AuditLogs.spec.ts › should apply both User and EntityType filters simultaneously (shard 1, 1 retry)
  • Features/KnowledgeCenter.spec.ts › Article mentions in description should working for Knowledge Center (shard 3, 1 retry)
  • Features/RTL.spec.ts › Verify Following widget functionality (shard 3, 1 retry)
  • Pages/CustomProperties.spec.ts › Duration (shard 4, 1 retry)
  • Pages/Entity.spec.ts › User as Owner with unsorted list (shard 5, 1 retry)
  • Pages/ExplorePageRightPanel.spec.ts › Should verify deleted user not visible in owner selection for topic (shard 5, 1 retry)
  • Pages/Glossary.spec.ts › Column dropdown drag-and-drop functionality for Glossary Terms table (shard 6, 1 retry)
  • Pages/InputOutputPorts.spec.ts › Port action dropdown visible with EditAll permission (shard 6, 1 retry)
  • Pages/Lineage/LineageFilters.spec.ts › Verify lineage schema filter selection (shard 6, 1 retry)
  • Pages/Lineage/LineageRightPanel.spec.ts › Verify custom properties tab IS visible for supported type: searchIndex (shard 6, 1 retry)
  • Pages/Lineage/PlatformLineage.spec.ts › Verify domain platform view (shard 6, 1 retry)

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

@sonika-shah

Copy link
Copy Markdown
Collaborator Author

Closing — this PR diagnosed the wrong layer. The actual root cause is in the Workflow Builder UI emitting Elasticsearch query DSL for event-based trigger filters, where the schema requires JsonLogic. FilterEntityImpl's behavior on main is correct given a properly-formatted JsonLogic filter.

Superseded by #28464, which fixes the UI to emit JsonLogic.

@gitar-bot

gitar-bot Bot commented May 27, 2026

Copy link
Copy Markdown
Code Review ✅ Approved

Restores the negated boolean check in FilterEntityImpl to correctly handle non-JsonLogic inputs in event-based workflows. Adds five unit tests to ensure proper coverage of the JSON filter evaluation logic.

Options

Display: compact → Showing less information.

Comment with these commands to change:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants