Skip to content

Revert "feat: migrate DeleteModal from MUI to untitledUI components" - #26519

Merged
chirag-madlani merged 1 commit into
mainfrom
revert-26461-migrated-delete-mui-modal
Mar 16, 2026
Merged

Revert "feat: migrate DeleteModal from MUI to untitledUI components"#26519
chirag-madlani merged 1 commit into
mainfrom
revert-26461-migrated-delete-mui-modal

Conversation

@Rohit0301

Copy link
Copy Markdown
Contributor

Reverts #26461

@Rohit0301
Rohit0301 requested a review from a team as a code owner March 16, 2026 11:25
@github-actions

Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@Rohit0301 Rohit0301 self-assigned this Mar 16, 2026
@@ -0,0 +1,157 @@
/*

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Quality: Revert re-introduces unused DeleteModal component (dead code)

The reverted components/Modals/DeleteModal/DeleteModal.tsx (plus its interface and test files) has zero external consumers — no file in the codebase imports it. It is a near-duplicate of components/common/DeleteModal/DeleteModalMUI.tsx (which is used by LearningResourcesPage), differing only in prop naming (entityName/onConfirm vs entityTitle/onDelete) and minor features (additionalStyle, custom message).

If the intent of this revert is to undo the untitledUI migration, consider also removing this dead component to avoid confusion. Otherwise, if it's meant for future use, at minimum add a consumer or document the intended usage.

Suggested fix:

If the component is not needed, remove the entire
`components/Modals/DeleteModal/` directory.
Otherwise, consolidate with DeleteModalMUI into
a single reusable component.

Was this helpful? React with 👍 / 👎 | Reply gitar fix to apply this suggestion

@gitar-bot

gitar-bot Bot commented Mar 16, 2026

Copy link
Copy Markdown

Analyzing CI failures

🔍 CI failure analysis for fe31d41: All 4 CI jobs failed at the PR label verification step because the pull request is missing the required 'safe to test' label. This is a gating mechanism blocking all builds from proceeding.

Overview

All 4 analyzed CI logs (ui-coverage-tests, build, java-checkstyle, py-checkstyle) fail identically at the same validation step. The root cause is consistent across all jobs: PR #26519 lacks the required 'safe to test' label that gates CI workflow execution. This is not a code or build issue, but a configuration/workflow requirement.

Failures

Missing Required 'safe to test' Label (confidence: high)

  • Type: configuration
  • Affected jobs: 67218063565 (ui-coverage-tests), 67218068382 (build), 67218063403 (java-checkstyle), 67218063498 (py-checkstyle)
  • Related to PR: yes
  • Root cause: The PR verification workflow enforces a gating label requirement. The pull request must have the 'safe to test' label before the CI pipeline proceeds. PR Revert "feat: migrate DeleteModal from MUI to untitledUI components" #26519 does not have this label applied, causing all four job workflows to fail at the "Verify PR labels" step with: "Error! This pull request does not contain any of the valid labels: ['safe to test']"
  • Suggested fix: Add the 'safe to test' label to this PR via the GitHub UI. This is typically done by a project maintainer. Since this is a revert PR (reverting a prior migration commit), a maintainer should label it to allow CI to proceed.

Summary

  • PR-related failures: 4 jobs blocked by missing label requirement
  • Infrastructure/flaky failures: None detected
  • Recommended action: Add the 'safe to test' label to PR Revert "feat: migrate DeleteModal from MUI to untitledUI components" #26519 in GitHub, then re-run the CI pipeline. All underlying build and test infrastructure is functional; this is purely a workflow gating issue.
Code Review ⚠️ Changes requested 0 resolved / 1 findings

Revert of DeleteModal migration reintroduces the MUI-based component as unused dead code. Remove the unused DeleteModal component before merging.

⚠️ Quality: Revert re-introduces unused DeleteModal component (dead code)

📄 openmetadata-ui/src/main/resources/ui/src/components/Modals/DeleteModal/DeleteModal.tsx:1 📄 openmetadata-ui/src/main/resources/ui/src/components/common/DeleteModal/DeleteModalMUI.tsx:1 📄 openmetadata-ui/src/main/resources/ui/src/components/Modals/DeleteModal/DeleteModal.interface.ts:1 📄 openmetadata-ui/src/main/resources/ui/src/components/Modals/DeleteModal/DeleteModal.test.tsx:1

The reverted components/Modals/DeleteModal/DeleteModal.tsx (plus its interface and test files) has zero external consumers — no file in the codebase imports it. It is a near-duplicate of components/common/DeleteModal/DeleteModalMUI.tsx (which is used by LearningResourcesPage), differing only in prop naming (entityName/onConfirm vs entityTitle/onDelete) and minor features (additionalStyle, custom message).

If the intent of this revert is to undo the untitledUI migration, consider also removing this dead component to avoid confusion. Otherwise, if it's meant for future use, at minimum add a consumer or document the intended usage.

Suggested fix
If the component is not needed, remove the entire
`components/Modals/DeleteModal/` directory.
Otherwise, consolidate with DeleteModalMUI into
a single reusable component.
🤖 Prompt for agents
Code Review: Revert of DeleteModal migration reintroduces the MUI-based component as unused dead code. Remove the unused DeleteModal component before merging.

1. ⚠️ Quality: Revert re-introduces unused DeleteModal component (dead code)
   Files: openmetadata-ui/src/main/resources/ui/src/components/Modals/DeleteModal/DeleteModal.tsx:1, openmetadata-ui/src/main/resources/ui/src/components/common/DeleteModal/DeleteModalMUI.tsx:1, openmetadata-ui/src/main/resources/ui/src/components/Modals/DeleteModal/DeleteModal.interface.ts:1, openmetadata-ui/src/main/resources/ui/src/components/Modals/DeleteModal/DeleteModal.test.tsx:1

   The reverted `components/Modals/DeleteModal/DeleteModal.tsx` (plus its interface and test files) has zero external consumers — no file in the codebase imports it. It is a near-duplicate of `components/common/DeleteModal/DeleteModalMUI.tsx` (which is used by `LearningResourcesPage`), differing only in prop naming (`entityName`/`onConfirm` vs `entityTitle`/`onDelete`) and minor features (`additionalStyle`, custom `message`).
   
   If the intent of this revert is to undo the untitledUI migration, consider also removing this dead component to avoid confusion. Otherwise, if it's meant for future use, at minimum add a consumer or document the intended usage.

   Suggested fix:
   If the component is not needed, remove the entire
   `components/Modals/DeleteModal/` directory.
   Otherwise, consolidate with DeleteModalMUI into
   a single reusable component.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@Rohit0301 Rohit0301 added the safe to test Add this label to run secure Github workflows on PRs label Mar 16, 2026
@chirag-madlani
chirag-madlani merged commit 74f655d into main Mar 16, 2026
23 of 27 checks passed
@chirag-madlani
chirag-madlani deleted the revert-26461-migrated-delete-mui-modal branch March 16, 2026 11:38
@github-actions

Copy link
Copy Markdown
Contributor

Jest test Coverage

UI tests summary

Lines Statements Branches Functions
Coverage: 66%
66.04% (57439/86967) 45.65% (30397/66579) 48.63% (9107/18726)

@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown
Contributor

🟡 Playwright Results — all passed (17 flaky)

✅ 3336 passed · ❌ 0 failed · 🟡 17 flaky · ⏭️ 183 skipped

Shard Passed Failed Flaky Skipped
🟡 Shard 1 454 0 1 2
🟡 Shard 2 304 0 1 1
🟡 Shard 3 656 0 5 33
🟡 Shard 4 722 0 5 47
🟡 Shard 5 590 0 1 67
🟡 Shard 6 610 0 4 33
🟡 17 flaky test(s) (passed on retry)
  • Features/CustomizeDetailPage.spec.ts › API Endpoint - customization should work (shard 1, 1 retry)
  • Features/ColumnBulkOperations.spec.ts › should expand STRUCT column to show nested fields (shard 2, 1 retry)
  • Features/DataQuality/TestCaseImportExportE2eFlow.spec.ts › EditAll User: Complete export-import-validate flow (shard 3, 1 retry)
  • Features/DataQuality/TestCaseIncidentPermissions.spec.ts › User with TEST_CASE.EDIT_ALL can see edit icon on incidents (shard 3, 1 retry)
  • Features/DataQuality/TestCaseResultPermissions.spec.ts › User with TEST_CASE.EDIT_ALL can see edit action on test case (shard 3, 1 retry)
  • Features/Permissions/GlossaryPermissions.spec.ts › Team-based permissions work correctly (shard 3, 1 retry)
  • Features/RestoreEntityInheritedFields.spec.ts › Validate restore with Inherited domain and data products assigned (shard 3, 1 retry)
  • Features/UserProfileOnlineStatus.spec.ts › Should update online status in real-time when user becomes active (shard 4, 1 retry)
  • Pages/Customproperties-part2.spec.ts › entityReferenceList shows item count, scrollable list, no expand toggle (shard 4, 1 retry)
  • Pages/Domains.spec.ts › Rename domain with owners and experts preserves assignments (shard 4, 1 retry)
  • Pages/Domains.spec.ts › Verify Domain entity API calls do not include invalid domains field in glossary term assets (shard 4, 1 retry)
  • Pages/Entity.spec.ts › Tag Add, Update and Remove (shard 4, 2 retries)
  • Pages/EntityDataConsumer.spec.ts › Glossary Term Add, Update and Remove (shard 5, 1 retry)
  • Pages/Glossary.spec.ts › Column dropdown drag-and-drop functionality for Glossary Terms table (shard 6, 1 retry)
  • Pages/InputOutputPorts.spec.ts › Remove single output port (shard 6, 1 retry)
  • Pages/Lineage.spec.ts › Lineage creation from SearchIndex entity (shard 6, 1 retry)
  • Pages/Users.spec.ts › Permissions for table details page for Data Consumer (shard 6, 1 retry)

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

JRDuncan pushed a commit to JRDuncan/OpenMetadata that referenced this pull request Mar 17, 2026
SumanMaharana added a commit that referenced this pull request Apr 8, 2026
* * fixed kafka sink linage

* fix kafka pipline linage topic to table matching

* chore(ui): add sidebar navigation (#26498)

* chore(ui): add sidebar navigation

* fix

* Migrated CustomControls and BulkImportVersionSummary (#26410)

* Migrated CustomControls and BulkImportVersionSummary

* migrated styled button component and playwright fixes

* fixed unit test

* fixed the bulkimportsummary modal and custom control tab item

* fixed the sonar issue

* addressed pr comments

* fixed playwright

* fixed playwright and lint issues

* addressed gitar comment

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* refactor: update toggle component and related fields to use UT_SWITCH type (#26425)

* refactor: update toggle component and related fields to use UT_SWITCH type

* addressed gitar comment

* fixed unit tests

* addressed PR comment

* Chore(deps): Bump undici in /openmetadata-ui/src/main/resources/ui (#26497)

Bumps [undici](https://github.com/nodejs/undici) from 6.23.0 to 6.24.1.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](nodejs/undici@v6.23.0...v6.24.1)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 6.24.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(ui): fix UserTeamSelectableList closing when clicking inside popover in modal (#26502)

* Remove unused `/api/v1/domains?limit=10000&fields=parent` call on every page load (#26485)

* Initial plan

* Remove unused domain list API call on every page load

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

* feat: migrate DeleteModal from MUI to untitledUI components (#26461)

* feat: migrate DeleteModal from MUI to untitledUI components

* fixed the import

* comment addressed

* fix: remove unnecessary span element from DeleteModal

* refactor: remove DeleteModal component and its associated files

* addressed gitar comment

* Added unit tests

* MINOR: Informix Schema update (#26483)

* MINOR: Informix Schema update

* fix

* Update generated TypeScript types

* Add supportdatabase

* Update generated TypeScript types

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>

* ui: make search placeholder in service main tabs dynamic (#26511)

* Revert "feat: migrate DeleteModal from MUI to untitledUI components (#26461)" (#26519)

This reverts commit 8327d22.

* Revert "refactor: update toggle component and related fields to use UT_SWITCH…" (#26520)

This reverts commit 59fe777.

* fix(ui): Test suite add-test-case modal and table UX (#26512)

* fix(ui): Test suite add-test-case modal and table UX

* fix footer padding

* Add eslint-plugin-playwright enforcement with CI check (#26494)

* Add eslint-plugin-playwright enforcement with CI check

Add eslint-plugin-playwright to catch common Playwright anti-patterns
automatically. 13 rules configured in two tiers:

- Error (blocks CI): no-networkidle, no-page-pause, no-focused-test
- Warn (tracks debt): missing-playwright-await, no-wait-for-timeout,
  no-force-option, no-element-handle, no-eval, no-skipped-test,
  prefer-web-first-assertions, no-useless-await, no-wait-for-selector,
  valid-expect

Changes:
- Install eslint-plugin-playwright, configure rules in eslint.config.mjs
- Add yarn lint:playwright script
- Repurpose ui-checkstyle.yml workflow to run Playwright lint on PRs
- Fix last networkidle usage in ClassificationConditionalRendering
- Remove stale eslint-disable comments for undefined rules
- Update PLAYWRIGHT_DEVELOPER_HANDBOOK with ESLint Enforcement section
- Update playwright, writing-playwright-tests, and playwright-validation
  skills to reference lint check

Current: 0 errors, 1657 warnings (CI passes cleanly)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix CodeQL cache poisoning alert: use pull_request instead of pull_request_target

Switch from pull_request_target to pull_request since this workflow
only runs yarn lint:playwright — no secrets or write permissions needed.
This eliminates the untrusted code execution risk flagged by CodeQL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix all 1657 Playwright ESLint warnings across 190 files

Resolve every playwright lint rule violation to enforce test quality:
- no-wait-for-selector (1290): replace page.waitForSelector() with locator.waitFor()
- no-wait-for-timeout (130): replace hardcoded waits with event-driven alternatives
- prefer-web-first-assertions (61): use toHaveText/toBeVisible/toHaveValue
- no-force-option (51): remove { force: true } bypassing actionability checks
- missing-playwright-await (40): add await to fire-and-forget assertions
- no-skipped-test (28): acknowledge skipped tests with eslint-disable reasons
- no-eval (11): replace page.$eval with locator APIs
- no-element-handle (10): replace page.$() with page.locator()
- no-useless-await (8): remove await from synchronous operations
- valid-expect (4): add matchers to bare expect() calls
- no-networkidle (2): replace networkidle with domcontentloaded

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove unnecessary ANTLR install and add --frozen-lockfile to CI

- Remove Install Antlr4 CLI step from playwright-lint job (not needed
  for ESLint, saves 10-30s and avoids external network dependency)
- Add --frozen-lockfile to yarn install for reproducible CI builds

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore ANTLR CLI install — required by yarn postinstall script

yarn install triggers build-check → js-antlr which needs antlr4 CLI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix trivially-true assertion in special character search test

Remove `(await tableRows.count()) >= 0` which is always true since
count() never returns negative. The assertion now properly validates
that either the table or empty state is visible after searching.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violations: add .first() to multi-element waitFor calls

The no-wait-for-selector conversion (waitForSelector → locator.waitFor)
introduced strict mode violations. Playwright's Locator API throws when
waitFor() matches multiple elements, unlike the old waitForSelector API.

Key patterns fixed:
- getByTestId('loader').waitFor() — multiple loaders on page
- getByTestId('select-owner-tabs').getByTestId('loader') — tab loaders
- locator('.ant-skeleton-active/content').waitFor() — multiple skeletons
- locator('table/thead th').waitFor() — multiple tables
- getByTestId('side-panel-classification') — multiple panels
- locator('.ant-select-dropdown:visible') — multiple dropdowns
- locator('.ant-popover').waitFor() — multiple popovers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violation in incidentManager assignee/owner-link

The getByTestId('assignee').getByTestId('owner-link') locator resolves
to 15 elements in the incident manager table (one per row). Adding
.first() matches the original waitForSelector semantics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix test failures from lint conversions: proper waits and retry logic

- GlossaryPermissions: add waitForAllLoadersToDisappear after page nav
- GlossaryP3Tests: restore waits for special char search + error state
- GlossaryStatusFilterLargeDataset: restore waitForTimeout for filter
  state settling (no reliable DOM element to wait for)
- AutoPilot: use waitForAllLoadersToDisappear instead of .first() to
  ensure ALL loaders are gone, increase banner timeout to 60s
- importUtils: use expect().toHaveCount(0) for scoped multi-element
  loader/skeleton waits instead of waitForSelector or .first()
- ColumnBulkOperations: increase response timeout, add retry polling
  for empty state verification
- DomainDataProductsWidgets: add poll-based wait for DOM update after
  asset removal instead of removed waitForTimeout
- customizeLandingPage: scope widget click to dialog to avoid matching
  stale elements on the page behind the modal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore force:true where genuinely needed with eslint-disable comments

The no-force-option ESLint rule removal broke tests where force was required
for Ant Select comboboxes (selected item overlay covers input), popover triggers
(partially obstructed by animation), data grid buttons (covered by overlay),
and drag-and-drop (row hover overlays). Each restoration includes an
eslint-disable-line comment explaining why force is necessary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Stabilize 4 flaky tests with retry wrappers and proper waits

- DataContractsSemanticRules: Wrap reload + status assertions in
  expect().toPass() to handle async backend contract validation
- DataContracts: Increase timeout on dynamically rendered row filter
  and column text assertions that contain UUID-suffixed names
- Customproperties-part2: Wrap user search-and-select loop in
  expect().toPass() retry to handle search dropdown rendering races
- UserProfileOnlineStatus: Add profile render wait, navigation
  completion guard, and increased badge visibility timeout

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Replace .first().waitFor loader pattern with waitForAllLoadersToDisappear

The pattern `page.getByTestId('loader').first().waitFor({ state: 'detached' })`
only waits for the FIRST loader element to detach. When multiple loaders exist
(nested components, tabs, popovers), this causes tests to proceed before the
page is fully loaded — the root cause of ~18 flaky tests.

Replaced ~500 occurrences across 103 files with the correct
`waitForAllLoadersToDisappear(page)` which uses `expect(loaders).toHaveCount(0)`
to wait for ALL loaders to disappear.

Scoped loader waits (e.g., within select-owner-tabs, test-case-container,
tags-container) are intentionally preserved since they correctly target a
specific container's single loader.

Also fixes UserProfileOnlineStatus.spec.ts:
- Added createOrFetchUser helper for idempotent user creation on retries
- Added afterAll cleanup for test users
- Replaced waitForURL with redirectToHomePage for navigation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>

* [Fix-25563] - Issue in Search Entity By Key  (#26482)

* Fix Issue Entity Not found , entity missing from search

* Fix Tests

* Fix Broken relationship issue for upstreamEntityRelationship

* Add Exponential Retry

* Remove Entity Not found from query

* Address Reciew Comments

* Address more review

* Fix Missing Columns Index

* Fix Join add stats

* Column stats and Merged Main

* Add separate custom bulk processor for column

* Fix reindex job falsely killed after 1 hour by orphan monitor

  The OrphanJobMonitor uses job.updatedAt as a liveness signal, but
  updatedAt was only set during state transitions (READY, RUNNING),
  never refreshed during processing. After 10 minutes the job appeared
  orphaned; at the 1-hour recovery window it was force-failed.

  Fix: touch updatedAt alongside the lock refresh (every 60s) so the
  staleness check stays satisfied while the coordinator is alive.

* Set Failure callback for columnBulkProcessor

* fix review comments

* address review comments from claude

* Review Mocked tests

* Address review comments. Both fixed:

                           1. retryOnConflict(3) restored on all 4 call sites — updateEntity and upsertDocument in both ElasticSearchEntityManager and OpenSearchEntityManager. This handles shard-level
                           version conflicts independently from the 429 retry logic in SearchRetryUtil.
                           2. drainPendingColumnFutures race fixed in both bulk sinks — replaced iterate + clear() with poll() loop, which atomically removes each element from the deque so no
                           concurrently-added futures can be lost

* Get proper error message from Elastic and OpenSearch

* Fix Breaking Test

* MINOR: Port py-tests improvements to py-tests-postgres workflow (#26517)

Apply the same structural improvements from the py-tests workflow:
- Add integration test sharding (shard-1/shard-2) for parallelism
- Replace `make run_python_tests` with nox integration-tests session
- Add explicit timeout-minutes (180) and descriptive job name
- Remove unnecessary fetch-depth: 0 from checkout
- Normalize indentation to 2-space with proper YAML style

* Use context to enhance recognition by default (#25856)

* Implement custom context enhancement for Presidio recognizers

Presidio's default context enhancement relies heavily on NLP and often fails
when analyzing individual values rather than full text. This implements a
custom context enhancement that:

- Boosts recognizer scores to MAX when context keywords match
- Applies a minimum score threshold (0.3) before enhancement
- Skips already-enhanced results to prevent double-boosting
- Introduces a decorator pattern for composing recognizer enhancements
- Adds eager_us_bank_recognizer with higher base scores for better results

The enhancement works by checking if any context words from the recognizer
match the provided context list, then boosting the confidence score to
maximum and setting the IS_SCORE_ENHANCED_BY_CONTEXT_KEY metadata flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Integrate context enhancement into recognizer factory

Updates PresidioRecognizerFactory to apply the new decorator pattern:

- All recognizers now use enhance_using_context decorator
- Confidence threshold filtering applied via filter_enhanced_results_below_threshold
- Decorators composed using decorate_recognizer for clean application
- Context passed to PatternRecognizer during creation

This ensures all enabled recognizers benefit from custom context enhancement
while maintaining backward compatibility with confidence thresholds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add migration to update PII tag recognizers with enhanced configs

Adds a v1.1.22 migration that updates existing PII tags with improved
recognizer configurations featuring context keywords and optimized patterns.

Changes:
- Add patchRecognizers method in CollectionDAO for updating tag recognizers
- Implement setRecognizersForSensitiveTags in MigrationUtil to load and apply
  recognizer configs from piiTagsWithRecognizers.json
- Update piiTagsWithRecognizers.json with context keywords for better
  classification accuracy
- Execute migration as post-DDL script for both MySQL and PostgreSQL

This migration ensures existing deployments benefit from the improved context
enhancement logic without manual reconfiguration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix potential NPE and type mismatch in migration utility

Addresses code review feedback:

1. Fix potential NullPointerException in setRecognizersForSensitiveTags
   - Use Boolean.TRUE.equals() instead of auto-unboxing for nullable Boolean
   - Prevents NPE when autoClassificationEnabled is absent from JSON
   - Follows existing pattern from v1120/MigrationUtil.java

2. Fix Boolean vs boolean type mismatch in updateTagRecognizers
   - Change isForceMigration parameter from boxed Boolean to primitive boolean
   - Matches caller signature and eliminates latent NPE risk
   - Maintains consistency across method signatures

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add context to NHSRecognizer

* Fix typing

* Fix broken unit tests

* Fix broken integration test

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Chore(UI): Remove tags customization option (#26514)

* Add regenerate-bot-tokens operation for JWT key rotation (#26477)

* Add regenerate-bot-tokens operation for JWT key rotation

Add a new `regenerate-bot-tokens` subcommand to OpenMetadataOperations
that regenerates JWT tokens for all bot users. This is needed when
rotating JWT signing keys or changing the cluster name (e.g., during
disaster recovery from POV to PROD).

The command:
- Iterates through all non-deleted bots with pagination
- Regenerates JWT tokens only for bots using JWT authentication
- Skips bots without associated users or non-JWT auth mechanisms
- Displays results in an ASCII table with status per bot

Closes open-metadata/openmetadata-collate#3157

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix review: add --expiry CLI parameter and null-guard paging

- Token expiry is now a CLI parameter (--expiry) defaulting to Unlimited,
  so operators can choose OneHour, One, Seven, Thirty, Sixty, Ninety, or
  Unlimited when regenerating
- Add null guard on getPaging() to prevent NPE on empty result sets

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Document regenerate-bot-tokens expiry values in help output

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Move bot token regeneration logic to server API

Move the token regeneration logic from the ops CLI into a new REST
endpoint PUT /v1/users/regenerateBotTokens (admin-only). The ops
command now calls the server API via HTTP, following the same pattern
as deploy-pipelines.

This keeps business logic in the server and the ops CLI as a thin
HTTP client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert server API approach, use direct DB access for token regeneration

After JWT key rotation, all existing bot tokens are invalid — including
the ingestion-bot token needed to authenticate against the server API.
This command must operate directly on the database to recover from
that state. Added a doc comment explaining the reasoning.

Also reverts the UserResource endpoint added in the previous commit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix compilation: use fully-qualified ResultList to match existing pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix botUser always null: fetch each bot individually via getByName

listAll/listAfter use setFieldsInBulk which only calls registered
fieldFetchers (tags, owners, etc.) — not the entity-specific setFields
override. BotRepository.setFields populates botUser via a relationship
lookup, but this is never called in bulk listing. Fetch each bot
individually with getByName which triggers setFields properly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add integration tests for regenerate-bot-tokens operation

Tests cover token regeneration producing new tokens, respecting
expiry parameters, and handling multiple bots independently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix flaky test: use different expiries to guarantee distinct tokens

Unlimited tokens generated in the same second produce identical JWTs
(same iat, null exp). Changed test to use different expiry values so
the tokens are guaranteed to differ.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* format

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* Fix #2730: Add tempLineageTables field to lineage details (#26487)

* Fix #2730: Lineage Add Temp Node Support

* missing sql file

* Update generated TypeScript types

* Update generated TypeScript types

* address guitar commens

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Feature: Fabric's lineage, usage and profiler support (#26373)

* Feature: Fabric's lineage, usage and profiler support

* Python Checkstyle

* Update generated TypeScript types

* fix: narrow Fabric fallback from except Exception to except ProgrammingError

* Addressed comments

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* chore(ui): Add Test Case list filters while creating a bundle suite (#26293)

* fix: Add Test Case list filters (Status, Test type, Table, Column) and E2E coverage

- Add filters to AddTestCaseList: Status (single), Test type (single), Table (multi), Column (multi)
- Use SearchDropdown for all four filters with hideCounts
- Client-side filtering for Table/Column; API refetch for Status/Test type
- Add AddTestCaseListFilters component and constants
- Add AddTestCaseList.utils (getTableFilterOptions, getColumnFilterOptions, filterTestCasesByTableAndColumn, getSelectedOptionsFromKeys)
- Unit tests: AddTestCaseList.component.test.tsx (filter tests), AddTestCaseList.utils.test.ts
- E2E: TestSuite.spec.ts - flatten steps, verify/apply/clear filters in create flow

Fixes https://github.com/open-metadata/openmetadata-collate/issues/3045

Made-with: Cursor

* fix lint

* address gitar-bot comments

* fix sonar issues

* address gitar-bot comments and sonar issues

* minor fix

* fix gitar-bot comments

* use backend filter for table type

* feat: added columnName parameter

* feat: adde columnname parameter

* feat: added colunmName argument

* convert column filter to server side filter

* address gitar-bot comments

* minor fix

* fix failing test

* address sonar issues

* Revert yarn.lock changes in openmetadata-ui-core-components

Made-with: Cursor

* address comment

* show select all filter in all test case lists

* address copilot comments

* add hideTableFilter and columnFilter props

* address comments

* hide table filter for for add pipeline under table

* fix failing test

* fix failing tests

---------

Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* added unit tests for kafkaconnect pipeline

* Add gitar fix

* MINOR - Column name Length patch (#26530)

* chore(bulkEndpoint): create bulk bundle suite body object

* patch(columnName): remove maxLength for column name

* patch(columnName): remove files from commit

* patch(columnName): remove unused code

* fix unit tests

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>

* update alert props (#26544)

* Fix: flaky domain rename data products count assertion (#26542)

* Fix: flaky domain rename data products count assertion

After a domain rename, the search index update for subdomain-level data
products can lag behind the DB update due to async reindexing by the
governance-bot. The `verifyDataProductsCount` helper was doing a one-shot
`textContent()` + `toBe()` assertion that failed when the search index
hadn't converged yet.

Changes:
- Update `verifyDataProductsCount` to accept optional `{ apiContext, domainFqn }`
  that uses `expect.poll()` to poll the data product search API until
  `hits.total.value` matches the expected count (following the same pattern
  used in glossary.ts, alert.ts, and other polling utilities)
- Switch from one-shot `toBe()` to auto-retrying `toHaveText()` for the UI check
- Pass polling options in the post-rename assertion in Domains.spec.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add HTTP status check in search API poll

Check response.ok() before parsing JSON in the expect.poll callback
to avoid silently treating 4xx/5xx errors as count=0.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove custom timeout from toHaveText assertion

The expect.poll already ensures the search index has converged before
the UI check, so the default Playwright timeout is sufficient.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Refactor InputOutputPortsTab to use Accordion components and update styles (#26450)

* Refactor InputOutputPortsTab to use Accordion components and update styles

* addressed PR comment

* fixed playwright test

* addressed gitar comment and fix sonar issues

* fixed overflow issue

* fixed ui issues

* minor fix

* addressed UI fixes

* py_format_check

* Added test for table format without topic prefix

* Removed commented code

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Rohit Jain <60229265+Rohit0301@users.noreply.github.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Akash Verma <138790903+akashverma0786@users.noreply.github.com>
Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Harsh Vador <58542468+harsh-vador@users.noreply.github.com>
Co-authored-by: Harshit Shah <harshit.shah@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: IceS2 <pablo.takara@getcollate.io>
Co-authored-by: Eugenio <eugenio.donaque@getcollate.io>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Sid <30566406+siddhant1@users.noreply.github.com>
Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
SaaiAravindhRaja pushed a commit to SaaiAravindhRaja/OpenMetadata that referenced this pull request Apr 12, 2026
* * fixed kafka sink linage

* fix kafka pipline linage topic to table matching

* chore(ui): add sidebar navigation (open-metadata#26498)

* chore(ui): add sidebar navigation

* fix

* Migrated CustomControls and BulkImportVersionSummary (open-metadata#26410)

* Migrated CustomControls and BulkImportVersionSummary

* migrated styled button component and playwright fixes

* fixed unit test

* fixed the bulkimportsummary modal and custom control tab item

* fixed the sonar issue

* addressed pr comments

* fixed playwright

* fixed playwright and lint issues

* addressed gitar comment

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* refactor: update toggle component and related fields to use UT_SWITCH type (open-metadata#26425)

* refactor: update toggle component and related fields to use UT_SWITCH type

* addressed gitar comment

* fixed unit tests

* addressed PR comment

* Chore(deps): Bump undici in /openmetadata-ui/src/main/resources/ui (open-metadata#26497)

Bumps [undici](https://github.com/nodejs/undici) from 6.23.0 to 6.24.1.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](nodejs/undici@v6.23.0...v6.24.1)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 6.24.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(ui): fix UserTeamSelectableList closing when clicking inside popover in modal (open-metadata#26502)

* Remove unused `/api/v1/domains?limit=10000&fields=parent` call on every page load (open-metadata#26485)

* Initial plan

* Remove unused domain list API call on every page load

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

* feat: migrate DeleteModal from MUI to untitledUI components (open-metadata#26461)

* feat: migrate DeleteModal from MUI to untitledUI components

* fixed the import

* comment addressed

* fix: remove unnecessary span element from DeleteModal

* refactor: remove DeleteModal component and its associated files

* addressed gitar comment

* Added unit tests

* MINOR: Informix Schema update (open-metadata#26483)

* MINOR: Informix Schema update

* fix

* Update generated TypeScript types

* Add supportdatabase

* Update generated TypeScript types

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>

* ui: make search placeholder in service main tabs dynamic (open-metadata#26511)

* Revert "feat: migrate DeleteModal from MUI to untitledUI components (open-metadata#26461)" (open-metadata#26519)

This reverts commit 8327d22.

* Revert "refactor: update toggle component and related fields to use UT_SWITCH…" (open-metadata#26520)

This reverts commit 59fe777.

* fix(ui): Test suite add-test-case modal and table UX (open-metadata#26512)

* fix(ui): Test suite add-test-case modal and table UX

* fix footer padding

* Add eslint-plugin-playwright enforcement with CI check (open-metadata#26494)

* Add eslint-plugin-playwright enforcement with CI check

Add eslint-plugin-playwright to catch common Playwright anti-patterns
automatically. 13 rules configured in two tiers:

- Error (blocks CI): no-networkidle, no-page-pause, no-focused-test
- Warn (tracks debt): missing-playwright-await, no-wait-for-timeout,
  no-force-option, no-element-handle, no-eval, no-skipped-test,
  prefer-web-first-assertions, no-useless-await, no-wait-for-selector,
  valid-expect

Changes:
- Install eslint-plugin-playwright, configure rules in eslint.config.mjs
- Add yarn lint:playwright script
- Repurpose ui-checkstyle.yml workflow to run Playwright lint on PRs
- Fix last networkidle usage in ClassificationConditionalRendering
- Remove stale eslint-disable comments for undefined rules
- Update PLAYWRIGHT_DEVELOPER_HANDBOOK with ESLint Enforcement section
- Update playwright, writing-playwright-tests, and playwright-validation
  skills to reference lint check

Current: 0 errors, 1657 warnings (CI passes cleanly)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix CodeQL cache poisoning alert: use pull_request instead of pull_request_target

Switch from pull_request_target to pull_request since this workflow
only runs yarn lint:playwright — no secrets or write permissions needed.
This eliminates the untrusted code execution risk flagged by CodeQL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix all 1657 Playwright ESLint warnings across 190 files

Resolve every playwright lint rule violation to enforce test quality:
- no-wait-for-selector (1290): replace page.waitForSelector() with locator.waitFor()
- no-wait-for-timeout (130): replace hardcoded waits with event-driven alternatives
- prefer-web-first-assertions (61): use toHaveText/toBeVisible/toHaveValue
- no-force-option (51): remove { force: true } bypassing actionability checks
- missing-playwright-await (40): add await to fire-and-forget assertions
- no-skipped-test (28): acknowledge skipped tests with eslint-disable reasons
- no-eval (11): replace page.$eval with locator APIs
- no-element-handle (10): replace page.$() with page.locator()
- no-useless-await (8): remove await from synchronous operations
- valid-expect (4): add matchers to bare expect() calls
- no-networkidle (2): replace networkidle with domcontentloaded

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove unnecessary ANTLR install and add --frozen-lockfile to CI

- Remove Install Antlr4 CLI step from playwright-lint job (not needed
  for ESLint, saves 10-30s and avoids external network dependency)
- Add --frozen-lockfile to yarn install for reproducible CI builds

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore ANTLR CLI install — required by yarn postinstall script

yarn install triggers build-check → js-antlr which needs antlr4 CLI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix trivially-true assertion in special character search test

Remove `(await tableRows.count()) >= 0` which is always true since
count() never returns negative. The assertion now properly validates
that either the table or empty state is visible after searching.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violations: add .first() to multi-element waitFor calls

The no-wait-for-selector conversion (waitForSelector → locator.waitFor)
introduced strict mode violations. Playwright's Locator API throws when
waitFor() matches multiple elements, unlike the old waitForSelector API.

Key patterns fixed:
- getByTestId('loader').waitFor() — multiple loaders on page
- getByTestId('select-owner-tabs').getByTestId('loader') — tab loaders
- locator('.ant-skeleton-active/content').waitFor() — multiple skeletons
- locator('table/thead th').waitFor() — multiple tables
- getByTestId('side-panel-classification') — multiple panels
- locator('.ant-select-dropdown:visible') — multiple dropdowns
- locator('.ant-popover').waitFor() — multiple popovers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violation in incidentManager assignee/owner-link

The getByTestId('assignee').getByTestId('owner-link') locator resolves
to 15 elements in the incident manager table (one per row). Adding
.first() matches the original waitForSelector semantics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix test failures from lint conversions: proper waits and retry logic

- GlossaryPermissions: add waitForAllLoadersToDisappear after page nav
- GlossaryP3Tests: restore waits for special char search + error state
- GlossaryStatusFilterLargeDataset: restore waitForTimeout for filter
  state settling (no reliable DOM element to wait for)
- AutoPilot: use waitForAllLoadersToDisappear instead of .first() to
  ensure ALL loaders are gone, increase banner timeout to 60s
- importUtils: use expect().toHaveCount(0) for scoped multi-element
  loader/skeleton waits instead of waitForSelector or .first()
- ColumnBulkOperations: increase response timeout, add retry polling
  for empty state verification
- DomainDataProductsWidgets: add poll-based wait for DOM update after
  asset removal instead of removed waitForTimeout
- customizeLandingPage: scope widget click to dialog to avoid matching
  stale elements on the page behind the modal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore force:true where genuinely needed with eslint-disable comments

The no-force-option ESLint rule removal broke tests where force was required
for Ant Select comboboxes (selected item overlay covers input), popover triggers
(partially obstructed by animation), data grid buttons (covered by overlay),
and drag-and-drop (row hover overlays). Each restoration includes an
eslint-disable-line comment explaining why force is necessary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Stabilize 4 flaky tests with retry wrappers and proper waits

- DataContractsSemanticRules: Wrap reload + status assertions in
  expect().toPass() to handle async backend contract validation
- DataContracts: Increase timeout on dynamically rendered row filter
  and column text assertions that contain UUID-suffixed names
- Customproperties-part2: Wrap user search-and-select loop in
  expect().toPass() retry to handle search dropdown rendering races
- UserProfileOnlineStatus: Add profile render wait, navigation
  completion guard, and increased badge visibility timeout

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Replace .first().waitFor loader pattern with waitForAllLoadersToDisappear

The pattern `page.getByTestId('loader').first().waitFor({ state: 'detached' })`
only waits for the FIRST loader element to detach. When multiple loaders exist
(nested components, tabs, popovers), this causes tests to proceed before the
page is fully loaded — the root cause of ~18 flaky tests.

Replaced ~500 occurrences across 103 files with the correct
`waitForAllLoadersToDisappear(page)` which uses `expect(loaders).toHaveCount(0)`
to wait for ALL loaders to disappear.

Scoped loader waits (e.g., within select-owner-tabs, test-case-container,
tags-container) are intentionally preserved since they correctly target a
specific container's single loader.

Also fixes UserProfileOnlineStatus.spec.ts:
- Added createOrFetchUser helper for idempotent user creation on retries
- Added afterAll cleanup for test users
- Replaced waitForURL with redirectToHomePage for navigation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>

* [Fix-25563] - Issue in Search Entity By Key  (open-metadata#26482)

* Fix Issue Entity Not found , entity missing from search

* Fix Tests

* Fix Broken relationship issue for upstreamEntityRelationship

* Add Exponential Retry

* Remove Entity Not found from query

* Address Reciew Comments

* Address more review

* Fix Missing Columns Index

* Fix Join add stats

* Column stats and Merged Main

* Add separate custom bulk processor for column

* Fix reindex job falsely killed after 1 hour by orphan monitor

  The OrphanJobMonitor uses job.updatedAt as a liveness signal, but
  updatedAt was only set during state transitions (READY, RUNNING),
  never refreshed during processing. After 10 minutes the job appeared
  orphaned; at the 1-hour recovery window it was force-failed.

  Fix: touch updatedAt alongside the lock refresh (every 60s) so the
  staleness check stays satisfied while the coordinator is alive.

* Set Failure callback for columnBulkProcessor

* fix review comments

* address review comments from claude

* Review Mocked tests

* Address review comments. Both fixed:

                           1. retryOnConflict(3) restored on all 4 call sites — updateEntity and upsertDocument in both ElasticSearchEntityManager and OpenSearchEntityManager. This handles shard-level
                           version conflicts independently from the 429 retry logic in SearchRetryUtil.
                           2. drainPendingColumnFutures race fixed in both bulk sinks — replaced iterate + clear() with poll() loop, which atomically removes each element from the deque so no
                           concurrently-added futures can be lost

* Get proper error message from Elastic and OpenSearch

* Fix Breaking Test

* MINOR: Port py-tests improvements to py-tests-postgres workflow (open-metadata#26517)

Apply the same structural improvements from the py-tests workflow:
- Add integration test sharding (shard-1/shard-2) for parallelism
- Replace `make run_python_tests` with nox integration-tests session
- Add explicit timeout-minutes (180) and descriptive job name
- Remove unnecessary fetch-depth: 0 from checkout
- Normalize indentation to 2-space with proper YAML style

* Use context to enhance recognition by default (open-metadata#25856)

* Implement custom context enhancement for Presidio recognizers

Presidio's default context enhancement relies heavily on NLP and often fails
when analyzing individual values rather than full text. This implements a
custom context enhancement that:

- Boosts recognizer scores to MAX when context keywords match
- Applies a minimum score threshold (0.3) before enhancement
- Skips already-enhanced results to prevent double-boosting
- Introduces a decorator pattern for composing recognizer enhancements
- Adds eager_us_bank_recognizer with higher base scores for better results

The enhancement works by checking if any context words from the recognizer
match the provided context list, then boosting the confidence score to
maximum and setting the IS_SCORE_ENHANCED_BY_CONTEXT_KEY metadata flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Integrate context enhancement into recognizer factory

Updates PresidioRecognizerFactory to apply the new decorator pattern:

- All recognizers now use enhance_using_context decorator
- Confidence threshold filtering applied via filter_enhanced_results_below_threshold
- Decorators composed using decorate_recognizer for clean application
- Context passed to PatternRecognizer during creation

This ensures all enabled recognizers benefit from custom context enhancement
while maintaining backward compatibility with confidence thresholds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add migration to update PII tag recognizers with enhanced configs

Adds a v1.1.22 migration that updates existing PII tags with improved
recognizer configurations featuring context keywords and optimized patterns.

Changes:
- Add patchRecognizers method in CollectionDAO for updating tag recognizers
- Implement setRecognizersForSensitiveTags in MigrationUtil to load and apply
  recognizer configs from piiTagsWithRecognizers.json
- Update piiTagsWithRecognizers.json with context keywords for better
  classification accuracy
- Execute migration as post-DDL script for both MySQL and PostgreSQL

This migration ensures existing deployments benefit from the improved context
enhancement logic without manual reconfiguration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix potential NPE and type mismatch in migration utility

Addresses code review feedback:

1. Fix potential NullPointerException in setRecognizersForSensitiveTags
   - Use Boolean.TRUE.equals() instead of auto-unboxing for nullable Boolean
   - Prevents NPE when autoClassificationEnabled is absent from JSON
   - Follows existing pattern from v1120/MigrationUtil.java

2. Fix Boolean vs boolean type mismatch in updateTagRecognizers
   - Change isForceMigration parameter from boxed Boolean to primitive boolean
   - Matches caller signature and eliminates latent NPE risk
   - Maintains consistency across method signatures

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add context to NHSRecognizer

* Fix typing

* Fix broken unit tests

* Fix broken integration test

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Chore(UI): Remove tags customization option (open-metadata#26514)

* Add regenerate-bot-tokens operation for JWT key rotation (open-metadata#26477)

* Add regenerate-bot-tokens operation for JWT key rotation

Add a new `regenerate-bot-tokens` subcommand to OpenMetadataOperations
that regenerates JWT tokens for all bot users. This is needed when
rotating JWT signing keys or changing the cluster name (e.g., during
disaster recovery from POV to PROD).

The command:
- Iterates through all non-deleted bots with pagination
- Regenerates JWT tokens only for bots using JWT authentication
- Skips bots without associated users or non-JWT auth mechanisms
- Displays results in an ASCII table with status per bot

Closes open-metadata/openmetadata-collate#3157

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix review: add --expiry CLI parameter and null-guard paging

- Token expiry is now a CLI parameter (--expiry) defaulting to Unlimited,
  so operators can choose OneHour, One, Seven, Thirty, Sixty, Ninety, or
  Unlimited when regenerating
- Add null guard on getPaging() to prevent NPE on empty result sets

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Document regenerate-bot-tokens expiry values in help output

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Move bot token regeneration logic to server API

Move the token regeneration logic from the ops CLI into a new REST
endpoint PUT /v1/users/regenerateBotTokens (admin-only). The ops
command now calls the server API via HTTP, following the same pattern
as deploy-pipelines.

This keeps business logic in the server and the ops CLI as a thin
HTTP client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert server API approach, use direct DB access for token regeneration

After JWT key rotation, all existing bot tokens are invalid — including
the ingestion-bot token needed to authenticate against the server API.
This command must operate directly on the database to recover from
that state. Added a doc comment explaining the reasoning.

Also reverts the UserResource endpoint added in the previous commit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix compilation: use fully-qualified ResultList to match existing pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix botUser always null: fetch each bot individually via getByName

listAll/listAfter use setFieldsInBulk which only calls registered
fieldFetchers (tags, owners, etc.) — not the entity-specific setFields
override. BotRepository.setFields populates botUser via a relationship
lookup, but this is never called in bulk listing. Fetch each bot
individually with getByName which triggers setFields properly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add integration tests for regenerate-bot-tokens operation

Tests cover token regeneration producing new tokens, respecting
expiry parameters, and handling multiple bots independently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix flaky test: use different expiries to guarantee distinct tokens

Unlimited tokens generated in the same second produce identical JWTs
(same iat, null exp). Changed test to use different expiry values so
the tokens are guaranteed to differ.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* format

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* Fix open-metadata#2730: Add tempLineageTables field to lineage details (open-metadata#26487)

* Fix open-metadata#2730: Lineage Add Temp Node Support

* missing sql file

* Update generated TypeScript types

* Update generated TypeScript types

* address guitar commens

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Feature: Fabric's lineage, usage and profiler support (open-metadata#26373)

* Feature: Fabric's lineage, usage and profiler support

* Python Checkstyle

* Update generated TypeScript types

* fix: narrow Fabric fallback from except Exception to except ProgrammingError

* Addressed comments

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* chore(ui): Add Test Case list filters while creating a bundle suite (open-metadata#26293)

* fix: Add Test Case list filters (Status, Test type, Table, Column) and E2E coverage

- Add filters to AddTestCaseList: Status (single), Test type (single), Table (multi), Column (multi)
- Use SearchDropdown for all four filters with hideCounts
- Client-side filtering for Table/Column; API refetch for Status/Test type
- Add AddTestCaseListFilters component and constants
- Add AddTestCaseList.utils (getTableFilterOptions, getColumnFilterOptions, filterTestCasesByTableAndColumn, getSelectedOptionsFromKeys)
- Unit tests: AddTestCaseList.component.test.tsx (filter tests), AddTestCaseList.utils.test.ts
- E2E: TestSuite.spec.ts - flatten steps, verify/apply/clear filters in create flow

Fixes https://github.com/open-metadata/openmetadata-collate/issues/3045

Made-with: Cursor

* fix lint

* address gitar-bot comments

* fix sonar issues

* address gitar-bot comments and sonar issues

* minor fix

* fix gitar-bot comments

* use backend filter for table type

* feat: added columnName parameter

* feat: adde columnname parameter

* feat: added colunmName argument

* convert column filter to server side filter

* address gitar-bot comments

* minor fix

* fix failing test

* address sonar issues

* Revert yarn.lock changes in openmetadata-ui-core-components

Made-with: Cursor

* address comment

* show select all filter in all test case lists

* address copilot comments

* add hideTableFilter and columnFilter props

* address comments

* hide table filter for for add pipeline under table

* fix failing test

* fix failing tests

---------

Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* added unit tests for kafkaconnect pipeline

* Add gitar fix

* MINOR - Column name Length patch (open-metadata#26530)

* chore(bulkEndpoint): create bulk bundle suite body object

* patch(columnName): remove maxLength for column name

* patch(columnName): remove files from commit

* patch(columnName): remove unused code

* fix unit tests

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>

* update alert props (open-metadata#26544)

* Fix: flaky domain rename data products count assertion (open-metadata#26542)

* Fix: flaky domain rename data products count assertion

After a domain rename, the search index update for subdomain-level data
products can lag behind the DB update due to async reindexing by the
governance-bot. The `verifyDataProductsCount` helper was doing a one-shot
`textContent()` + `toBe()` assertion that failed when the search index
hadn't converged yet.

Changes:
- Update `verifyDataProductsCount` to accept optional `{ apiContext, domainFqn }`
  that uses `expect.poll()` to poll the data product search API until
  `hits.total.value` matches the expected count (following the same pattern
  used in glossary.ts, alert.ts, and other polling utilities)
- Switch from one-shot `toBe()` to auto-retrying `toHaveText()` for the UI check
- Pass polling options in the post-rename assertion in Domains.spec.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add HTTP status check in search API poll

Check response.ok() before parsing JSON in the expect.poll callback
to avoid silently treating 4xx/5xx errors as count=0.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove custom timeout from toHaveText assertion

The expect.poll already ensures the search index has converged before
the UI check, so the default Playwright timeout is sufficient.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Refactor InputOutputPortsTab to use Accordion components and update styles (open-metadata#26450)

* Refactor InputOutputPortsTab to use Accordion components and update styles

* addressed PR comment

* fixed playwright test

* addressed gitar comment and fix sonar issues

* fixed overflow issue

* fixed ui issues

* minor fix

* addressed UI fixes

* py_format_check

* Added test for table format without topic prefix

* Removed commented code

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Rohit Jain <60229265+Rohit0301@users.noreply.github.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Akash Verma <138790903+akashverma0786@users.noreply.github.com>
Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Harsh Vador <58542468+harsh-vador@users.noreply.github.com>
Co-authored-by: Harshit Shah <harshit.shah@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: IceS2 <pablo.takara@getcollate.io>
Co-authored-by: Eugenio <eugenio.donaque@getcollate.io>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Sid <30566406+siddhant1@users.noreply.github.com>
Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
SaaiAravindhRaja pushed a commit to SaaiAravindhRaja/OpenMetadata that referenced this pull request Apr 12, 2026
* * fixed kafka sink linage

* fix kafka pipline linage topic to table matching

* chore(ui): add sidebar navigation (open-metadata#26498)

* chore(ui): add sidebar navigation

* fix

* Migrated CustomControls and BulkImportVersionSummary (open-metadata#26410)

* Migrated CustomControls and BulkImportVersionSummary

* migrated styled button component and playwright fixes

* fixed unit test

* fixed the bulkimportsummary modal and custom control tab item

* fixed the sonar issue

* addressed pr comments

* fixed playwright

* fixed playwright and lint issues

* addressed gitar comment

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* refactor: update toggle component and related fields to use UT_SWITCH type (open-metadata#26425)

* refactor: update toggle component and related fields to use UT_SWITCH type

* addressed gitar comment

* fixed unit tests

* addressed PR comment

* Chore(deps): Bump undici in /openmetadata-ui/src/main/resources/ui (open-metadata#26497)

Bumps [undici](https://github.com/nodejs/undici) from 6.23.0 to 6.24.1.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](nodejs/undici@v6.23.0...v6.24.1)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 6.24.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(ui): fix UserTeamSelectableList closing when clicking inside popover in modal (open-metadata#26502)

* Remove unused `/api/v1/domains?limit=10000&fields=parent` call on every page load (open-metadata#26485)

* Initial plan

* Remove unused domain list API call on every page load

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

* feat: migrate DeleteModal from MUI to untitledUI components (open-metadata#26461)

* feat: migrate DeleteModal from MUI to untitledUI components

* fixed the import

* comment addressed

* fix: remove unnecessary span element from DeleteModal

* refactor: remove DeleteModal component and its associated files

* addressed gitar comment

* Added unit tests

* MINOR: Informix Schema update (open-metadata#26483)

* MINOR: Informix Schema update

* fix

* Update generated TypeScript types

* Add supportdatabase

* Update generated TypeScript types

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>

* ui: make search placeholder in service main tabs dynamic (open-metadata#26511)

* Revert "feat: migrate DeleteModal from MUI to untitledUI components (open-metadata#26461)" (open-metadata#26519)

This reverts commit 8327d22.

* Revert "refactor: update toggle component and related fields to use UT_SWITCH…" (open-metadata#26520)

This reverts commit 59fe777.

* fix(ui): Test suite add-test-case modal and table UX (open-metadata#26512)

* fix(ui): Test suite add-test-case modal and table UX

* fix footer padding

* Add eslint-plugin-playwright enforcement with CI check (open-metadata#26494)

* Add eslint-plugin-playwright enforcement with CI check

Add eslint-plugin-playwright to catch common Playwright anti-patterns
automatically. 13 rules configured in two tiers:

- Error (blocks CI): no-networkidle, no-page-pause, no-focused-test
- Warn (tracks debt): missing-playwright-await, no-wait-for-timeout,
  no-force-option, no-element-handle, no-eval, no-skipped-test,
  prefer-web-first-assertions, no-useless-await, no-wait-for-selector,
  valid-expect

Changes:
- Install eslint-plugin-playwright, configure rules in eslint.config.mjs
- Add yarn lint:playwright script
- Repurpose ui-checkstyle.yml workflow to run Playwright lint on PRs
- Fix last networkidle usage in ClassificationConditionalRendering
- Remove stale eslint-disable comments for undefined rules
- Update PLAYWRIGHT_DEVELOPER_HANDBOOK with ESLint Enforcement section
- Update playwright, writing-playwright-tests, and playwright-validation
  skills to reference lint check

Current: 0 errors, 1657 warnings (CI passes cleanly)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix CodeQL cache poisoning alert: use pull_request instead of pull_request_target

Switch from pull_request_target to pull_request since this workflow
only runs yarn lint:playwright — no secrets or write permissions needed.
This eliminates the untrusted code execution risk flagged by CodeQL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix all 1657 Playwright ESLint warnings across 190 files

Resolve every playwright lint rule violation to enforce test quality:
- no-wait-for-selector (1290): replace page.waitForSelector() with locator.waitFor()
- no-wait-for-timeout (130): replace hardcoded waits with event-driven alternatives
- prefer-web-first-assertions (61): use toHaveText/toBeVisible/toHaveValue
- no-force-option (51): remove { force: true } bypassing actionability checks
- missing-playwright-await (40): add await to fire-and-forget assertions
- no-skipped-test (28): acknowledge skipped tests with eslint-disable reasons
- no-eval (11): replace page.$eval with locator APIs
- no-element-handle (10): replace page.$() with page.locator()
- no-useless-await (8): remove await from synchronous operations
- valid-expect (4): add matchers to bare expect() calls
- no-networkidle (2): replace networkidle with domcontentloaded

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove unnecessary ANTLR install and add --frozen-lockfile to CI

- Remove Install Antlr4 CLI step from playwright-lint job (not needed
  for ESLint, saves 10-30s and avoids external network dependency)
- Add --frozen-lockfile to yarn install for reproducible CI builds

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore ANTLR CLI install — required by yarn postinstall script

yarn install triggers build-check → js-antlr which needs antlr4 CLI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix trivially-true assertion in special character search test

Remove `(await tableRows.count()) >= 0` which is always true since
count() never returns negative. The assertion now properly validates
that either the table or empty state is visible after searching.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violations: add .first() to multi-element waitFor calls

The no-wait-for-selector conversion (waitForSelector → locator.waitFor)
introduced strict mode violations. Playwright's Locator API throws when
waitFor() matches multiple elements, unlike the old waitForSelector API.

Key patterns fixed:
- getByTestId('loader').waitFor() — multiple loaders on page
- getByTestId('select-owner-tabs').getByTestId('loader') — tab loaders
- locator('.ant-skeleton-active/content').waitFor() — multiple skeletons
- locator('table/thead th').waitFor() — multiple tables
- getByTestId('side-panel-classification') — multiple panels
- locator('.ant-select-dropdown:visible') — multiple dropdowns
- locator('.ant-popover').waitFor() — multiple popovers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violation in incidentManager assignee/owner-link

The getByTestId('assignee').getByTestId('owner-link') locator resolves
to 15 elements in the incident manager table (one per row). Adding
.first() matches the original waitForSelector semantics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix test failures from lint conversions: proper waits and retry logic

- GlossaryPermissions: add waitForAllLoadersToDisappear after page nav
- GlossaryP3Tests: restore waits for special char search + error state
- GlossaryStatusFilterLargeDataset: restore waitForTimeout for filter
  state settling (no reliable DOM element to wait for)
- AutoPilot: use waitForAllLoadersToDisappear instead of .first() to
  ensure ALL loaders are gone, increase banner timeout to 60s
- importUtils: use expect().toHaveCount(0) for scoped multi-element
  loader/skeleton waits instead of waitForSelector or .first()
- ColumnBulkOperations: increase response timeout, add retry polling
  for empty state verification
- DomainDataProductsWidgets: add poll-based wait for DOM update after
  asset removal instead of removed waitForTimeout
- customizeLandingPage: scope widget click to dialog to avoid matching
  stale elements on the page behind the modal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore force:true where genuinely needed with eslint-disable comments

The no-force-option ESLint rule removal broke tests where force was required
for Ant Select comboboxes (selected item overlay covers input), popover triggers
(partially obstructed by animation), data grid buttons (covered by overlay),
and drag-and-drop (row hover overlays). Each restoration includes an
eslint-disable-line comment explaining why force is necessary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Stabilize 4 flaky tests with retry wrappers and proper waits

- DataContractsSemanticRules: Wrap reload + status assertions in
  expect().toPass() to handle async backend contract validation
- DataContracts: Increase timeout on dynamically rendered row filter
  and column text assertions that contain UUID-suffixed names
- Customproperties-part2: Wrap user search-and-select loop in
  expect().toPass() retry to handle search dropdown rendering races
- UserProfileOnlineStatus: Add profile render wait, navigation
  completion guard, and increased badge visibility timeout

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Replace .first().waitFor loader pattern with waitForAllLoadersToDisappear

The pattern `page.getByTestId('loader').first().waitFor({ state: 'detached' })`
only waits for the FIRST loader element to detach. When multiple loaders exist
(nested components, tabs, popovers), this causes tests to proceed before the
page is fully loaded — the root cause of ~18 flaky tests.

Replaced ~500 occurrences across 103 files with the correct
`waitForAllLoadersToDisappear(page)` which uses `expect(loaders).toHaveCount(0)`
to wait for ALL loaders to disappear.

Scoped loader waits (e.g., within select-owner-tabs, test-case-container,
tags-container) are intentionally preserved since they correctly target a
specific container's single loader.

Also fixes UserProfileOnlineStatus.spec.ts:
- Added createOrFetchUser helper for idempotent user creation on retries
- Added afterAll cleanup for test users
- Replaced waitForURL with redirectToHomePage for navigation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>

* [Fix-25563] - Issue in Search Entity By Key  (open-metadata#26482)

* Fix Issue Entity Not found , entity missing from search

* Fix Tests

* Fix Broken relationship issue for upstreamEntityRelationship

* Add Exponential Retry

* Remove Entity Not found from query

* Address Reciew Comments

* Address more review

* Fix Missing Columns Index

* Fix Join add stats

* Column stats and Merged Main

* Add separate custom bulk processor for column

* Fix reindex job falsely killed after 1 hour by orphan monitor

  The OrphanJobMonitor uses job.updatedAt as a liveness signal, but
  updatedAt was only set during state transitions (READY, RUNNING),
  never refreshed during processing. After 10 minutes the job appeared
  orphaned; at the 1-hour recovery window it was force-failed.

  Fix: touch updatedAt alongside the lock refresh (every 60s) so the
  staleness check stays satisfied while the coordinator is alive.

* Set Failure callback for columnBulkProcessor

* fix review comments

* address review comments from claude

* Review Mocked tests

* Address review comments. Both fixed:

                           1. retryOnConflict(3) restored on all 4 call sites — updateEntity and upsertDocument in both ElasticSearchEntityManager and OpenSearchEntityManager. This handles shard-level
                           version conflicts independently from the 429 retry logic in SearchRetryUtil.
                           2. drainPendingColumnFutures race fixed in both bulk sinks — replaced iterate + clear() with poll() loop, which atomically removes each element from the deque so no
                           concurrently-added futures can be lost

* Get proper error message from Elastic and OpenSearch

* Fix Breaking Test

* MINOR: Port py-tests improvements to py-tests-postgres workflow (open-metadata#26517)

Apply the same structural improvements from the py-tests workflow:
- Add integration test sharding (shard-1/shard-2) for parallelism
- Replace `make run_python_tests` with nox integration-tests session
- Add explicit timeout-minutes (180) and descriptive job name
- Remove unnecessary fetch-depth: 0 from checkout
- Normalize indentation to 2-space with proper YAML style

* Use context to enhance recognition by default (open-metadata#25856)

* Implement custom context enhancement for Presidio recognizers

Presidio's default context enhancement relies heavily on NLP and often fails
when analyzing individual values rather than full text. This implements a
custom context enhancement that:

- Boosts recognizer scores to MAX when context keywords match
- Applies a minimum score threshold (0.3) before enhancement
- Skips already-enhanced results to prevent double-boosting
- Introduces a decorator pattern for composing recognizer enhancements
- Adds eager_us_bank_recognizer with higher base scores for better results

The enhancement works by checking if any context words from the recognizer
match the provided context list, then boosting the confidence score to
maximum and setting the IS_SCORE_ENHANCED_BY_CONTEXT_KEY metadata flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Integrate context enhancement into recognizer factory

Updates PresidioRecognizerFactory to apply the new decorator pattern:

- All recognizers now use enhance_using_context decorator
- Confidence threshold filtering applied via filter_enhanced_results_below_threshold
- Decorators composed using decorate_recognizer for clean application
- Context passed to PatternRecognizer during creation

This ensures all enabled recognizers benefit from custom context enhancement
while maintaining backward compatibility with confidence thresholds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add migration to update PII tag recognizers with enhanced configs

Adds a v1.1.22 migration that updates existing PII tags with improved
recognizer configurations featuring context keywords and optimized patterns.

Changes:
- Add patchRecognizers method in CollectionDAO for updating tag recognizers
- Implement setRecognizersForSensitiveTags in MigrationUtil to load and apply
  recognizer configs from piiTagsWithRecognizers.json
- Update piiTagsWithRecognizers.json with context keywords for better
  classification accuracy
- Execute migration as post-DDL script for both MySQL and PostgreSQL

This migration ensures existing deployments benefit from the improved context
enhancement logic without manual reconfiguration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix potential NPE and type mismatch in migration utility

Addresses code review feedback:

1. Fix potential NullPointerException in setRecognizersForSensitiveTags
   - Use Boolean.TRUE.equals() instead of auto-unboxing for nullable Boolean
   - Prevents NPE when autoClassificationEnabled is absent from JSON
   - Follows existing pattern from v1120/MigrationUtil.java

2. Fix Boolean vs boolean type mismatch in updateTagRecognizers
   - Change isForceMigration parameter from boxed Boolean to primitive boolean
   - Matches caller signature and eliminates latent NPE risk
   - Maintains consistency across method signatures

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add context to NHSRecognizer

* Fix typing

* Fix broken unit tests

* Fix broken integration test

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Chore(UI): Remove tags customization option (open-metadata#26514)

* Add regenerate-bot-tokens operation for JWT key rotation (open-metadata#26477)

* Add regenerate-bot-tokens operation for JWT key rotation

Add a new `regenerate-bot-tokens` subcommand to OpenMetadataOperations
that regenerates JWT tokens for all bot users. This is needed when
rotating JWT signing keys or changing the cluster name (e.g., during
disaster recovery from POV to PROD).

The command:
- Iterates through all non-deleted bots with pagination
- Regenerates JWT tokens only for bots using JWT authentication
- Skips bots without associated users or non-JWT auth mechanisms
- Displays results in an ASCII table with status per bot

Closes open-metadata/openmetadata-collate#3157

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix review: add --expiry CLI parameter and null-guard paging

- Token expiry is now a CLI parameter (--expiry) defaulting to Unlimited,
  so operators can choose OneHour, One, Seven, Thirty, Sixty, Ninety, or
  Unlimited when regenerating
- Add null guard on getPaging() to prevent NPE on empty result sets

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Document regenerate-bot-tokens expiry values in help output

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Move bot token regeneration logic to server API

Move the token regeneration logic from the ops CLI into a new REST
endpoint PUT /v1/users/regenerateBotTokens (admin-only). The ops
command now calls the server API via HTTP, following the same pattern
as deploy-pipelines.

This keeps business logic in the server and the ops CLI as a thin
HTTP client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert server API approach, use direct DB access for token regeneration

After JWT key rotation, all existing bot tokens are invalid — including
the ingestion-bot token needed to authenticate against the server API.
This command must operate directly on the database to recover from
that state. Added a doc comment explaining the reasoning.

Also reverts the UserResource endpoint added in the previous commit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix compilation: use fully-qualified ResultList to match existing pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix botUser always null: fetch each bot individually via getByName

listAll/listAfter use setFieldsInBulk which only calls registered
fieldFetchers (tags, owners, etc.) — not the entity-specific setFields
override. BotRepository.setFields populates botUser via a relationship
lookup, but this is never called in bulk listing. Fetch each bot
individually with getByName which triggers setFields properly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add integration tests for regenerate-bot-tokens operation

Tests cover token regeneration producing new tokens, respecting
expiry parameters, and handling multiple bots independently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix flaky test: use different expiries to guarantee distinct tokens

Unlimited tokens generated in the same second produce identical JWTs
(same iat, null exp). Changed test to use different expiry values so
the tokens are guaranteed to differ.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* format

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* Fix open-metadata#2730: Add tempLineageTables field to lineage details (open-metadata#26487)

* Fix open-metadata#2730: Lineage Add Temp Node Support

* missing sql file

* Update generated TypeScript types

* Update generated TypeScript types

* address guitar commens

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Feature: Fabric's lineage, usage and profiler support (open-metadata#26373)

* Feature: Fabric's lineage, usage and profiler support

* Python Checkstyle

* Update generated TypeScript types

* fix: narrow Fabric fallback from except Exception to except ProgrammingError

* Addressed comments

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* chore(ui): Add Test Case list filters while creating a bundle suite (open-metadata#26293)

* fix: Add Test Case list filters (Status, Test type, Table, Column) and E2E coverage

- Add filters to AddTestCaseList: Status (single), Test type (single), Table (multi), Column (multi)
- Use SearchDropdown for all four filters with hideCounts
- Client-side filtering for Table/Column; API refetch for Status/Test type
- Add AddTestCaseListFilters component and constants
- Add AddTestCaseList.utils (getTableFilterOptions, getColumnFilterOptions, filterTestCasesByTableAndColumn, getSelectedOptionsFromKeys)
- Unit tests: AddTestCaseList.component.test.tsx (filter tests), AddTestCaseList.utils.test.ts
- E2E: TestSuite.spec.ts - flatten steps, verify/apply/clear filters in create flow

Fixes https://github.com/open-metadata/openmetadata-collate/issues/3045

Made-with: Cursor

* fix lint

* address gitar-bot comments

* fix sonar issues

* address gitar-bot comments and sonar issues

* minor fix

* fix gitar-bot comments

* use backend filter for table type

* feat: added columnName parameter

* feat: adde columnname parameter

* feat: added colunmName argument

* convert column filter to server side filter

* address gitar-bot comments

* minor fix

* fix failing test

* address sonar issues

* Revert yarn.lock changes in openmetadata-ui-core-components

Made-with: Cursor

* address comment

* show select all filter in all test case lists

* address copilot comments

* add hideTableFilter and columnFilter props

* address comments

* hide table filter for for add pipeline under table

* fix failing test

* fix failing tests

---------

Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* added unit tests for kafkaconnect pipeline

* Add gitar fix

* MINOR - Column name Length patch (open-metadata#26530)

* chore(bulkEndpoint): create bulk bundle suite body object

* patch(columnName): remove maxLength for column name

* patch(columnName): remove files from commit

* patch(columnName): remove unused code

* fix unit tests

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>

* update alert props (open-metadata#26544)

* Fix: flaky domain rename data products count assertion (open-metadata#26542)

* Fix: flaky domain rename data products count assertion

After a domain rename, the search index update for subdomain-level data
products can lag behind the DB update due to async reindexing by the
governance-bot. The `verifyDataProductsCount` helper was doing a one-shot
`textContent()` + `toBe()` assertion that failed when the search index
hadn't converged yet.

Changes:
- Update `verifyDataProductsCount` to accept optional `{ apiContext, domainFqn }`
  that uses `expect.poll()` to poll the data product search API until
  `hits.total.value` matches the expected count (following the same pattern
  used in glossary.ts, alert.ts, and other polling utilities)
- Switch from one-shot `toBe()` to auto-retrying `toHaveText()` for the UI check
- Pass polling options in the post-rename assertion in Domains.spec.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add HTTP status check in search API poll

Check response.ok() before parsing JSON in the expect.poll callback
to avoid silently treating 4xx/5xx errors as count=0.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove custom timeout from toHaveText assertion

The expect.poll already ensures the search index has converged before
the UI check, so the default Playwright timeout is sufficient.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Refactor InputOutputPortsTab to use Accordion components and update styles (open-metadata#26450)

* Refactor InputOutputPortsTab to use Accordion components and update styles

* addressed PR comment

* fixed playwright test

* addressed gitar comment and fix sonar issues

* fixed overflow issue

* fixed ui issues

* minor fix

* addressed UI fixes

* py_format_check

* Added test for table format without topic prefix

* Removed commented code

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Rohit Jain <60229265+Rohit0301@users.noreply.github.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Akash Verma <138790903+akashverma0786@users.noreply.github.com>
Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Harsh Vador <58542468+harsh-vador@users.noreply.github.com>
Co-authored-by: Harshit Shah <harshit.shah@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: IceS2 <pablo.takara@getcollate.io>
Co-authored-by: Eugenio <eugenio.donaque@getcollate.io>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Sid <30566406+siddhant1@users.noreply.github.com>
Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants