Skip to content

Remove unused /api/v1/domains?limit=10000&fields=parent call on every page load - #26485

Merged
siddhant1 merged 4 commits into
mainfrom
copilot/remove-unused-domain-api-call
Mar 16, 2026
Merged

Remove unused /api/v1/domains?limit=10000&fields=parent call on every page load#26485
siddhant1 merged 4 commits into
mainfrom
copilot/remove-unused-domain-api-call

Conversation

Copilot AI commented Mar 13, 2026

Copy link
Copy Markdown
Contributor

Every page load triggered a GET /api/v1/domains?limit=10000&fields=parent call via fetchDomainList in AuthProvider. The fetched data was written to Zustand (domains, domainOptions) but never read by any component — making the call entirely dead weight.

Changes

  • AuthProvider.tsx — Remove fetchDomainList and its two call sites (getLoggedInUserDetails, handleSuccessfulLogin); drop now-unused imports (getDomainList, ES_MAX_PAGE_SIZE, useDomainStore)
  • DomainDetailPage.component.tsx — Remove updateDomains([response], false) call (was updating store state nobody reads); remove useDomainStore import
  • useDomainStore.ts — Drop unused state (domains, domainOptions, domainLoading) and actions (updateDomains, updateDomainLoading, setDomains)
  • store.interface.ts — Remove corresponding DomainStore interface properties
  • DomainUtils.tsx — Remove getDomainOptions (was only consumed by the removed store actions)
  • Tests — Remove mocks/tests for deleted code (DomainDetailPage.test.tsx, NavBar.test.tsx, DomainUtils.test.tsx)

Type of change:

  • Bug fix
  • Improvement
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>
Copilot AI changed the title [WIP] Remove unused API call for domain data Remove unused /api/v1/domains?limit=10000&fields=parent call on every page load Mar 13, 2026
Copilot AI requested a review from karanh37 March 13, 2026 14:34
@karanh37 karanh37 added the safe to test Add this label to run secure Github workflows on PRs label Mar 13, 2026
@karanh37
karanh37 marked this pull request as ready for review March 13, 2026 14:43
@karanh37
karanh37 requested a review from a team as a code owner March 13, 2026 14:43
@github-actions

github-actions Bot commented Mar 13, 2026

Copy link
Copy Markdown
Contributor

🟡 Playwright Results — all passed (29 flaky)

✅ 3315 passed · ❌ 0 failed · 🟡 29 flaky · ⏭️ 183 skipped

Shard Passed Failed Flaky Skipped
🟡 Shard 1 451 0 4 2
🟡 Shard 2 302 0 3 1
🟡 Shard 3 653 0 5 33
🟡 Shard 4 722 0 8 47
🟡 Shard 5 588 0 3 67
🟡 Shard 6 599 0 6 33
🟡 29 flaky test(s) (passed on retry)
  • Features/CustomizeDetailPage.spec.ts › Database - customization should work (shard 1, 1 retry)
  • Features/CustomizeDetailPage.spec.ts › API Endpoint - customization should work (shard 1, 1 retry)
  • Features/Pagination.spec.ts › should test Data Models normal pagination (shard 1, 1 retry)
  • Flow/Tour.spec.ts › Tour should work from help section (shard 1, 2 retries)
  • Features/ColumnBulkOperations.spec.ts › should load the page with stats cards and grid data (shard 2, 1 retry)
  • Features/ColumnBulkOperations.spec.ts › should show success notification after bulk update (shard 2, 1 retry)
  • Flow/CustomizeWidgets.spec.ts › My Tasks Widget (shard 2, 1 retry)
  • Features/BulkImport.spec.ts › Database (shard 3, 1 retry)
  • Features/DataQuality/TestCaseIncidentPermissions.spec.ts › User with TEST_CASE.EDIT_ALL can see edit icon on incidents (shard 3, 1 retry)
  • Features/DataQuality/TestCaseResultPermissions.spec.ts › User with only VIEW cannot PATCH results (shard 3, 1 retry)
  • Features/ImpactAnalysis.spec.ts › Verify Downstream connections (shard 3, 1 retry)
  • Features/Permissions/GlossaryPermissions.spec.ts › Team-based permissions work correctly (shard 3, 1 retry)
  • Features/UserProfileOnlineStatus.spec.ts › Should update online status in real-time when user becomes active (shard 4, 1 retry)
  • Flow/NotificationAlerts.spec.ts › Multiple Filters Alert (shard 4, 1 retry)
  • Flow/ObservabilityAlerts.spec.ts › Test Suite alert (shard 4, 1 retry)
  • Pages/Customproperties-part2.spec.ts › entityReferenceList shows item count, scrollable list, no expand toggle (shard 4, 1 retry)
  • Pages/DataContractInheritance.spec.ts › Edit Inherited Contract - Creates new asset contract instead of modifying parent (shard 4, 1 retry)
  • Pages/DataProductAndSubdomains.spec.ts › Add expert to data product via UI (shard 4, 1 retry)
  • Pages/Domains.spec.ts › Follow/unfollow subdomain and create nested sub domain (shard 4, 1 retry)
  • Pages/DomainUIInteractions.spec.ts › Add owner to domain via UI (shard 4, 1 retry)
  • Pages/EntityDataConsumer.spec.ts › Glossary Term Add, Update and Remove (shard 5, 1 retry)
  • Pages/EntityDataSteward.spec.ts › Tier Add, Update and Remove (shard 5, 1 retry)
  • Pages/ExplorePageRightPanel.spec.ts › Should verify deleted user not visible in owner selection for table (shard 5, 1 retry)
  • Pages/HyperlinkCustomProperty.spec.ts › should display URL when no display text is provided (shard 6, 1 retry)
  • Pages/InputOutputPorts.spec.ts › Fullscreen lineage is interactive (shard 6, 1 retry)
  • Pages/Login.spec.ts › Refresh should work (shard 6, 1 retry)
  • Pages/ODCSImportExport.spec.ts › Multi-object ODCS contract - object selector shows all schema objects (shard 6, 1 retry)
  • Pages/Users.spec.ts › Permissions for table details page for Data Consumer (shard 6, 1 retry)
  • VersionPages/ServiceEntityVersionPage.spec.ts › Mlmodel Service (shard 6, 1 retry)

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

@gitar-bot

gitar-bot Bot commented Mar 14, 2026

Copy link
Copy Markdown
Code Review ✅ Approved

Removes an unused /api/v1/domains API call that was executing on every page load, reducing unnecessary network overhead. No issues found.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@github-actions

Copy link
Copy Markdown
Contributor

Jest test Coverage

UI tests summary

Lines Statements Branches Functions
Coverage: 66%
66.05% (57464/86996) 45.66% (30404/66583) 48.63% (9112/18734)

@sonarqubecloud

Copy link
Copy Markdown

@siddhant1
siddhant1 merged commit 38aba38 into main Mar 16, 2026
25 checks passed
@siddhant1
siddhant1 deleted the copilot/remove-unused-domain-api-call branch March 16, 2026 07:35
siddhant1 pushed a commit that referenced this pull request Mar 16, 2026
siddhant1 added a commit that referenced this pull request Mar 16, 2026
…ry page load (#26485) (#26508)

Cherry-picked from 38aba38 to 1.12.2

Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
JRDuncan pushed a commit to JRDuncan/OpenMetadata that referenced this pull request Mar 17, 2026
…ry page load (open-metadata#26485)

* Initial plan

* Remove unused domain list API call on every page load

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>
SumanMaharana added a commit that referenced this pull request Apr 8, 2026
* * fixed kafka sink linage

* fix kafka pipline linage topic to table matching

* chore(ui): add sidebar navigation (#26498)

* chore(ui): add sidebar navigation

* fix

* Migrated CustomControls and BulkImportVersionSummary (#26410)

* Migrated CustomControls and BulkImportVersionSummary

* migrated styled button component and playwright fixes

* fixed unit test

* fixed the bulkimportsummary modal and custom control tab item

* fixed the sonar issue

* addressed pr comments

* fixed playwright

* fixed playwright and lint issues

* addressed gitar comment

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* refactor: update toggle component and related fields to use UT_SWITCH type (#26425)

* refactor: update toggle component and related fields to use UT_SWITCH type

* addressed gitar comment

* fixed unit tests

* addressed PR comment

* Chore(deps): Bump undici in /openmetadata-ui/src/main/resources/ui (#26497)

Bumps [undici](https://github.com/nodejs/undici) from 6.23.0 to 6.24.1.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](nodejs/undici@v6.23.0...v6.24.1)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 6.24.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(ui): fix UserTeamSelectableList closing when clicking inside popover in modal (#26502)

* Remove unused `/api/v1/domains?limit=10000&fields=parent` call on every page load (#26485)

* Initial plan

* Remove unused domain list API call on every page load

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

* feat: migrate DeleteModal from MUI to untitledUI components (#26461)

* feat: migrate DeleteModal from MUI to untitledUI components

* fixed the import

* comment addressed

* fix: remove unnecessary span element from DeleteModal

* refactor: remove DeleteModal component and its associated files

* addressed gitar comment

* Added unit tests

* MINOR: Informix Schema update (#26483)

* MINOR: Informix Schema update

* fix

* Update generated TypeScript types

* Add supportdatabase

* Update generated TypeScript types

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>

* ui: make search placeholder in service main tabs dynamic (#26511)

* Revert "feat: migrate DeleteModal from MUI to untitledUI components (#26461)" (#26519)

This reverts commit 8327d22.

* Revert "refactor: update toggle component and related fields to use UT_SWITCH…" (#26520)

This reverts commit 59fe777.

* fix(ui): Test suite add-test-case modal and table UX (#26512)

* fix(ui): Test suite add-test-case modal and table UX

* fix footer padding

* Add eslint-plugin-playwright enforcement with CI check (#26494)

* Add eslint-plugin-playwright enforcement with CI check

Add eslint-plugin-playwright to catch common Playwright anti-patterns
automatically. 13 rules configured in two tiers:

- Error (blocks CI): no-networkidle, no-page-pause, no-focused-test
- Warn (tracks debt): missing-playwright-await, no-wait-for-timeout,
  no-force-option, no-element-handle, no-eval, no-skipped-test,
  prefer-web-first-assertions, no-useless-await, no-wait-for-selector,
  valid-expect

Changes:
- Install eslint-plugin-playwright, configure rules in eslint.config.mjs
- Add yarn lint:playwright script
- Repurpose ui-checkstyle.yml workflow to run Playwright lint on PRs
- Fix last networkidle usage in ClassificationConditionalRendering
- Remove stale eslint-disable comments for undefined rules
- Update PLAYWRIGHT_DEVELOPER_HANDBOOK with ESLint Enforcement section
- Update playwright, writing-playwright-tests, and playwright-validation
  skills to reference lint check

Current: 0 errors, 1657 warnings (CI passes cleanly)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix CodeQL cache poisoning alert: use pull_request instead of pull_request_target

Switch from pull_request_target to pull_request since this workflow
only runs yarn lint:playwright — no secrets or write permissions needed.
This eliminates the untrusted code execution risk flagged by CodeQL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix all 1657 Playwright ESLint warnings across 190 files

Resolve every playwright lint rule violation to enforce test quality:
- no-wait-for-selector (1290): replace page.waitForSelector() with locator.waitFor()
- no-wait-for-timeout (130): replace hardcoded waits with event-driven alternatives
- prefer-web-first-assertions (61): use toHaveText/toBeVisible/toHaveValue
- no-force-option (51): remove { force: true } bypassing actionability checks
- missing-playwright-await (40): add await to fire-and-forget assertions
- no-skipped-test (28): acknowledge skipped tests with eslint-disable reasons
- no-eval (11): replace page.$eval with locator APIs
- no-element-handle (10): replace page.$() with page.locator()
- no-useless-await (8): remove await from synchronous operations
- valid-expect (4): add matchers to bare expect() calls
- no-networkidle (2): replace networkidle with domcontentloaded

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove unnecessary ANTLR install and add --frozen-lockfile to CI

- Remove Install Antlr4 CLI step from playwright-lint job (not needed
  for ESLint, saves 10-30s and avoids external network dependency)
- Add --frozen-lockfile to yarn install for reproducible CI builds

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore ANTLR CLI install — required by yarn postinstall script

yarn install triggers build-check → js-antlr which needs antlr4 CLI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix trivially-true assertion in special character search test

Remove `(await tableRows.count()) >= 0` which is always true since
count() never returns negative. The assertion now properly validates
that either the table or empty state is visible after searching.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violations: add .first() to multi-element waitFor calls

The no-wait-for-selector conversion (waitForSelector → locator.waitFor)
introduced strict mode violations. Playwright's Locator API throws when
waitFor() matches multiple elements, unlike the old waitForSelector API.

Key patterns fixed:
- getByTestId('loader').waitFor() — multiple loaders on page
- getByTestId('select-owner-tabs').getByTestId('loader') — tab loaders
- locator('.ant-skeleton-active/content').waitFor() — multiple skeletons
- locator('table/thead th').waitFor() — multiple tables
- getByTestId('side-panel-classification') — multiple panels
- locator('.ant-select-dropdown:visible') — multiple dropdowns
- locator('.ant-popover').waitFor() — multiple popovers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violation in incidentManager assignee/owner-link

The getByTestId('assignee').getByTestId('owner-link') locator resolves
to 15 elements in the incident manager table (one per row). Adding
.first() matches the original waitForSelector semantics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix test failures from lint conversions: proper waits and retry logic

- GlossaryPermissions: add waitForAllLoadersToDisappear after page nav
- GlossaryP3Tests: restore waits for special char search + error state
- GlossaryStatusFilterLargeDataset: restore waitForTimeout for filter
  state settling (no reliable DOM element to wait for)
- AutoPilot: use waitForAllLoadersToDisappear instead of .first() to
  ensure ALL loaders are gone, increase banner timeout to 60s
- importUtils: use expect().toHaveCount(0) for scoped multi-element
  loader/skeleton waits instead of waitForSelector or .first()
- ColumnBulkOperations: increase response timeout, add retry polling
  for empty state verification
- DomainDataProductsWidgets: add poll-based wait for DOM update after
  asset removal instead of removed waitForTimeout
- customizeLandingPage: scope widget click to dialog to avoid matching
  stale elements on the page behind the modal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore force:true where genuinely needed with eslint-disable comments

The no-force-option ESLint rule removal broke tests where force was required
for Ant Select comboboxes (selected item overlay covers input), popover triggers
(partially obstructed by animation), data grid buttons (covered by overlay),
and drag-and-drop (row hover overlays). Each restoration includes an
eslint-disable-line comment explaining why force is necessary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Stabilize 4 flaky tests with retry wrappers and proper waits

- DataContractsSemanticRules: Wrap reload + status assertions in
  expect().toPass() to handle async backend contract validation
- DataContracts: Increase timeout on dynamically rendered row filter
  and column text assertions that contain UUID-suffixed names
- Customproperties-part2: Wrap user search-and-select loop in
  expect().toPass() retry to handle search dropdown rendering races
- UserProfileOnlineStatus: Add profile render wait, navigation
  completion guard, and increased badge visibility timeout

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Replace .first().waitFor loader pattern with waitForAllLoadersToDisappear

The pattern `page.getByTestId('loader').first().waitFor({ state: 'detached' })`
only waits for the FIRST loader element to detach. When multiple loaders exist
(nested components, tabs, popovers), this causes tests to proceed before the
page is fully loaded — the root cause of ~18 flaky tests.

Replaced ~500 occurrences across 103 files with the correct
`waitForAllLoadersToDisappear(page)` which uses `expect(loaders).toHaveCount(0)`
to wait for ALL loaders to disappear.

Scoped loader waits (e.g., within select-owner-tabs, test-case-container,
tags-container) are intentionally preserved since they correctly target a
specific container's single loader.

Also fixes UserProfileOnlineStatus.spec.ts:
- Added createOrFetchUser helper for idempotent user creation on retries
- Added afterAll cleanup for test users
- Replaced waitForURL with redirectToHomePage for navigation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>

* [Fix-25563] - Issue in Search Entity By Key  (#26482)

* Fix Issue Entity Not found , entity missing from search

* Fix Tests

* Fix Broken relationship issue for upstreamEntityRelationship

* Add Exponential Retry

* Remove Entity Not found from query

* Address Reciew Comments

* Address more review

* Fix Missing Columns Index

* Fix Join add stats

* Column stats and Merged Main

* Add separate custom bulk processor for column

* Fix reindex job falsely killed after 1 hour by orphan monitor

  The OrphanJobMonitor uses job.updatedAt as a liveness signal, but
  updatedAt was only set during state transitions (READY, RUNNING),
  never refreshed during processing. After 10 minutes the job appeared
  orphaned; at the 1-hour recovery window it was force-failed.

  Fix: touch updatedAt alongside the lock refresh (every 60s) so the
  staleness check stays satisfied while the coordinator is alive.

* Set Failure callback for columnBulkProcessor

* fix review comments

* address review comments from claude

* Review Mocked tests

* Address review comments. Both fixed:

                           1. retryOnConflict(3) restored on all 4 call sites — updateEntity and upsertDocument in both ElasticSearchEntityManager and OpenSearchEntityManager. This handles shard-level
                           version conflicts independently from the 429 retry logic in SearchRetryUtil.
                           2. drainPendingColumnFutures race fixed in both bulk sinks — replaced iterate + clear() with poll() loop, which atomically removes each element from the deque so no
                           concurrently-added futures can be lost

* Get proper error message from Elastic and OpenSearch

* Fix Breaking Test

* MINOR: Port py-tests improvements to py-tests-postgres workflow (#26517)

Apply the same structural improvements from the py-tests workflow:
- Add integration test sharding (shard-1/shard-2) for parallelism
- Replace `make run_python_tests` with nox integration-tests session
- Add explicit timeout-minutes (180) and descriptive job name
- Remove unnecessary fetch-depth: 0 from checkout
- Normalize indentation to 2-space with proper YAML style

* Use context to enhance recognition by default (#25856)

* Implement custom context enhancement for Presidio recognizers

Presidio's default context enhancement relies heavily on NLP and often fails
when analyzing individual values rather than full text. This implements a
custom context enhancement that:

- Boosts recognizer scores to MAX when context keywords match
- Applies a minimum score threshold (0.3) before enhancement
- Skips already-enhanced results to prevent double-boosting
- Introduces a decorator pattern for composing recognizer enhancements
- Adds eager_us_bank_recognizer with higher base scores for better results

The enhancement works by checking if any context words from the recognizer
match the provided context list, then boosting the confidence score to
maximum and setting the IS_SCORE_ENHANCED_BY_CONTEXT_KEY metadata flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Integrate context enhancement into recognizer factory

Updates PresidioRecognizerFactory to apply the new decorator pattern:

- All recognizers now use enhance_using_context decorator
- Confidence threshold filtering applied via filter_enhanced_results_below_threshold
- Decorators composed using decorate_recognizer for clean application
- Context passed to PatternRecognizer during creation

This ensures all enabled recognizers benefit from custom context enhancement
while maintaining backward compatibility with confidence thresholds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add migration to update PII tag recognizers with enhanced configs

Adds a v1.1.22 migration that updates existing PII tags with improved
recognizer configurations featuring context keywords and optimized patterns.

Changes:
- Add patchRecognizers method in CollectionDAO for updating tag recognizers
- Implement setRecognizersForSensitiveTags in MigrationUtil to load and apply
  recognizer configs from piiTagsWithRecognizers.json
- Update piiTagsWithRecognizers.json with context keywords for better
  classification accuracy
- Execute migration as post-DDL script for both MySQL and PostgreSQL

This migration ensures existing deployments benefit from the improved context
enhancement logic without manual reconfiguration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix potential NPE and type mismatch in migration utility

Addresses code review feedback:

1. Fix potential NullPointerException in setRecognizersForSensitiveTags
   - Use Boolean.TRUE.equals() instead of auto-unboxing for nullable Boolean
   - Prevents NPE when autoClassificationEnabled is absent from JSON
   - Follows existing pattern from v1120/MigrationUtil.java

2. Fix Boolean vs boolean type mismatch in updateTagRecognizers
   - Change isForceMigration parameter from boxed Boolean to primitive boolean
   - Matches caller signature and eliminates latent NPE risk
   - Maintains consistency across method signatures

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add context to NHSRecognizer

* Fix typing

* Fix broken unit tests

* Fix broken integration test

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Chore(UI): Remove tags customization option (#26514)

* Add regenerate-bot-tokens operation for JWT key rotation (#26477)

* Add regenerate-bot-tokens operation for JWT key rotation

Add a new `regenerate-bot-tokens` subcommand to OpenMetadataOperations
that regenerates JWT tokens for all bot users. This is needed when
rotating JWT signing keys or changing the cluster name (e.g., during
disaster recovery from POV to PROD).

The command:
- Iterates through all non-deleted bots with pagination
- Regenerates JWT tokens only for bots using JWT authentication
- Skips bots without associated users or non-JWT auth mechanisms
- Displays results in an ASCII table with status per bot

Closes open-metadata/openmetadata-collate#3157

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix review: add --expiry CLI parameter and null-guard paging

- Token expiry is now a CLI parameter (--expiry) defaulting to Unlimited,
  so operators can choose OneHour, One, Seven, Thirty, Sixty, Ninety, or
  Unlimited when regenerating
- Add null guard on getPaging() to prevent NPE on empty result sets

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Document regenerate-bot-tokens expiry values in help output

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Move bot token regeneration logic to server API

Move the token regeneration logic from the ops CLI into a new REST
endpoint PUT /v1/users/regenerateBotTokens (admin-only). The ops
command now calls the server API via HTTP, following the same pattern
as deploy-pipelines.

This keeps business logic in the server and the ops CLI as a thin
HTTP client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert server API approach, use direct DB access for token regeneration

After JWT key rotation, all existing bot tokens are invalid — including
the ingestion-bot token needed to authenticate against the server API.
This command must operate directly on the database to recover from
that state. Added a doc comment explaining the reasoning.

Also reverts the UserResource endpoint added in the previous commit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix compilation: use fully-qualified ResultList to match existing pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix botUser always null: fetch each bot individually via getByName

listAll/listAfter use setFieldsInBulk which only calls registered
fieldFetchers (tags, owners, etc.) — not the entity-specific setFields
override. BotRepository.setFields populates botUser via a relationship
lookup, but this is never called in bulk listing. Fetch each bot
individually with getByName which triggers setFields properly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add integration tests for regenerate-bot-tokens operation

Tests cover token regeneration producing new tokens, respecting
expiry parameters, and handling multiple bots independently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix flaky test: use different expiries to guarantee distinct tokens

Unlimited tokens generated in the same second produce identical JWTs
(same iat, null exp). Changed test to use different expiry values so
the tokens are guaranteed to differ.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* format

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* Fix #2730: Add tempLineageTables field to lineage details (#26487)

* Fix #2730: Lineage Add Temp Node Support

* missing sql file

* Update generated TypeScript types

* Update generated TypeScript types

* address guitar commens

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Feature: Fabric's lineage, usage and profiler support (#26373)

* Feature: Fabric's lineage, usage and profiler support

* Python Checkstyle

* Update generated TypeScript types

* fix: narrow Fabric fallback from except Exception to except ProgrammingError

* Addressed comments

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* chore(ui): Add Test Case list filters while creating a bundle suite (#26293)

* fix: Add Test Case list filters (Status, Test type, Table, Column) and E2E coverage

- Add filters to AddTestCaseList: Status (single), Test type (single), Table (multi), Column (multi)
- Use SearchDropdown for all four filters with hideCounts
- Client-side filtering for Table/Column; API refetch for Status/Test type
- Add AddTestCaseListFilters component and constants
- Add AddTestCaseList.utils (getTableFilterOptions, getColumnFilterOptions, filterTestCasesByTableAndColumn, getSelectedOptionsFromKeys)
- Unit tests: AddTestCaseList.component.test.tsx (filter tests), AddTestCaseList.utils.test.ts
- E2E: TestSuite.spec.ts - flatten steps, verify/apply/clear filters in create flow

Fixes https://github.com/open-metadata/openmetadata-collate/issues/3045

Made-with: Cursor

* fix lint

* address gitar-bot comments

* fix sonar issues

* address gitar-bot comments and sonar issues

* minor fix

* fix gitar-bot comments

* use backend filter for table type

* feat: added columnName parameter

* feat: adde columnname parameter

* feat: added colunmName argument

* convert column filter to server side filter

* address gitar-bot comments

* minor fix

* fix failing test

* address sonar issues

* Revert yarn.lock changes in openmetadata-ui-core-components

Made-with: Cursor

* address comment

* show select all filter in all test case lists

* address copilot comments

* add hideTableFilter and columnFilter props

* address comments

* hide table filter for for add pipeline under table

* fix failing test

* fix failing tests

---------

Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* added unit tests for kafkaconnect pipeline

* Add gitar fix

* MINOR - Column name Length patch (#26530)

* chore(bulkEndpoint): create bulk bundle suite body object

* patch(columnName): remove maxLength for column name

* patch(columnName): remove files from commit

* patch(columnName): remove unused code

* fix unit tests

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>

* update alert props (#26544)

* Fix: flaky domain rename data products count assertion (#26542)

* Fix: flaky domain rename data products count assertion

After a domain rename, the search index update for subdomain-level data
products can lag behind the DB update due to async reindexing by the
governance-bot. The `verifyDataProductsCount` helper was doing a one-shot
`textContent()` + `toBe()` assertion that failed when the search index
hadn't converged yet.

Changes:
- Update `verifyDataProductsCount` to accept optional `{ apiContext, domainFqn }`
  that uses `expect.poll()` to poll the data product search API until
  `hits.total.value` matches the expected count (following the same pattern
  used in glossary.ts, alert.ts, and other polling utilities)
- Switch from one-shot `toBe()` to auto-retrying `toHaveText()` for the UI check
- Pass polling options in the post-rename assertion in Domains.spec.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add HTTP status check in search API poll

Check response.ok() before parsing JSON in the expect.poll callback
to avoid silently treating 4xx/5xx errors as count=0.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove custom timeout from toHaveText assertion

The expect.poll already ensures the search index has converged before
the UI check, so the default Playwright timeout is sufficient.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Refactor InputOutputPortsTab to use Accordion components and update styles (#26450)

* Refactor InputOutputPortsTab to use Accordion components and update styles

* addressed PR comment

* fixed playwright test

* addressed gitar comment and fix sonar issues

* fixed overflow issue

* fixed ui issues

* minor fix

* addressed UI fixes

* py_format_check

* Added test for table format without topic prefix

* Removed commented code

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Rohit Jain <60229265+Rohit0301@users.noreply.github.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Akash Verma <138790903+akashverma0786@users.noreply.github.com>
Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Harsh Vador <58542468+harsh-vador@users.noreply.github.com>
Co-authored-by: Harshit Shah <harshit.shah@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: IceS2 <pablo.takara@getcollate.io>
Co-authored-by: Eugenio <eugenio.donaque@getcollate.io>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Sid <30566406+siddhant1@users.noreply.github.com>
Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
SaaiAravindhRaja pushed a commit to SaaiAravindhRaja/OpenMetadata that referenced this pull request Apr 12, 2026
* * fixed kafka sink linage

* fix kafka pipline linage topic to table matching

* chore(ui): add sidebar navigation (open-metadata#26498)

* chore(ui): add sidebar navigation

* fix

* Migrated CustomControls and BulkImportVersionSummary (open-metadata#26410)

* Migrated CustomControls and BulkImportVersionSummary

* migrated styled button component and playwright fixes

* fixed unit test

* fixed the bulkimportsummary modal and custom control tab item

* fixed the sonar issue

* addressed pr comments

* fixed playwright

* fixed playwright and lint issues

* addressed gitar comment

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* refactor: update toggle component and related fields to use UT_SWITCH type (open-metadata#26425)

* refactor: update toggle component and related fields to use UT_SWITCH type

* addressed gitar comment

* fixed unit tests

* addressed PR comment

* Chore(deps): Bump undici in /openmetadata-ui/src/main/resources/ui (open-metadata#26497)

Bumps [undici](https://github.com/nodejs/undici) from 6.23.0 to 6.24.1.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](nodejs/undici@v6.23.0...v6.24.1)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 6.24.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(ui): fix UserTeamSelectableList closing when clicking inside popover in modal (open-metadata#26502)

* Remove unused `/api/v1/domains?limit=10000&fields=parent` call on every page load (open-metadata#26485)

* Initial plan

* Remove unused domain list API call on every page load

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

* feat: migrate DeleteModal from MUI to untitledUI components (open-metadata#26461)

* feat: migrate DeleteModal from MUI to untitledUI components

* fixed the import

* comment addressed

* fix: remove unnecessary span element from DeleteModal

* refactor: remove DeleteModal component and its associated files

* addressed gitar comment

* Added unit tests

* MINOR: Informix Schema update (open-metadata#26483)

* MINOR: Informix Schema update

* fix

* Update generated TypeScript types

* Add supportdatabase

* Update generated TypeScript types

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>

* ui: make search placeholder in service main tabs dynamic (open-metadata#26511)

* Revert "feat: migrate DeleteModal from MUI to untitledUI components (open-metadata#26461)" (open-metadata#26519)

This reverts commit 8327d22.

* Revert "refactor: update toggle component and related fields to use UT_SWITCH…" (open-metadata#26520)

This reverts commit 59fe777.

* fix(ui): Test suite add-test-case modal and table UX (open-metadata#26512)

* fix(ui): Test suite add-test-case modal and table UX

* fix footer padding

* Add eslint-plugin-playwright enforcement with CI check (open-metadata#26494)

* Add eslint-plugin-playwright enforcement with CI check

Add eslint-plugin-playwright to catch common Playwright anti-patterns
automatically. 13 rules configured in two tiers:

- Error (blocks CI): no-networkidle, no-page-pause, no-focused-test
- Warn (tracks debt): missing-playwright-await, no-wait-for-timeout,
  no-force-option, no-element-handle, no-eval, no-skipped-test,
  prefer-web-first-assertions, no-useless-await, no-wait-for-selector,
  valid-expect

Changes:
- Install eslint-plugin-playwright, configure rules in eslint.config.mjs
- Add yarn lint:playwright script
- Repurpose ui-checkstyle.yml workflow to run Playwright lint on PRs
- Fix last networkidle usage in ClassificationConditionalRendering
- Remove stale eslint-disable comments for undefined rules
- Update PLAYWRIGHT_DEVELOPER_HANDBOOK with ESLint Enforcement section
- Update playwright, writing-playwright-tests, and playwright-validation
  skills to reference lint check

Current: 0 errors, 1657 warnings (CI passes cleanly)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix CodeQL cache poisoning alert: use pull_request instead of pull_request_target

Switch from pull_request_target to pull_request since this workflow
only runs yarn lint:playwright — no secrets or write permissions needed.
This eliminates the untrusted code execution risk flagged by CodeQL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix all 1657 Playwright ESLint warnings across 190 files

Resolve every playwright lint rule violation to enforce test quality:
- no-wait-for-selector (1290): replace page.waitForSelector() with locator.waitFor()
- no-wait-for-timeout (130): replace hardcoded waits with event-driven alternatives
- prefer-web-first-assertions (61): use toHaveText/toBeVisible/toHaveValue
- no-force-option (51): remove { force: true } bypassing actionability checks
- missing-playwright-await (40): add await to fire-and-forget assertions
- no-skipped-test (28): acknowledge skipped tests with eslint-disable reasons
- no-eval (11): replace page.$eval with locator APIs
- no-element-handle (10): replace page.$() with page.locator()
- no-useless-await (8): remove await from synchronous operations
- valid-expect (4): add matchers to bare expect() calls
- no-networkidle (2): replace networkidle with domcontentloaded

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove unnecessary ANTLR install and add --frozen-lockfile to CI

- Remove Install Antlr4 CLI step from playwright-lint job (not needed
  for ESLint, saves 10-30s and avoids external network dependency)
- Add --frozen-lockfile to yarn install for reproducible CI builds

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore ANTLR CLI install — required by yarn postinstall script

yarn install triggers build-check → js-antlr which needs antlr4 CLI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix trivially-true assertion in special character search test

Remove `(await tableRows.count()) >= 0` which is always true since
count() never returns negative. The assertion now properly validates
that either the table or empty state is visible after searching.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violations: add .first() to multi-element waitFor calls

The no-wait-for-selector conversion (waitForSelector → locator.waitFor)
introduced strict mode violations. Playwright's Locator API throws when
waitFor() matches multiple elements, unlike the old waitForSelector API.

Key patterns fixed:
- getByTestId('loader').waitFor() — multiple loaders on page
- getByTestId('select-owner-tabs').getByTestId('loader') — tab loaders
- locator('.ant-skeleton-active/content').waitFor() — multiple skeletons
- locator('table/thead th').waitFor() — multiple tables
- getByTestId('side-panel-classification') — multiple panels
- locator('.ant-select-dropdown:visible') — multiple dropdowns
- locator('.ant-popover').waitFor() — multiple popovers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violation in incidentManager assignee/owner-link

The getByTestId('assignee').getByTestId('owner-link') locator resolves
to 15 elements in the incident manager table (one per row). Adding
.first() matches the original waitForSelector semantics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix test failures from lint conversions: proper waits and retry logic

- GlossaryPermissions: add waitForAllLoadersToDisappear after page nav
- GlossaryP3Tests: restore waits for special char search + error state
- GlossaryStatusFilterLargeDataset: restore waitForTimeout for filter
  state settling (no reliable DOM element to wait for)
- AutoPilot: use waitForAllLoadersToDisappear instead of .first() to
  ensure ALL loaders are gone, increase banner timeout to 60s
- importUtils: use expect().toHaveCount(0) for scoped multi-element
  loader/skeleton waits instead of waitForSelector or .first()
- ColumnBulkOperations: increase response timeout, add retry polling
  for empty state verification
- DomainDataProductsWidgets: add poll-based wait for DOM update after
  asset removal instead of removed waitForTimeout
- customizeLandingPage: scope widget click to dialog to avoid matching
  stale elements on the page behind the modal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore force:true where genuinely needed with eslint-disable comments

The no-force-option ESLint rule removal broke tests where force was required
for Ant Select comboboxes (selected item overlay covers input), popover triggers
(partially obstructed by animation), data grid buttons (covered by overlay),
and drag-and-drop (row hover overlays). Each restoration includes an
eslint-disable-line comment explaining why force is necessary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Stabilize 4 flaky tests with retry wrappers and proper waits

- DataContractsSemanticRules: Wrap reload + status assertions in
  expect().toPass() to handle async backend contract validation
- DataContracts: Increase timeout on dynamically rendered row filter
  and column text assertions that contain UUID-suffixed names
- Customproperties-part2: Wrap user search-and-select loop in
  expect().toPass() retry to handle search dropdown rendering races
- UserProfileOnlineStatus: Add profile render wait, navigation
  completion guard, and increased badge visibility timeout

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Replace .first().waitFor loader pattern with waitForAllLoadersToDisappear

The pattern `page.getByTestId('loader').first().waitFor({ state: 'detached' })`
only waits for the FIRST loader element to detach. When multiple loaders exist
(nested components, tabs, popovers), this causes tests to proceed before the
page is fully loaded — the root cause of ~18 flaky tests.

Replaced ~500 occurrences across 103 files with the correct
`waitForAllLoadersToDisappear(page)` which uses `expect(loaders).toHaveCount(0)`
to wait for ALL loaders to disappear.

Scoped loader waits (e.g., within select-owner-tabs, test-case-container,
tags-container) are intentionally preserved since they correctly target a
specific container's single loader.

Also fixes UserProfileOnlineStatus.spec.ts:
- Added createOrFetchUser helper for idempotent user creation on retries
- Added afterAll cleanup for test users
- Replaced waitForURL with redirectToHomePage for navigation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>

* [Fix-25563] - Issue in Search Entity By Key  (open-metadata#26482)

* Fix Issue Entity Not found , entity missing from search

* Fix Tests

* Fix Broken relationship issue for upstreamEntityRelationship

* Add Exponential Retry

* Remove Entity Not found from query

* Address Reciew Comments

* Address more review

* Fix Missing Columns Index

* Fix Join add stats

* Column stats and Merged Main

* Add separate custom bulk processor for column

* Fix reindex job falsely killed after 1 hour by orphan monitor

  The OrphanJobMonitor uses job.updatedAt as a liveness signal, but
  updatedAt was only set during state transitions (READY, RUNNING),
  never refreshed during processing. After 10 minutes the job appeared
  orphaned; at the 1-hour recovery window it was force-failed.

  Fix: touch updatedAt alongside the lock refresh (every 60s) so the
  staleness check stays satisfied while the coordinator is alive.

* Set Failure callback for columnBulkProcessor

* fix review comments

* address review comments from claude

* Review Mocked tests

* Address review comments. Both fixed:

                           1. retryOnConflict(3) restored on all 4 call sites — updateEntity and upsertDocument in both ElasticSearchEntityManager and OpenSearchEntityManager. This handles shard-level
                           version conflicts independently from the 429 retry logic in SearchRetryUtil.
                           2. drainPendingColumnFutures race fixed in both bulk sinks — replaced iterate + clear() with poll() loop, which atomically removes each element from the deque so no
                           concurrently-added futures can be lost

* Get proper error message from Elastic and OpenSearch

* Fix Breaking Test

* MINOR: Port py-tests improvements to py-tests-postgres workflow (open-metadata#26517)

Apply the same structural improvements from the py-tests workflow:
- Add integration test sharding (shard-1/shard-2) for parallelism
- Replace `make run_python_tests` with nox integration-tests session
- Add explicit timeout-minutes (180) and descriptive job name
- Remove unnecessary fetch-depth: 0 from checkout
- Normalize indentation to 2-space with proper YAML style

* Use context to enhance recognition by default (open-metadata#25856)

* Implement custom context enhancement for Presidio recognizers

Presidio's default context enhancement relies heavily on NLP and often fails
when analyzing individual values rather than full text. This implements a
custom context enhancement that:

- Boosts recognizer scores to MAX when context keywords match
- Applies a minimum score threshold (0.3) before enhancement
- Skips already-enhanced results to prevent double-boosting
- Introduces a decorator pattern for composing recognizer enhancements
- Adds eager_us_bank_recognizer with higher base scores for better results

The enhancement works by checking if any context words from the recognizer
match the provided context list, then boosting the confidence score to
maximum and setting the IS_SCORE_ENHANCED_BY_CONTEXT_KEY metadata flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Integrate context enhancement into recognizer factory

Updates PresidioRecognizerFactory to apply the new decorator pattern:

- All recognizers now use enhance_using_context decorator
- Confidence threshold filtering applied via filter_enhanced_results_below_threshold
- Decorators composed using decorate_recognizer for clean application
- Context passed to PatternRecognizer during creation

This ensures all enabled recognizers benefit from custom context enhancement
while maintaining backward compatibility with confidence thresholds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add migration to update PII tag recognizers with enhanced configs

Adds a v1.1.22 migration that updates existing PII tags with improved
recognizer configurations featuring context keywords and optimized patterns.

Changes:
- Add patchRecognizers method in CollectionDAO for updating tag recognizers
- Implement setRecognizersForSensitiveTags in MigrationUtil to load and apply
  recognizer configs from piiTagsWithRecognizers.json
- Update piiTagsWithRecognizers.json with context keywords for better
  classification accuracy
- Execute migration as post-DDL script for both MySQL and PostgreSQL

This migration ensures existing deployments benefit from the improved context
enhancement logic without manual reconfiguration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix potential NPE and type mismatch in migration utility

Addresses code review feedback:

1. Fix potential NullPointerException in setRecognizersForSensitiveTags
   - Use Boolean.TRUE.equals() instead of auto-unboxing for nullable Boolean
   - Prevents NPE when autoClassificationEnabled is absent from JSON
   - Follows existing pattern from v1120/MigrationUtil.java

2. Fix Boolean vs boolean type mismatch in updateTagRecognizers
   - Change isForceMigration parameter from boxed Boolean to primitive boolean
   - Matches caller signature and eliminates latent NPE risk
   - Maintains consistency across method signatures

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add context to NHSRecognizer

* Fix typing

* Fix broken unit tests

* Fix broken integration test

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Chore(UI): Remove tags customization option (open-metadata#26514)

* Add regenerate-bot-tokens operation for JWT key rotation (open-metadata#26477)

* Add regenerate-bot-tokens operation for JWT key rotation

Add a new `regenerate-bot-tokens` subcommand to OpenMetadataOperations
that regenerates JWT tokens for all bot users. This is needed when
rotating JWT signing keys or changing the cluster name (e.g., during
disaster recovery from POV to PROD).

The command:
- Iterates through all non-deleted bots with pagination
- Regenerates JWT tokens only for bots using JWT authentication
- Skips bots without associated users or non-JWT auth mechanisms
- Displays results in an ASCII table with status per bot

Closes open-metadata/openmetadata-collate#3157

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix review: add --expiry CLI parameter and null-guard paging

- Token expiry is now a CLI parameter (--expiry) defaulting to Unlimited,
  so operators can choose OneHour, One, Seven, Thirty, Sixty, Ninety, or
  Unlimited when regenerating
- Add null guard on getPaging() to prevent NPE on empty result sets

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Document regenerate-bot-tokens expiry values in help output

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Move bot token regeneration logic to server API

Move the token regeneration logic from the ops CLI into a new REST
endpoint PUT /v1/users/regenerateBotTokens (admin-only). The ops
command now calls the server API via HTTP, following the same pattern
as deploy-pipelines.

This keeps business logic in the server and the ops CLI as a thin
HTTP client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert server API approach, use direct DB access for token regeneration

After JWT key rotation, all existing bot tokens are invalid — including
the ingestion-bot token needed to authenticate against the server API.
This command must operate directly on the database to recover from
that state. Added a doc comment explaining the reasoning.

Also reverts the UserResource endpoint added in the previous commit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix compilation: use fully-qualified ResultList to match existing pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix botUser always null: fetch each bot individually via getByName

listAll/listAfter use setFieldsInBulk which only calls registered
fieldFetchers (tags, owners, etc.) — not the entity-specific setFields
override. BotRepository.setFields populates botUser via a relationship
lookup, but this is never called in bulk listing. Fetch each bot
individually with getByName which triggers setFields properly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add integration tests for regenerate-bot-tokens operation

Tests cover token regeneration producing new tokens, respecting
expiry parameters, and handling multiple bots independently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix flaky test: use different expiries to guarantee distinct tokens

Unlimited tokens generated in the same second produce identical JWTs
(same iat, null exp). Changed test to use different expiry values so
the tokens are guaranteed to differ.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* format

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* Fix open-metadata#2730: Add tempLineageTables field to lineage details (open-metadata#26487)

* Fix open-metadata#2730: Lineage Add Temp Node Support

* missing sql file

* Update generated TypeScript types

* Update generated TypeScript types

* address guitar commens

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Feature: Fabric's lineage, usage and profiler support (open-metadata#26373)

* Feature: Fabric's lineage, usage and profiler support

* Python Checkstyle

* Update generated TypeScript types

* fix: narrow Fabric fallback from except Exception to except ProgrammingError

* Addressed comments

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* chore(ui): Add Test Case list filters while creating a bundle suite (open-metadata#26293)

* fix: Add Test Case list filters (Status, Test type, Table, Column) and E2E coverage

- Add filters to AddTestCaseList: Status (single), Test type (single), Table (multi), Column (multi)
- Use SearchDropdown for all four filters with hideCounts
- Client-side filtering for Table/Column; API refetch for Status/Test type
- Add AddTestCaseListFilters component and constants
- Add AddTestCaseList.utils (getTableFilterOptions, getColumnFilterOptions, filterTestCasesByTableAndColumn, getSelectedOptionsFromKeys)
- Unit tests: AddTestCaseList.component.test.tsx (filter tests), AddTestCaseList.utils.test.ts
- E2E: TestSuite.spec.ts - flatten steps, verify/apply/clear filters in create flow

Fixes https://github.com/open-metadata/openmetadata-collate/issues/3045

Made-with: Cursor

* fix lint

* address gitar-bot comments

* fix sonar issues

* address gitar-bot comments and sonar issues

* minor fix

* fix gitar-bot comments

* use backend filter for table type

* feat: added columnName parameter

* feat: adde columnname parameter

* feat: added colunmName argument

* convert column filter to server side filter

* address gitar-bot comments

* minor fix

* fix failing test

* address sonar issues

* Revert yarn.lock changes in openmetadata-ui-core-components

Made-with: Cursor

* address comment

* show select all filter in all test case lists

* address copilot comments

* add hideTableFilter and columnFilter props

* address comments

* hide table filter for for add pipeline under table

* fix failing test

* fix failing tests

---------

Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* added unit tests for kafkaconnect pipeline

* Add gitar fix

* MINOR - Column name Length patch (open-metadata#26530)

* chore(bulkEndpoint): create bulk bundle suite body object

* patch(columnName): remove maxLength for column name

* patch(columnName): remove files from commit

* patch(columnName): remove unused code

* fix unit tests

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>

* update alert props (open-metadata#26544)

* Fix: flaky domain rename data products count assertion (open-metadata#26542)

* Fix: flaky domain rename data products count assertion

After a domain rename, the search index update for subdomain-level data
products can lag behind the DB update due to async reindexing by the
governance-bot. The `verifyDataProductsCount` helper was doing a one-shot
`textContent()` + `toBe()` assertion that failed when the search index
hadn't converged yet.

Changes:
- Update `verifyDataProductsCount` to accept optional `{ apiContext, domainFqn }`
  that uses `expect.poll()` to poll the data product search API until
  `hits.total.value` matches the expected count (following the same pattern
  used in glossary.ts, alert.ts, and other polling utilities)
- Switch from one-shot `toBe()` to auto-retrying `toHaveText()` for the UI check
- Pass polling options in the post-rename assertion in Domains.spec.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add HTTP status check in search API poll

Check response.ok() before parsing JSON in the expect.poll callback
to avoid silently treating 4xx/5xx errors as count=0.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove custom timeout from toHaveText assertion

The expect.poll already ensures the search index has converged before
the UI check, so the default Playwright timeout is sufficient.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Refactor InputOutputPortsTab to use Accordion components and update styles (open-metadata#26450)

* Refactor InputOutputPortsTab to use Accordion components and update styles

* addressed PR comment

* fixed playwright test

* addressed gitar comment and fix sonar issues

* fixed overflow issue

* fixed ui issues

* minor fix

* addressed UI fixes

* py_format_check

* Added test for table format without topic prefix

* Removed commented code

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Rohit Jain <60229265+Rohit0301@users.noreply.github.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Akash Verma <138790903+akashverma0786@users.noreply.github.com>
Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Harsh Vador <58542468+harsh-vador@users.noreply.github.com>
Co-authored-by: Harshit Shah <harshit.shah@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: IceS2 <pablo.takara@getcollate.io>
Co-authored-by: Eugenio <eugenio.donaque@getcollate.io>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Sid <30566406+siddhant1@users.noreply.github.com>
Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
SaaiAravindhRaja pushed a commit to SaaiAravindhRaja/OpenMetadata that referenced this pull request Apr 12, 2026
* * fixed kafka sink linage

* fix kafka pipline linage topic to table matching

* chore(ui): add sidebar navigation (open-metadata#26498)

* chore(ui): add sidebar navigation

* fix

* Migrated CustomControls and BulkImportVersionSummary (open-metadata#26410)

* Migrated CustomControls and BulkImportVersionSummary

* migrated styled button component and playwright fixes

* fixed unit test

* fixed the bulkimportsummary modal and custom control tab item

* fixed the sonar issue

* addressed pr comments

* fixed playwright

* fixed playwright and lint issues

* addressed gitar comment

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* refactor: update toggle component and related fields to use UT_SWITCH type (open-metadata#26425)

* refactor: update toggle component and related fields to use UT_SWITCH type

* addressed gitar comment

* fixed unit tests

* addressed PR comment

* Chore(deps): Bump undici in /openmetadata-ui/src/main/resources/ui (open-metadata#26497)

Bumps [undici](https://github.com/nodejs/undici) from 6.23.0 to 6.24.1.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](nodejs/undici@v6.23.0...v6.24.1)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 6.24.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(ui): fix UserTeamSelectableList closing when clicking inside popover in modal (open-metadata#26502)

* Remove unused `/api/v1/domains?limit=10000&fields=parent` call on every page load (open-metadata#26485)

* Initial plan

* Remove unused domain list API call on every page load

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

* feat: migrate DeleteModal from MUI to untitledUI components (open-metadata#26461)

* feat: migrate DeleteModal from MUI to untitledUI components

* fixed the import

* comment addressed

* fix: remove unnecessary span element from DeleteModal

* refactor: remove DeleteModal component and its associated files

* addressed gitar comment

* Added unit tests

* MINOR: Informix Schema update (open-metadata#26483)

* MINOR: Informix Schema update

* fix

* Update generated TypeScript types

* Add supportdatabase

* Update generated TypeScript types

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>

* ui: make search placeholder in service main tabs dynamic (open-metadata#26511)

* Revert "feat: migrate DeleteModal from MUI to untitledUI components (open-metadata#26461)" (open-metadata#26519)

This reverts commit 8327d22.

* Revert "refactor: update toggle component and related fields to use UT_SWITCH…" (open-metadata#26520)

This reverts commit 59fe777.

* fix(ui): Test suite add-test-case modal and table UX (open-metadata#26512)

* fix(ui): Test suite add-test-case modal and table UX

* fix footer padding

* Add eslint-plugin-playwright enforcement with CI check (open-metadata#26494)

* Add eslint-plugin-playwright enforcement with CI check

Add eslint-plugin-playwright to catch common Playwright anti-patterns
automatically. 13 rules configured in two tiers:

- Error (blocks CI): no-networkidle, no-page-pause, no-focused-test
- Warn (tracks debt): missing-playwright-await, no-wait-for-timeout,
  no-force-option, no-element-handle, no-eval, no-skipped-test,
  prefer-web-first-assertions, no-useless-await, no-wait-for-selector,
  valid-expect

Changes:
- Install eslint-plugin-playwright, configure rules in eslint.config.mjs
- Add yarn lint:playwright script
- Repurpose ui-checkstyle.yml workflow to run Playwright lint on PRs
- Fix last networkidle usage in ClassificationConditionalRendering
- Remove stale eslint-disable comments for undefined rules
- Update PLAYWRIGHT_DEVELOPER_HANDBOOK with ESLint Enforcement section
- Update playwright, writing-playwright-tests, and playwright-validation
  skills to reference lint check

Current: 0 errors, 1657 warnings (CI passes cleanly)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix CodeQL cache poisoning alert: use pull_request instead of pull_request_target

Switch from pull_request_target to pull_request since this workflow
only runs yarn lint:playwright — no secrets or write permissions needed.
This eliminates the untrusted code execution risk flagged by CodeQL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix all 1657 Playwright ESLint warnings across 190 files

Resolve every playwright lint rule violation to enforce test quality:
- no-wait-for-selector (1290): replace page.waitForSelector() with locator.waitFor()
- no-wait-for-timeout (130): replace hardcoded waits with event-driven alternatives
- prefer-web-first-assertions (61): use toHaveText/toBeVisible/toHaveValue
- no-force-option (51): remove { force: true } bypassing actionability checks
- missing-playwright-await (40): add await to fire-and-forget assertions
- no-skipped-test (28): acknowledge skipped tests with eslint-disable reasons
- no-eval (11): replace page.$eval with locator APIs
- no-element-handle (10): replace page.$() with page.locator()
- no-useless-await (8): remove await from synchronous operations
- valid-expect (4): add matchers to bare expect() calls
- no-networkidle (2): replace networkidle with domcontentloaded

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove unnecessary ANTLR install and add --frozen-lockfile to CI

- Remove Install Antlr4 CLI step from playwright-lint job (not needed
  for ESLint, saves 10-30s and avoids external network dependency)
- Add --frozen-lockfile to yarn install for reproducible CI builds

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore ANTLR CLI install — required by yarn postinstall script

yarn install triggers build-check → js-antlr which needs antlr4 CLI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix trivially-true assertion in special character search test

Remove `(await tableRows.count()) >= 0` which is always true since
count() never returns negative. The assertion now properly validates
that either the table or empty state is visible after searching.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violations: add .first() to multi-element waitFor calls

The no-wait-for-selector conversion (waitForSelector → locator.waitFor)
introduced strict mode violations. Playwright's Locator API throws when
waitFor() matches multiple elements, unlike the old waitForSelector API.

Key patterns fixed:
- getByTestId('loader').waitFor() — multiple loaders on page
- getByTestId('select-owner-tabs').getByTestId('loader') — tab loaders
- locator('.ant-skeleton-active/content').waitFor() — multiple skeletons
- locator('table/thead th').waitFor() — multiple tables
- getByTestId('side-panel-classification') — multiple panels
- locator('.ant-select-dropdown:visible') — multiple dropdowns
- locator('.ant-popover').waitFor() — multiple popovers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violation in incidentManager assignee/owner-link

The getByTestId('assignee').getByTestId('owner-link') locator resolves
to 15 elements in the incident manager table (one per row). Adding
.first() matches the original waitForSelector semantics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix test failures from lint conversions: proper waits and retry logic

- GlossaryPermissions: add waitForAllLoadersToDisappear after page nav
- GlossaryP3Tests: restore waits for special char search + error state
- GlossaryStatusFilterLargeDataset: restore waitForTimeout for filter
  state settling (no reliable DOM element to wait for)
- AutoPilot: use waitForAllLoadersToDisappear instead of .first() to
  ensure ALL loaders are gone, increase banner timeout to 60s
- importUtils: use expect().toHaveCount(0) for scoped multi-element
  loader/skeleton waits instead of waitForSelector or .first()
- ColumnBulkOperations: increase response timeout, add retry polling
  for empty state verification
- DomainDataProductsWidgets: add poll-based wait for DOM update after
  asset removal instead of removed waitForTimeout
- customizeLandingPage: scope widget click to dialog to avoid matching
  stale elements on the page behind the modal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore force:true where genuinely needed with eslint-disable comments

The no-force-option ESLint rule removal broke tests where force was required
for Ant Select comboboxes (selected item overlay covers input), popover triggers
(partially obstructed by animation), data grid buttons (covered by overlay),
and drag-and-drop (row hover overlays). Each restoration includes an
eslint-disable-line comment explaining why force is necessary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Stabilize 4 flaky tests with retry wrappers and proper waits

- DataContractsSemanticRules: Wrap reload + status assertions in
  expect().toPass() to handle async backend contract validation
- DataContracts: Increase timeout on dynamically rendered row filter
  and column text assertions that contain UUID-suffixed names
- Customproperties-part2: Wrap user search-and-select loop in
  expect().toPass() retry to handle search dropdown rendering races
- UserProfileOnlineStatus: Add profile render wait, navigation
  completion guard, and increased badge visibility timeout

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Replace .first().waitFor loader pattern with waitForAllLoadersToDisappear

The pattern `page.getByTestId('loader').first().waitFor({ state: 'detached' })`
only waits for the FIRST loader element to detach. When multiple loaders exist
(nested components, tabs, popovers), this causes tests to proceed before the
page is fully loaded — the root cause of ~18 flaky tests.

Replaced ~500 occurrences across 103 files with the correct
`waitForAllLoadersToDisappear(page)` which uses `expect(loaders).toHaveCount(0)`
to wait for ALL loaders to disappear.

Scoped loader waits (e.g., within select-owner-tabs, test-case-container,
tags-container) are intentionally preserved since they correctly target a
specific container's single loader.

Also fixes UserProfileOnlineStatus.spec.ts:
- Added createOrFetchUser helper for idempotent user creation on retries
- Added afterAll cleanup for test users
- Replaced waitForURL with redirectToHomePage for navigation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>

* [Fix-25563] - Issue in Search Entity By Key  (open-metadata#26482)

* Fix Issue Entity Not found , entity missing from search

* Fix Tests

* Fix Broken relationship issue for upstreamEntityRelationship

* Add Exponential Retry

* Remove Entity Not found from query

* Address Reciew Comments

* Address more review

* Fix Missing Columns Index

* Fix Join add stats

* Column stats and Merged Main

* Add separate custom bulk processor for column

* Fix reindex job falsely killed after 1 hour by orphan monitor

  The OrphanJobMonitor uses job.updatedAt as a liveness signal, but
  updatedAt was only set during state transitions (READY, RUNNING),
  never refreshed during processing. After 10 minutes the job appeared
  orphaned; at the 1-hour recovery window it was force-failed.

  Fix: touch updatedAt alongside the lock refresh (every 60s) so the
  staleness check stays satisfied while the coordinator is alive.

* Set Failure callback for columnBulkProcessor

* fix review comments

* address review comments from claude

* Review Mocked tests

* Address review comments. Both fixed:

                           1. retryOnConflict(3) restored on all 4 call sites — updateEntity and upsertDocument in both ElasticSearchEntityManager and OpenSearchEntityManager. This handles shard-level
                           version conflicts independently from the 429 retry logic in SearchRetryUtil.
                           2. drainPendingColumnFutures race fixed in both bulk sinks — replaced iterate + clear() with poll() loop, which atomically removes each element from the deque so no
                           concurrently-added futures can be lost

* Get proper error message from Elastic and OpenSearch

* Fix Breaking Test

* MINOR: Port py-tests improvements to py-tests-postgres workflow (open-metadata#26517)

Apply the same structural improvements from the py-tests workflow:
- Add integration test sharding (shard-1/shard-2) for parallelism
- Replace `make run_python_tests` with nox integration-tests session
- Add explicit timeout-minutes (180) and descriptive job name
- Remove unnecessary fetch-depth: 0 from checkout
- Normalize indentation to 2-space with proper YAML style

* Use context to enhance recognition by default (open-metadata#25856)

* Implement custom context enhancement for Presidio recognizers

Presidio's default context enhancement relies heavily on NLP and often fails
when analyzing individual values rather than full text. This implements a
custom context enhancement that:

- Boosts recognizer scores to MAX when context keywords match
- Applies a minimum score threshold (0.3) before enhancement
- Skips already-enhanced results to prevent double-boosting
- Introduces a decorator pattern for composing recognizer enhancements
- Adds eager_us_bank_recognizer with higher base scores for better results

The enhancement works by checking if any context words from the recognizer
match the provided context list, then boosting the confidence score to
maximum and setting the IS_SCORE_ENHANCED_BY_CONTEXT_KEY metadata flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Integrate context enhancement into recognizer factory

Updates PresidioRecognizerFactory to apply the new decorator pattern:

- All recognizers now use enhance_using_context decorator
- Confidence threshold filtering applied via filter_enhanced_results_below_threshold
- Decorators composed using decorate_recognizer for clean application
- Context passed to PatternRecognizer during creation

This ensures all enabled recognizers benefit from custom context enhancement
while maintaining backward compatibility with confidence thresholds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add migration to update PII tag recognizers with enhanced configs

Adds a v1.1.22 migration that updates existing PII tags with improved
recognizer configurations featuring context keywords and optimized patterns.

Changes:
- Add patchRecognizers method in CollectionDAO for updating tag recognizers
- Implement setRecognizersForSensitiveTags in MigrationUtil to load and apply
  recognizer configs from piiTagsWithRecognizers.json
- Update piiTagsWithRecognizers.json with context keywords for better
  classification accuracy
- Execute migration as post-DDL script for both MySQL and PostgreSQL

This migration ensures existing deployments benefit from the improved context
enhancement logic without manual reconfiguration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix potential NPE and type mismatch in migration utility

Addresses code review feedback:

1. Fix potential NullPointerException in setRecognizersForSensitiveTags
   - Use Boolean.TRUE.equals() instead of auto-unboxing for nullable Boolean
   - Prevents NPE when autoClassificationEnabled is absent from JSON
   - Follows existing pattern from v1120/MigrationUtil.java

2. Fix Boolean vs boolean type mismatch in updateTagRecognizers
   - Change isForceMigration parameter from boxed Boolean to primitive boolean
   - Matches caller signature and eliminates latent NPE risk
   - Maintains consistency across method signatures

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add context to NHSRecognizer

* Fix typing

* Fix broken unit tests

* Fix broken integration test

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Chore(UI): Remove tags customization option (open-metadata#26514)

* Add regenerate-bot-tokens operation for JWT key rotation (open-metadata#26477)

* Add regenerate-bot-tokens operation for JWT key rotation

Add a new `regenerate-bot-tokens` subcommand to OpenMetadataOperations
that regenerates JWT tokens for all bot users. This is needed when
rotating JWT signing keys or changing the cluster name (e.g., during
disaster recovery from POV to PROD).

The command:
- Iterates through all non-deleted bots with pagination
- Regenerates JWT tokens only for bots using JWT authentication
- Skips bots without associated users or non-JWT auth mechanisms
- Displays results in an ASCII table with status per bot

Closes open-metadata/openmetadata-collate#3157

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix review: add --expiry CLI parameter and null-guard paging

- Token expiry is now a CLI parameter (--expiry) defaulting to Unlimited,
  so operators can choose OneHour, One, Seven, Thirty, Sixty, Ninety, or
  Unlimited when regenerating
- Add null guard on getPaging() to prevent NPE on empty result sets

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Document regenerate-bot-tokens expiry values in help output

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Move bot token regeneration logic to server API

Move the token regeneration logic from the ops CLI into a new REST
endpoint PUT /v1/users/regenerateBotTokens (admin-only). The ops
command now calls the server API via HTTP, following the same pattern
as deploy-pipelines.

This keeps business logic in the server and the ops CLI as a thin
HTTP client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert server API approach, use direct DB access for token regeneration

After JWT key rotation, all existing bot tokens are invalid — including
the ingestion-bot token needed to authenticate against the server API.
This command must operate directly on the database to recover from
that state. Added a doc comment explaining the reasoning.

Also reverts the UserResource endpoint added in the previous commit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix compilation: use fully-qualified ResultList to match existing pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix botUser always null: fetch each bot individually via getByName

listAll/listAfter use setFieldsInBulk which only calls registered
fieldFetchers (tags, owners, etc.) — not the entity-specific setFields
override. BotRepository.setFields populates botUser via a relationship
lookup, but this is never called in bulk listing. Fetch each bot
individually with getByName which triggers setFields properly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add integration tests for regenerate-bot-tokens operation

Tests cover token regeneration producing new tokens, respecting
expiry parameters, and handling multiple bots independently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix flaky test: use different expiries to guarantee distinct tokens

Unlimited tokens generated in the same second produce identical JWTs
(same iat, null exp). Changed test to use different expiry values so
the tokens are guaranteed to differ.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* format

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* Fix open-metadata#2730: Add tempLineageTables field to lineage details (open-metadata#26487)

* Fix open-metadata#2730: Lineage Add Temp Node Support

* missing sql file

* Update generated TypeScript types

* Update generated TypeScript types

* address guitar commens

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Feature: Fabric's lineage, usage and profiler support (open-metadata#26373)

* Feature: Fabric's lineage, usage and profiler support

* Python Checkstyle

* Update generated TypeScript types

* fix: narrow Fabric fallback from except Exception to except ProgrammingError

* Addressed comments

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* chore(ui): Add Test Case list filters while creating a bundle suite (open-metadata#26293)

* fix: Add Test Case list filters (Status, Test type, Table, Column) and E2E coverage

- Add filters to AddTestCaseList: Status (single), Test type (single), Table (multi), Column (multi)
- Use SearchDropdown for all four filters with hideCounts
- Client-side filtering for Table/Column; API refetch for Status/Test type
- Add AddTestCaseListFilters component and constants
- Add AddTestCaseList.utils (getTableFilterOptions, getColumnFilterOptions, filterTestCasesByTableAndColumn, getSelectedOptionsFromKeys)
- Unit tests: AddTestCaseList.component.test.tsx (filter tests), AddTestCaseList.utils.test.ts
- E2E: TestSuite.spec.ts - flatten steps, verify/apply/clear filters in create flow

Fixes https://github.com/open-metadata/openmetadata-collate/issues/3045

Made-with: Cursor

* fix lint

* address gitar-bot comments

* fix sonar issues

* address gitar-bot comments and sonar issues

* minor fix

* fix gitar-bot comments

* use backend filter for table type

* feat: added columnName parameter

* feat: adde columnname parameter

* feat: added colunmName argument

* convert column filter to server side filter

* address gitar-bot comments

* minor fix

* fix failing test

* address sonar issues

* Revert yarn.lock changes in openmetadata-ui-core-components

Made-with: Cursor

* address comment

* show select all filter in all test case lists

* address copilot comments

* add hideTableFilter and columnFilter props

* address comments

* hide table filter for for add pipeline under table

* fix failing test

* fix failing tests

---------

Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* added unit tests for kafkaconnect pipeline

* Add gitar fix

* MINOR - Column name Length patch (open-metadata#26530)

* chore(bulkEndpoint): create bulk bundle suite body object

* patch(columnName): remove maxLength for column name

* patch(columnName): remove files from commit

* patch(columnName): remove unused code

* fix unit tests

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>

* update alert props (open-metadata#26544)

* Fix: flaky domain rename data products count assertion (open-metadata#26542)

* Fix: flaky domain rename data products count assertion

After a domain rename, the search index update for subdomain-level data
products can lag behind the DB update due to async reindexing by the
governance-bot. The `verifyDataProductsCount` helper was doing a one-shot
`textContent()` + `toBe()` assertion that failed when the search index
hadn't converged yet.

Changes:
- Update `verifyDataProductsCount` to accept optional `{ apiContext, domainFqn }`
  that uses `expect.poll()` to poll the data product search API until
  `hits.total.value` matches the expected count (following the same pattern
  used in glossary.ts, alert.ts, and other polling utilities)
- Switch from one-shot `toBe()` to auto-retrying `toHaveText()` for the UI check
- Pass polling options in the post-rename assertion in Domains.spec.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add HTTP status check in search API poll

Check response.ok() before parsing JSON in the expect.poll callback
to avoid silently treating 4xx/5xx errors as count=0.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove custom timeout from toHaveText assertion

The expect.poll already ensures the search index has converged before
the UI check, so the default Playwright timeout is sufficient.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Refactor InputOutputPortsTab to use Accordion components and update styles (open-metadata#26450)

* Refactor InputOutputPortsTab to use Accordion components and update styles

* addressed PR comment

* fixed playwright test

* addressed gitar comment and fix sonar issues

* fixed overflow issue

* fixed ui issues

* minor fix

* addressed UI fixes

* py_format_check

* Added test for table format without topic prefix

* Removed commented code

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Rohit Jain <60229265+Rohit0301@users.noreply.github.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Akash Verma <138790903+akashverma0786@users.noreply.github.com>
Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Harsh Vador <58542468+harsh-vador@users.noreply.github.com>
Co-authored-by: Harshit Shah <harshit.shah@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: IceS2 <pablo.takara@getcollate.io>
Co-authored-by: Eugenio <eugenio.donaque@getcollate.io>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Sid <30566406+siddhant1@users.noreply.github.com>
Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants