Skip to content

refactor: update toggle component and related fields to use UT_SWITCH type - #26425

Merged
Rohit0301 merged 7 commits into
mainfrom
migrate-mui-switch
Mar 16, 2026
Merged

refactor: update toggle component and related fields to use UT_SWITCH type#26425
Rohit0301 merged 7 commits into
mainfrom
migrate-mui-switch

Conversation

@Rohit0301

@Rohit0301 Rohit0301 commented Mar 12, 2026

Copy link
Copy Markdown
Contributor

Describe your changes:

Removed MUISwitch component and using core-ui Toggle component in form utils.

collate PR - https://github.com/open-metadata/openmetadata-collate/pull/3174

Fixes

I worked on ... because ...

Type of change:

  • Bug fix
  • Improvement
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

Summary by Gitar

  • Removed deprecated component:
    • Deleted MUISwitch component and all associated test files
  • Updated field type enum:
    • Renamed FieldTypes.SWITCH_MUI to FieldTypes.UT_SWITCH in FormUtils interface
  • Migrated to core-ui Toggle:
    • Replaced MUISwitch with Toggle from @openmetadata/ui-core-components in formUtils
    • Updated form field configuration to use Toggle props (isDisabled, isSelected)
  • Exported Toggle interface:
    • Made ToggleProps exportable from toggle component
  • Updated form fields:
    • Removed muiLabel property from tag form fields, using label instead
    • Refactored disabled field and mutually exclusive field configurations

This will update automatically on new commits.

@github-actions

Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

Comment thread openmetadata-ui/src/main/resources/ui/src/utils/DashboardServiceUtils.ts Outdated
Comment thread openmetadata-ui/src/main/resources/ui/src/utils/formUtils.tsx
@Rohit0301
Rohit0301 force-pushed the migrate-mui-switch branch from 4af4d81 to 55a819a Compare March 12, 2026 10:29
@github-actions

Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@github-actions

Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@github-actions

Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@github-actions

github-actions Bot commented Mar 12, 2026

Copy link
Copy Markdown
Contributor

Jest test Coverage

UI tests summary

Lines Statements Branches Functions
Coverage: 65%
65.97% (57381/86977) 45.52% (30272/66497) 48.47% (9077/18727)

Comment thread openmetadata-ui/src/main/resources/ui/src/utils/formUtils.tsx Outdated
@gitar-bot

gitar-bot Bot commented Mar 13, 2026

Copy link
Copy Markdown
Code Review 👍 Approved with suggestions 2 resolved / 3 findings

Migrates toggle component and related fields to use UT_SWITCH type with unit tests fixed and previous breaking change addressed. Consider removing the dead muiLabel property from the FieldProp interface to clean up the refactor.

💡 Quality: Dead muiLabel property left in FieldProp interface

📄 openmetadata-ui/src/main/resources/ui/src/interface/FormUtils.interface.ts:85

The muiLabel property on the FieldProp interface (line 85 of FormUtils.interface.ts) is no longer used anywhere in the codebase — all callers that previously set muiLabel have been updated to use label instead, and the consumer in formUtils.tsx no longer reads it. This dead property should be removed to complete the cleanup and avoid confusing future contributors.

Suggested fix
Remove the `muiLabel?: ReactNode;` line from the `FieldProp` interface in FormUtils.interface.ts.
✅ 2 resolved
Bug: SSRS dashboard schema removed, breaking SSRS service config

📄 openmetadata-ui/src/main/resources/ui/src/utils/DashboardServiceUtils.ts:142
The import of ssrsConnection was removed and schema = ssrsConnection was commented out in the Ssrs case of getDashboardConfig. This causes the function to return an empty object {} as the schema for SSRS (since schema is initialized to {} at line 45). This will break the SSRS service setup/edit flow by rendering no connection configuration fields.

This change appears accidental and unrelated to the toggle/switch refactor.

Bug: Toggle spread may override explicit label prop

📄 openmetadata-ui/src/main/resources/ui/src/utils/formUtils.tsx:468 📄 openmetadata-ui/src/main/resources/ui/src/pages/TagsPage/tagFormFields.tsx:178
In the UT_SWITCH case in formUtils.tsx, label is not destructured out of props as ToggleProps, but it's explicitly set on the <Toggle> component. The {...switchRest} spread comes after the explicit label prop, so if any caller passes label inside the field's props object, it would override the computed label={typeof label === 'string' ? label : undefined}. This is a latent issue — current callers don't pass label in props, but it's fragile.

Additionally, other props like data-testid and className in switchRest are passed through correctly, but inputProps (used in the disabled field's props) is not a valid Toggle prop and will be silently ignored or trigger a React warning.

🤖 Prompt for agents
Code Review: Migrates toggle component and related fields to use UT_SWITCH type with unit tests fixed and previous breaking change addressed. Consider removing the dead `muiLabel` property from the FieldProp interface to clean up the refactor.

1. 💡 Quality: Dead `muiLabel` property left in FieldProp interface
   Files: openmetadata-ui/src/main/resources/ui/src/interface/FormUtils.interface.ts:85

   The `muiLabel` property on the `FieldProp` interface (line 85 of FormUtils.interface.ts) is no longer used anywhere in the codebase — all callers that previously set `muiLabel` have been updated to use `label` instead, and the consumer in `formUtils.tsx` no longer reads it. This dead property should be removed to complete the cleanup and avoid confusing future contributors.

   Suggested fix:
   Remove the `muiLabel?: ReactNode;` line from the `FieldProp` interface in FormUtils.interface.ts.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@sonarqubecloud

Copy link
Copy Markdown

@Rohit0301
Rohit0301 merged commit 59fe777 into main Mar 16, 2026
26 checks passed
@Rohit0301
Rohit0301 deleted the migrate-mui-switch branch March 16, 2026 06:13
JRDuncan pushed a commit to JRDuncan/OpenMetadata that referenced this pull request Mar 17, 2026
… type (open-metadata#26425)

* refactor: update toggle component and related fields to use UT_SWITCH type

* addressed gitar comment

* fixed unit tests

* addressed PR comment
SumanMaharana added a commit that referenced this pull request Apr 8, 2026
* * fixed kafka sink linage

* fix kafka pipline linage topic to table matching

* chore(ui): add sidebar navigation (#26498)

* chore(ui): add sidebar navigation

* fix

* Migrated CustomControls and BulkImportVersionSummary (#26410)

* Migrated CustomControls and BulkImportVersionSummary

* migrated styled button component and playwright fixes

* fixed unit test

* fixed the bulkimportsummary modal and custom control tab item

* fixed the sonar issue

* addressed pr comments

* fixed playwright

* fixed playwright and lint issues

* addressed gitar comment

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* refactor: update toggle component and related fields to use UT_SWITCH type (#26425)

* refactor: update toggle component and related fields to use UT_SWITCH type

* addressed gitar comment

* fixed unit tests

* addressed PR comment

* Chore(deps): Bump undici in /openmetadata-ui/src/main/resources/ui (#26497)

Bumps [undici](https://github.com/nodejs/undici) from 6.23.0 to 6.24.1.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](nodejs/undici@v6.23.0...v6.24.1)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 6.24.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(ui): fix UserTeamSelectableList closing when clicking inside popover in modal (#26502)

* Remove unused `/api/v1/domains?limit=10000&fields=parent` call on every page load (#26485)

* Initial plan

* Remove unused domain list API call on every page load

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

* feat: migrate DeleteModal from MUI to untitledUI components (#26461)

* feat: migrate DeleteModal from MUI to untitledUI components

* fixed the import

* comment addressed

* fix: remove unnecessary span element from DeleteModal

* refactor: remove DeleteModal component and its associated files

* addressed gitar comment

* Added unit tests

* MINOR: Informix Schema update (#26483)

* MINOR: Informix Schema update

* fix

* Update generated TypeScript types

* Add supportdatabase

* Update generated TypeScript types

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>

* ui: make search placeholder in service main tabs dynamic (#26511)

* Revert "feat: migrate DeleteModal from MUI to untitledUI components (#26461)" (#26519)

This reverts commit 8327d22.

* Revert "refactor: update toggle component and related fields to use UT_SWITCH…" (#26520)

This reverts commit 59fe777.

* fix(ui): Test suite add-test-case modal and table UX (#26512)

* fix(ui): Test suite add-test-case modal and table UX

* fix footer padding

* Add eslint-plugin-playwright enforcement with CI check (#26494)

* Add eslint-plugin-playwright enforcement with CI check

Add eslint-plugin-playwright to catch common Playwright anti-patterns
automatically. 13 rules configured in two tiers:

- Error (blocks CI): no-networkidle, no-page-pause, no-focused-test
- Warn (tracks debt): missing-playwright-await, no-wait-for-timeout,
  no-force-option, no-element-handle, no-eval, no-skipped-test,
  prefer-web-first-assertions, no-useless-await, no-wait-for-selector,
  valid-expect

Changes:
- Install eslint-plugin-playwright, configure rules in eslint.config.mjs
- Add yarn lint:playwright script
- Repurpose ui-checkstyle.yml workflow to run Playwright lint on PRs
- Fix last networkidle usage in ClassificationConditionalRendering
- Remove stale eslint-disable comments for undefined rules
- Update PLAYWRIGHT_DEVELOPER_HANDBOOK with ESLint Enforcement section
- Update playwright, writing-playwright-tests, and playwright-validation
  skills to reference lint check

Current: 0 errors, 1657 warnings (CI passes cleanly)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix CodeQL cache poisoning alert: use pull_request instead of pull_request_target

Switch from pull_request_target to pull_request since this workflow
only runs yarn lint:playwright — no secrets or write permissions needed.
This eliminates the untrusted code execution risk flagged by CodeQL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix all 1657 Playwright ESLint warnings across 190 files

Resolve every playwright lint rule violation to enforce test quality:
- no-wait-for-selector (1290): replace page.waitForSelector() with locator.waitFor()
- no-wait-for-timeout (130): replace hardcoded waits with event-driven alternatives
- prefer-web-first-assertions (61): use toHaveText/toBeVisible/toHaveValue
- no-force-option (51): remove { force: true } bypassing actionability checks
- missing-playwright-await (40): add await to fire-and-forget assertions
- no-skipped-test (28): acknowledge skipped tests with eslint-disable reasons
- no-eval (11): replace page.$eval with locator APIs
- no-element-handle (10): replace page.$() with page.locator()
- no-useless-await (8): remove await from synchronous operations
- valid-expect (4): add matchers to bare expect() calls
- no-networkidle (2): replace networkidle with domcontentloaded

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove unnecessary ANTLR install and add --frozen-lockfile to CI

- Remove Install Antlr4 CLI step from playwright-lint job (not needed
  for ESLint, saves 10-30s and avoids external network dependency)
- Add --frozen-lockfile to yarn install for reproducible CI builds

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore ANTLR CLI install — required by yarn postinstall script

yarn install triggers build-check → js-antlr which needs antlr4 CLI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix trivially-true assertion in special character search test

Remove `(await tableRows.count()) >= 0` which is always true since
count() never returns negative. The assertion now properly validates
that either the table or empty state is visible after searching.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violations: add .first() to multi-element waitFor calls

The no-wait-for-selector conversion (waitForSelector → locator.waitFor)
introduced strict mode violations. Playwright's Locator API throws when
waitFor() matches multiple elements, unlike the old waitForSelector API.

Key patterns fixed:
- getByTestId('loader').waitFor() — multiple loaders on page
- getByTestId('select-owner-tabs').getByTestId('loader') — tab loaders
- locator('.ant-skeleton-active/content').waitFor() — multiple skeletons
- locator('table/thead th').waitFor() — multiple tables
- getByTestId('side-panel-classification') — multiple panels
- locator('.ant-select-dropdown:visible') — multiple dropdowns
- locator('.ant-popover').waitFor() — multiple popovers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violation in incidentManager assignee/owner-link

The getByTestId('assignee').getByTestId('owner-link') locator resolves
to 15 elements in the incident manager table (one per row). Adding
.first() matches the original waitForSelector semantics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix test failures from lint conversions: proper waits and retry logic

- GlossaryPermissions: add waitForAllLoadersToDisappear after page nav
- GlossaryP3Tests: restore waits for special char search + error state
- GlossaryStatusFilterLargeDataset: restore waitForTimeout for filter
  state settling (no reliable DOM element to wait for)
- AutoPilot: use waitForAllLoadersToDisappear instead of .first() to
  ensure ALL loaders are gone, increase banner timeout to 60s
- importUtils: use expect().toHaveCount(0) for scoped multi-element
  loader/skeleton waits instead of waitForSelector or .first()
- ColumnBulkOperations: increase response timeout, add retry polling
  for empty state verification
- DomainDataProductsWidgets: add poll-based wait for DOM update after
  asset removal instead of removed waitForTimeout
- customizeLandingPage: scope widget click to dialog to avoid matching
  stale elements on the page behind the modal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore force:true where genuinely needed with eslint-disable comments

The no-force-option ESLint rule removal broke tests where force was required
for Ant Select comboboxes (selected item overlay covers input), popover triggers
(partially obstructed by animation), data grid buttons (covered by overlay),
and drag-and-drop (row hover overlays). Each restoration includes an
eslint-disable-line comment explaining why force is necessary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Stabilize 4 flaky tests with retry wrappers and proper waits

- DataContractsSemanticRules: Wrap reload + status assertions in
  expect().toPass() to handle async backend contract validation
- DataContracts: Increase timeout on dynamically rendered row filter
  and column text assertions that contain UUID-suffixed names
- Customproperties-part2: Wrap user search-and-select loop in
  expect().toPass() retry to handle search dropdown rendering races
- UserProfileOnlineStatus: Add profile render wait, navigation
  completion guard, and increased badge visibility timeout

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Replace .first().waitFor loader pattern with waitForAllLoadersToDisappear

The pattern `page.getByTestId('loader').first().waitFor({ state: 'detached' })`
only waits for the FIRST loader element to detach. When multiple loaders exist
(nested components, tabs, popovers), this causes tests to proceed before the
page is fully loaded — the root cause of ~18 flaky tests.

Replaced ~500 occurrences across 103 files with the correct
`waitForAllLoadersToDisappear(page)` which uses `expect(loaders).toHaveCount(0)`
to wait for ALL loaders to disappear.

Scoped loader waits (e.g., within select-owner-tabs, test-case-container,
tags-container) are intentionally preserved since they correctly target a
specific container's single loader.

Also fixes UserProfileOnlineStatus.spec.ts:
- Added createOrFetchUser helper for idempotent user creation on retries
- Added afterAll cleanup for test users
- Replaced waitForURL with redirectToHomePage for navigation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>

* [Fix-25563] - Issue in Search Entity By Key  (#26482)

* Fix Issue Entity Not found , entity missing from search

* Fix Tests

* Fix Broken relationship issue for upstreamEntityRelationship

* Add Exponential Retry

* Remove Entity Not found from query

* Address Reciew Comments

* Address more review

* Fix Missing Columns Index

* Fix Join add stats

* Column stats and Merged Main

* Add separate custom bulk processor for column

* Fix reindex job falsely killed after 1 hour by orphan monitor

  The OrphanJobMonitor uses job.updatedAt as a liveness signal, but
  updatedAt was only set during state transitions (READY, RUNNING),
  never refreshed during processing. After 10 minutes the job appeared
  orphaned; at the 1-hour recovery window it was force-failed.

  Fix: touch updatedAt alongside the lock refresh (every 60s) so the
  staleness check stays satisfied while the coordinator is alive.

* Set Failure callback for columnBulkProcessor

* fix review comments

* address review comments from claude

* Review Mocked tests

* Address review comments. Both fixed:

                           1. retryOnConflict(3) restored on all 4 call sites — updateEntity and upsertDocument in both ElasticSearchEntityManager and OpenSearchEntityManager. This handles shard-level
                           version conflicts independently from the 429 retry logic in SearchRetryUtil.
                           2. drainPendingColumnFutures race fixed in both bulk sinks — replaced iterate + clear() with poll() loop, which atomically removes each element from the deque so no
                           concurrently-added futures can be lost

* Get proper error message from Elastic and OpenSearch

* Fix Breaking Test

* MINOR: Port py-tests improvements to py-tests-postgres workflow (#26517)

Apply the same structural improvements from the py-tests workflow:
- Add integration test sharding (shard-1/shard-2) for parallelism
- Replace `make run_python_tests` with nox integration-tests session
- Add explicit timeout-minutes (180) and descriptive job name
- Remove unnecessary fetch-depth: 0 from checkout
- Normalize indentation to 2-space with proper YAML style

* Use context to enhance recognition by default (#25856)

* Implement custom context enhancement for Presidio recognizers

Presidio's default context enhancement relies heavily on NLP and often fails
when analyzing individual values rather than full text. This implements a
custom context enhancement that:

- Boosts recognizer scores to MAX when context keywords match
- Applies a minimum score threshold (0.3) before enhancement
- Skips already-enhanced results to prevent double-boosting
- Introduces a decorator pattern for composing recognizer enhancements
- Adds eager_us_bank_recognizer with higher base scores for better results

The enhancement works by checking if any context words from the recognizer
match the provided context list, then boosting the confidence score to
maximum and setting the IS_SCORE_ENHANCED_BY_CONTEXT_KEY metadata flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Integrate context enhancement into recognizer factory

Updates PresidioRecognizerFactory to apply the new decorator pattern:

- All recognizers now use enhance_using_context decorator
- Confidence threshold filtering applied via filter_enhanced_results_below_threshold
- Decorators composed using decorate_recognizer for clean application
- Context passed to PatternRecognizer during creation

This ensures all enabled recognizers benefit from custom context enhancement
while maintaining backward compatibility with confidence thresholds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add migration to update PII tag recognizers with enhanced configs

Adds a v1.1.22 migration that updates existing PII tags with improved
recognizer configurations featuring context keywords and optimized patterns.

Changes:
- Add patchRecognizers method in CollectionDAO for updating tag recognizers
- Implement setRecognizersForSensitiveTags in MigrationUtil to load and apply
  recognizer configs from piiTagsWithRecognizers.json
- Update piiTagsWithRecognizers.json with context keywords for better
  classification accuracy
- Execute migration as post-DDL script for both MySQL and PostgreSQL

This migration ensures existing deployments benefit from the improved context
enhancement logic without manual reconfiguration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix potential NPE and type mismatch in migration utility

Addresses code review feedback:

1. Fix potential NullPointerException in setRecognizersForSensitiveTags
   - Use Boolean.TRUE.equals() instead of auto-unboxing for nullable Boolean
   - Prevents NPE when autoClassificationEnabled is absent from JSON
   - Follows existing pattern from v1120/MigrationUtil.java

2. Fix Boolean vs boolean type mismatch in updateTagRecognizers
   - Change isForceMigration parameter from boxed Boolean to primitive boolean
   - Matches caller signature and eliminates latent NPE risk
   - Maintains consistency across method signatures

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add context to NHSRecognizer

* Fix typing

* Fix broken unit tests

* Fix broken integration test

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Chore(UI): Remove tags customization option (#26514)

* Add regenerate-bot-tokens operation for JWT key rotation (#26477)

* Add regenerate-bot-tokens operation for JWT key rotation

Add a new `regenerate-bot-tokens` subcommand to OpenMetadataOperations
that regenerates JWT tokens for all bot users. This is needed when
rotating JWT signing keys or changing the cluster name (e.g., during
disaster recovery from POV to PROD).

The command:
- Iterates through all non-deleted bots with pagination
- Regenerates JWT tokens only for bots using JWT authentication
- Skips bots without associated users or non-JWT auth mechanisms
- Displays results in an ASCII table with status per bot

Closes open-metadata/openmetadata-collate#3157

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix review: add --expiry CLI parameter and null-guard paging

- Token expiry is now a CLI parameter (--expiry) defaulting to Unlimited,
  so operators can choose OneHour, One, Seven, Thirty, Sixty, Ninety, or
  Unlimited when regenerating
- Add null guard on getPaging() to prevent NPE on empty result sets

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Document regenerate-bot-tokens expiry values in help output

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Move bot token regeneration logic to server API

Move the token regeneration logic from the ops CLI into a new REST
endpoint PUT /v1/users/regenerateBotTokens (admin-only). The ops
command now calls the server API via HTTP, following the same pattern
as deploy-pipelines.

This keeps business logic in the server and the ops CLI as a thin
HTTP client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert server API approach, use direct DB access for token regeneration

After JWT key rotation, all existing bot tokens are invalid — including
the ingestion-bot token needed to authenticate against the server API.
This command must operate directly on the database to recover from
that state. Added a doc comment explaining the reasoning.

Also reverts the UserResource endpoint added in the previous commit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix compilation: use fully-qualified ResultList to match existing pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix botUser always null: fetch each bot individually via getByName

listAll/listAfter use setFieldsInBulk which only calls registered
fieldFetchers (tags, owners, etc.) — not the entity-specific setFields
override. BotRepository.setFields populates botUser via a relationship
lookup, but this is never called in bulk listing. Fetch each bot
individually with getByName which triggers setFields properly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add integration tests for regenerate-bot-tokens operation

Tests cover token regeneration producing new tokens, respecting
expiry parameters, and handling multiple bots independently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix flaky test: use different expiries to guarantee distinct tokens

Unlimited tokens generated in the same second produce identical JWTs
(same iat, null exp). Changed test to use different expiry values so
the tokens are guaranteed to differ.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* format

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* Fix #2730: Add tempLineageTables field to lineage details (#26487)

* Fix #2730: Lineage Add Temp Node Support

* missing sql file

* Update generated TypeScript types

* Update generated TypeScript types

* address guitar commens

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Feature: Fabric's lineage, usage and profiler support (#26373)

* Feature: Fabric's lineage, usage and profiler support

* Python Checkstyle

* Update generated TypeScript types

* fix: narrow Fabric fallback from except Exception to except ProgrammingError

* Addressed comments

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* chore(ui): Add Test Case list filters while creating a bundle suite (#26293)

* fix: Add Test Case list filters (Status, Test type, Table, Column) and E2E coverage

- Add filters to AddTestCaseList: Status (single), Test type (single), Table (multi), Column (multi)
- Use SearchDropdown for all four filters with hideCounts
- Client-side filtering for Table/Column; API refetch for Status/Test type
- Add AddTestCaseListFilters component and constants
- Add AddTestCaseList.utils (getTableFilterOptions, getColumnFilterOptions, filterTestCasesByTableAndColumn, getSelectedOptionsFromKeys)
- Unit tests: AddTestCaseList.component.test.tsx (filter tests), AddTestCaseList.utils.test.ts
- E2E: TestSuite.spec.ts - flatten steps, verify/apply/clear filters in create flow

Fixes https://github.com/open-metadata/openmetadata-collate/issues/3045

Made-with: Cursor

* fix lint

* address gitar-bot comments

* fix sonar issues

* address gitar-bot comments and sonar issues

* minor fix

* fix gitar-bot comments

* use backend filter for table type

* feat: added columnName parameter

* feat: adde columnname parameter

* feat: added colunmName argument

* convert column filter to server side filter

* address gitar-bot comments

* minor fix

* fix failing test

* address sonar issues

* Revert yarn.lock changes in openmetadata-ui-core-components

Made-with: Cursor

* address comment

* show select all filter in all test case lists

* address copilot comments

* add hideTableFilter and columnFilter props

* address comments

* hide table filter for for add pipeline under table

* fix failing test

* fix failing tests

---------

Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* added unit tests for kafkaconnect pipeline

* Add gitar fix

* MINOR - Column name Length patch (#26530)

* chore(bulkEndpoint): create bulk bundle suite body object

* patch(columnName): remove maxLength for column name

* patch(columnName): remove files from commit

* patch(columnName): remove unused code

* fix unit tests

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>

* update alert props (#26544)

* Fix: flaky domain rename data products count assertion (#26542)

* Fix: flaky domain rename data products count assertion

After a domain rename, the search index update for subdomain-level data
products can lag behind the DB update due to async reindexing by the
governance-bot. The `verifyDataProductsCount` helper was doing a one-shot
`textContent()` + `toBe()` assertion that failed when the search index
hadn't converged yet.

Changes:
- Update `verifyDataProductsCount` to accept optional `{ apiContext, domainFqn }`
  that uses `expect.poll()` to poll the data product search API until
  `hits.total.value` matches the expected count (following the same pattern
  used in glossary.ts, alert.ts, and other polling utilities)
- Switch from one-shot `toBe()` to auto-retrying `toHaveText()` for the UI check
- Pass polling options in the post-rename assertion in Domains.spec.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add HTTP status check in search API poll

Check response.ok() before parsing JSON in the expect.poll callback
to avoid silently treating 4xx/5xx errors as count=0.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove custom timeout from toHaveText assertion

The expect.poll already ensures the search index has converged before
the UI check, so the default Playwright timeout is sufficient.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Refactor InputOutputPortsTab to use Accordion components and update styles (#26450)

* Refactor InputOutputPortsTab to use Accordion components and update styles

* addressed PR comment

* fixed playwright test

* addressed gitar comment and fix sonar issues

* fixed overflow issue

* fixed ui issues

* minor fix

* addressed UI fixes

* py_format_check

* Added test for table format without topic prefix

* Removed commented code

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Rohit Jain <60229265+Rohit0301@users.noreply.github.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Akash Verma <138790903+akashverma0786@users.noreply.github.com>
Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Harsh Vador <58542468+harsh-vador@users.noreply.github.com>
Co-authored-by: Harshit Shah <harshit.shah@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: IceS2 <pablo.takara@getcollate.io>
Co-authored-by: Eugenio <eugenio.donaque@getcollate.io>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Sid <30566406+siddhant1@users.noreply.github.com>
Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
SaaiAravindhRaja pushed a commit to SaaiAravindhRaja/OpenMetadata that referenced this pull request Apr 12, 2026
* * fixed kafka sink linage

* fix kafka pipline linage topic to table matching

* chore(ui): add sidebar navigation (open-metadata#26498)

* chore(ui): add sidebar navigation

* fix

* Migrated CustomControls and BulkImportVersionSummary (open-metadata#26410)

* Migrated CustomControls and BulkImportVersionSummary

* migrated styled button component and playwright fixes

* fixed unit test

* fixed the bulkimportsummary modal and custom control tab item

* fixed the sonar issue

* addressed pr comments

* fixed playwright

* fixed playwright and lint issues

* addressed gitar comment

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* refactor: update toggle component and related fields to use UT_SWITCH type (open-metadata#26425)

* refactor: update toggle component and related fields to use UT_SWITCH type

* addressed gitar comment

* fixed unit tests

* addressed PR comment

* Chore(deps): Bump undici in /openmetadata-ui/src/main/resources/ui (open-metadata#26497)

Bumps [undici](https://github.com/nodejs/undici) from 6.23.0 to 6.24.1.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](nodejs/undici@v6.23.0...v6.24.1)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 6.24.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(ui): fix UserTeamSelectableList closing when clicking inside popover in modal (open-metadata#26502)

* Remove unused `/api/v1/domains?limit=10000&fields=parent` call on every page load (open-metadata#26485)

* Initial plan

* Remove unused domain list API call on every page load

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

* feat: migrate DeleteModal from MUI to untitledUI components (open-metadata#26461)

* feat: migrate DeleteModal from MUI to untitledUI components

* fixed the import

* comment addressed

* fix: remove unnecessary span element from DeleteModal

* refactor: remove DeleteModal component and its associated files

* addressed gitar comment

* Added unit tests

* MINOR: Informix Schema update (open-metadata#26483)

* MINOR: Informix Schema update

* fix

* Update generated TypeScript types

* Add supportdatabase

* Update generated TypeScript types

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>

* ui: make search placeholder in service main tabs dynamic (open-metadata#26511)

* Revert "feat: migrate DeleteModal from MUI to untitledUI components (open-metadata#26461)" (open-metadata#26519)

This reverts commit 8327d22.

* Revert "refactor: update toggle component and related fields to use UT_SWITCH…" (open-metadata#26520)

This reverts commit 59fe777.

* fix(ui): Test suite add-test-case modal and table UX (open-metadata#26512)

* fix(ui): Test suite add-test-case modal and table UX

* fix footer padding

* Add eslint-plugin-playwright enforcement with CI check (open-metadata#26494)

* Add eslint-plugin-playwright enforcement with CI check

Add eslint-plugin-playwright to catch common Playwright anti-patterns
automatically. 13 rules configured in two tiers:

- Error (blocks CI): no-networkidle, no-page-pause, no-focused-test
- Warn (tracks debt): missing-playwright-await, no-wait-for-timeout,
  no-force-option, no-element-handle, no-eval, no-skipped-test,
  prefer-web-first-assertions, no-useless-await, no-wait-for-selector,
  valid-expect

Changes:
- Install eslint-plugin-playwright, configure rules in eslint.config.mjs
- Add yarn lint:playwright script
- Repurpose ui-checkstyle.yml workflow to run Playwright lint on PRs
- Fix last networkidle usage in ClassificationConditionalRendering
- Remove stale eslint-disable comments for undefined rules
- Update PLAYWRIGHT_DEVELOPER_HANDBOOK with ESLint Enforcement section
- Update playwright, writing-playwright-tests, and playwright-validation
  skills to reference lint check

Current: 0 errors, 1657 warnings (CI passes cleanly)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix CodeQL cache poisoning alert: use pull_request instead of pull_request_target

Switch from pull_request_target to pull_request since this workflow
only runs yarn lint:playwright — no secrets or write permissions needed.
This eliminates the untrusted code execution risk flagged by CodeQL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix all 1657 Playwright ESLint warnings across 190 files

Resolve every playwright lint rule violation to enforce test quality:
- no-wait-for-selector (1290): replace page.waitForSelector() with locator.waitFor()
- no-wait-for-timeout (130): replace hardcoded waits with event-driven alternatives
- prefer-web-first-assertions (61): use toHaveText/toBeVisible/toHaveValue
- no-force-option (51): remove { force: true } bypassing actionability checks
- missing-playwright-await (40): add await to fire-and-forget assertions
- no-skipped-test (28): acknowledge skipped tests with eslint-disable reasons
- no-eval (11): replace page.$eval with locator APIs
- no-element-handle (10): replace page.$() with page.locator()
- no-useless-await (8): remove await from synchronous operations
- valid-expect (4): add matchers to bare expect() calls
- no-networkidle (2): replace networkidle with domcontentloaded

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove unnecessary ANTLR install and add --frozen-lockfile to CI

- Remove Install Antlr4 CLI step from playwright-lint job (not needed
  for ESLint, saves 10-30s and avoids external network dependency)
- Add --frozen-lockfile to yarn install for reproducible CI builds

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore ANTLR CLI install — required by yarn postinstall script

yarn install triggers build-check → js-antlr which needs antlr4 CLI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix trivially-true assertion in special character search test

Remove `(await tableRows.count()) >= 0` which is always true since
count() never returns negative. The assertion now properly validates
that either the table or empty state is visible after searching.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violations: add .first() to multi-element waitFor calls

The no-wait-for-selector conversion (waitForSelector → locator.waitFor)
introduced strict mode violations. Playwright's Locator API throws when
waitFor() matches multiple elements, unlike the old waitForSelector API.

Key patterns fixed:
- getByTestId('loader').waitFor() — multiple loaders on page
- getByTestId('select-owner-tabs').getByTestId('loader') — tab loaders
- locator('.ant-skeleton-active/content').waitFor() — multiple skeletons
- locator('table/thead th').waitFor() — multiple tables
- getByTestId('side-panel-classification') — multiple panels
- locator('.ant-select-dropdown:visible') — multiple dropdowns
- locator('.ant-popover').waitFor() — multiple popovers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violation in incidentManager assignee/owner-link

The getByTestId('assignee').getByTestId('owner-link') locator resolves
to 15 elements in the incident manager table (one per row). Adding
.first() matches the original waitForSelector semantics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix test failures from lint conversions: proper waits and retry logic

- GlossaryPermissions: add waitForAllLoadersToDisappear after page nav
- GlossaryP3Tests: restore waits for special char search + error state
- GlossaryStatusFilterLargeDataset: restore waitForTimeout for filter
  state settling (no reliable DOM element to wait for)
- AutoPilot: use waitForAllLoadersToDisappear instead of .first() to
  ensure ALL loaders are gone, increase banner timeout to 60s
- importUtils: use expect().toHaveCount(0) for scoped multi-element
  loader/skeleton waits instead of waitForSelector or .first()
- ColumnBulkOperations: increase response timeout, add retry polling
  for empty state verification
- DomainDataProductsWidgets: add poll-based wait for DOM update after
  asset removal instead of removed waitForTimeout
- customizeLandingPage: scope widget click to dialog to avoid matching
  stale elements on the page behind the modal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore force:true where genuinely needed with eslint-disable comments

The no-force-option ESLint rule removal broke tests where force was required
for Ant Select comboboxes (selected item overlay covers input), popover triggers
(partially obstructed by animation), data grid buttons (covered by overlay),
and drag-and-drop (row hover overlays). Each restoration includes an
eslint-disable-line comment explaining why force is necessary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Stabilize 4 flaky tests with retry wrappers and proper waits

- DataContractsSemanticRules: Wrap reload + status assertions in
  expect().toPass() to handle async backend contract validation
- DataContracts: Increase timeout on dynamically rendered row filter
  and column text assertions that contain UUID-suffixed names
- Customproperties-part2: Wrap user search-and-select loop in
  expect().toPass() retry to handle search dropdown rendering races
- UserProfileOnlineStatus: Add profile render wait, navigation
  completion guard, and increased badge visibility timeout

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Replace .first().waitFor loader pattern with waitForAllLoadersToDisappear

The pattern `page.getByTestId('loader').first().waitFor({ state: 'detached' })`
only waits for the FIRST loader element to detach. When multiple loaders exist
(nested components, tabs, popovers), this causes tests to proceed before the
page is fully loaded — the root cause of ~18 flaky tests.

Replaced ~500 occurrences across 103 files with the correct
`waitForAllLoadersToDisappear(page)` which uses `expect(loaders).toHaveCount(0)`
to wait for ALL loaders to disappear.

Scoped loader waits (e.g., within select-owner-tabs, test-case-container,
tags-container) are intentionally preserved since they correctly target a
specific container's single loader.

Also fixes UserProfileOnlineStatus.spec.ts:
- Added createOrFetchUser helper for idempotent user creation on retries
- Added afterAll cleanup for test users
- Replaced waitForURL with redirectToHomePage for navigation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>

* [Fix-25563] - Issue in Search Entity By Key  (open-metadata#26482)

* Fix Issue Entity Not found , entity missing from search

* Fix Tests

* Fix Broken relationship issue for upstreamEntityRelationship

* Add Exponential Retry

* Remove Entity Not found from query

* Address Reciew Comments

* Address more review

* Fix Missing Columns Index

* Fix Join add stats

* Column stats and Merged Main

* Add separate custom bulk processor for column

* Fix reindex job falsely killed after 1 hour by orphan monitor

  The OrphanJobMonitor uses job.updatedAt as a liveness signal, but
  updatedAt was only set during state transitions (READY, RUNNING),
  never refreshed during processing. After 10 minutes the job appeared
  orphaned; at the 1-hour recovery window it was force-failed.

  Fix: touch updatedAt alongside the lock refresh (every 60s) so the
  staleness check stays satisfied while the coordinator is alive.

* Set Failure callback for columnBulkProcessor

* fix review comments

* address review comments from claude

* Review Mocked tests

* Address review comments. Both fixed:

                           1. retryOnConflict(3) restored on all 4 call sites — updateEntity and upsertDocument in both ElasticSearchEntityManager and OpenSearchEntityManager. This handles shard-level
                           version conflicts independently from the 429 retry logic in SearchRetryUtil.
                           2. drainPendingColumnFutures race fixed in both bulk sinks — replaced iterate + clear() with poll() loop, which atomically removes each element from the deque so no
                           concurrently-added futures can be lost

* Get proper error message from Elastic and OpenSearch

* Fix Breaking Test

* MINOR: Port py-tests improvements to py-tests-postgres workflow (open-metadata#26517)

Apply the same structural improvements from the py-tests workflow:
- Add integration test sharding (shard-1/shard-2) for parallelism
- Replace `make run_python_tests` with nox integration-tests session
- Add explicit timeout-minutes (180) and descriptive job name
- Remove unnecessary fetch-depth: 0 from checkout
- Normalize indentation to 2-space with proper YAML style

* Use context to enhance recognition by default (open-metadata#25856)

* Implement custom context enhancement for Presidio recognizers

Presidio's default context enhancement relies heavily on NLP and often fails
when analyzing individual values rather than full text. This implements a
custom context enhancement that:

- Boosts recognizer scores to MAX when context keywords match
- Applies a minimum score threshold (0.3) before enhancement
- Skips already-enhanced results to prevent double-boosting
- Introduces a decorator pattern for composing recognizer enhancements
- Adds eager_us_bank_recognizer with higher base scores for better results

The enhancement works by checking if any context words from the recognizer
match the provided context list, then boosting the confidence score to
maximum and setting the IS_SCORE_ENHANCED_BY_CONTEXT_KEY metadata flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Integrate context enhancement into recognizer factory

Updates PresidioRecognizerFactory to apply the new decorator pattern:

- All recognizers now use enhance_using_context decorator
- Confidence threshold filtering applied via filter_enhanced_results_below_threshold
- Decorators composed using decorate_recognizer for clean application
- Context passed to PatternRecognizer during creation

This ensures all enabled recognizers benefit from custom context enhancement
while maintaining backward compatibility with confidence thresholds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add migration to update PII tag recognizers with enhanced configs

Adds a v1.1.22 migration that updates existing PII tags with improved
recognizer configurations featuring context keywords and optimized patterns.

Changes:
- Add patchRecognizers method in CollectionDAO for updating tag recognizers
- Implement setRecognizersForSensitiveTags in MigrationUtil to load and apply
  recognizer configs from piiTagsWithRecognizers.json
- Update piiTagsWithRecognizers.json with context keywords for better
  classification accuracy
- Execute migration as post-DDL script for both MySQL and PostgreSQL

This migration ensures existing deployments benefit from the improved context
enhancement logic without manual reconfiguration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix potential NPE and type mismatch in migration utility

Addresses code review feedback:

1. Fix potential NullPointerException in setRecognizersForSensitiveTags
   - Use Boolean.TRUE.equals() instead of auto-unboxing for nullable Boolean
   - Prevents NPE when autoClassificationEnabled is absent from JSON
   - Follows existing pattern from v1120/MigrationUtil.java

2. Fix Boolean vs boolean type mismatch in updateTagRecognizers
   - Change isForceMigration parameter from boxed Boolean to primitive boolean
   - Matches caller signature and eliminates latent NPE risk
   - Maintains consistency across method signatures

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add context to NHSRecognizer

* Fix typing

* Fix broken unit tests

* Fix broken integration test

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Chore(UI): Remove tags customization option (open-metadata#26514)

* Add regenerate-bot-tokens operation for JWT key rotation (open-metadata#26477)

* Add regenerate-bot-tokens operation for JWT key rotation

Add a new `regenerate-bot-tokens` subcommand to OpenMetadataOperations
that regenerates JWT tokens for all bot users. This is needed when
rotating JWT signing keys or changing the cluster name (e.g., during
disaster recovery from POV to PROD).

The command:
- Iterates through all non-deleted bots with pagination
- Regenerates JWT tokens only for bots using JWT authentication
- Skips bots without associated users or non-JWT auth mechanisms
- Displays results in an ASCII table with status per bot

Closes open-metadata/openmetadata-collate#3157

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix review: add --expiry CLI parameter and null-guard paging

- Token expiry is now a CLI parameter (--expiry) defaulting to Unlimited,
  so operators can choose OneHour, One, Seven, Thirty, Sixty, Ninety, or
  Unlimited when regenerating
- Add null guard on getPaging() to prevent NPE on empty result sets

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Document regenerate-bot-tokens expiry values in help output

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Move bot token regeneration logic to server API

Move the token regeneration logic from the ops CLI into a new REST
endpoint PUT /v1/users/regenerateBotTokens (admin-only). The ops
command now calls the server API via HTTP, following the same pattern
as deploy-pipelines.

This keeps business logic in the server and the ops CLI as a thin
HTTP client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert server API approach, use direct DB access for token regeneration

After JWT key rotation, all existing bot tokens are invalid — including
the ingestion-bot token needed to authenticate against the server API.
This command must operate directly on the database to recover from
that state. Added a doc comment explaining the reasoning.

Also reverts the UserResource endpoint added in the previous commit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix compilation: use fully-qualified ResultList to match existing pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix botUser always null: fetch each bot individually via getByName

listAll/listAfter use setFieldsInBulk which only calls registered
fieldFetchers (tags, owners, etc.) — not the entity-specific setFields
override. BotRepository.setFields populates botUser via a relationship
lookup, but this is never called in bulk listing. Fetch each bot
individually with getByName which triggers setFields properly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add integration tests for regenerate-bot-tokens operation

Tests cover token regeneration producing new tokens, respecting
expiry parameters, and handling multiple bots independently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix flaky test: use different expiries to guarantee distinct tokens

Unlimited tokens generated in the same second produce identical JWTs
(same iat, null exp). Changed test to use different expiry values so
the tokens are guaranteed to differ.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* format

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* Fix open-metadata#2730: Add tempLineageTables field to lineage details (open-metadata#26487)

* Fix open-metadata#2730: Lineage Add Temp Node Support

* missing sql file

* Update generated TypeScript types

* Update generated TypeScript types

* address guitar commens

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Feature: Fabric's lineage, usage and profiler support (open-metadata#26373)

* Feature: Fabric's lineage, usage and profiler support

* Python Checkstyle

* Update generated TypeScript types

* fix: narrow Fabric fallback from except Exception to except ProgrammingError

* Addressed comments

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* chore(ui): Add Test Case list filters while creating a bundle suite (open-metadata#26293)

* fix: Add Test Case list filters (Status, Test type, Table, Column) and E2E coverage

- Add filters to AddTestCaseList: Status (single), Test type (single), Table (multi), Column (multi)
- Use SearchDropdown for all four filters with hideCounts
- Client-side filtering for Table/Column; API refetch for Status/Test type
- Add AddTestCaseListFilters component and constants
- Add AddTestCaseList.utils (getTableFilterOptions, getColumnFilterOptions, filterTestCasesByTableAndColumn, getSelectedOptionsFromKeys)
- Unit tests: AddTestCaseList.component.test.tsx (filter tests), AddTestCaseList.utils.test.ts
- E2E: TestSuite.spec.ts - flatten steps, verify/apply/clear filters in create flow

Fixes https://github.com/open-metadata/openmetadata-collate/issues/3045

Made-with: Cursor

* fix lint

* address gitar-bot comments

* fix sonar issues

* address gitar-bot comments and sonar issues

* minor fix

* fix gitar-bot comments

* use backend filter for table type

* feat: added columnName parameter

* feat: adde columnname parameter

* feat: added colunmName argument

* convert column filter to server side filter

* address gitar-bot comments

* minor fix

* fix failing test

* address sonar issues

* Revert yarn.lock changes in openmetadata-ui-core-components

Made-with: Cursor

* address comment

* show select all filter in all test case lists

* address copilot comments

* add hideTableFilter and columnFilter props

* address comments

* hide table filter for for add pipeline under table

* fix failing test

* fix failing tests

---------

Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* added unit tests for kafkaconnect pipeline

* Add gitar fix

* MINOR - Column name Length patch (open-metadata#26530)

* chore(bulkEndpoint): create bulk bundle suite body object

* patch(columnName): remove maxLength for column name

* patch(columnName): remove files from commit

* patch(columnName): remove unused code

* fix unit tests

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>

* update alert props (open-metadata#26544)

* Fix: flaky domain rename data products count assertion (open-metadata#26542)

* Fix: flaky domain rename data products count assertion

After a domain rename, the search index update for subdomain-level data
products can lag behind the DB update due to async reindexing by the
governance-bot. The `verifyDataProductsCount` helper was doing a one-shot
`textContent()` + `toBe()` assertion that failed when the search index
hadn't converged yet.

Changes:
- Update `verifyDataProductsCount` to accept optional `{ apiContext, domainFqn }`
  that uses `expect.poll()` to poll the data product search API until
  `hits.total.value` matches the expected count (following the same pattern
  used in glossary.ts, alert.ts, and other polling utilities)
- Switch from one-shot `toBe()` to auto-retrying `toHaveText()` for the UI check
- Pass polling options in the post-rename assertion in Domains.spec.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add HTTP status check in search API poll

Check response.ok() before parsing JSON in the expect.poll callback
to avoid silently treating 4xx/5xx errors as count=0.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove custom timeout from toHaveText assertion

The expect.poll already ensures the search index has converged before
the UI check, so the default Playwright timeout is sufficient.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Refactor InputOutputPortsTab to use Accordion components and update styles (open-metadata#26450)

* Refactor InputOutputPortsTab to use Accordion components and update styles

* addressed PR comment

* fixed playwright test

* addressed gitar comment and fix sonar issues

* fixed overflow issue

* fixed ui issues

* minor fix

* addressed UI fixes

* py_format_check

* Added test for table format without topic prefix

* Removed commented code

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Rohit Jain <60229265+Rohit0301@users.noreply.github.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Akash Verma <138790903+akashverma0786@users.noreply.github.com>
Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Harsh Vador <58542468+harsh-vador@users.noreply.github.com>
Co-authored-by: Harshit Shah <harshit.shah@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: IceS2 <pablo.takara@getcollate.io>
Co-authored-by: Eugenio <eugenio.donaque@getcollate.io>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Sid <30566406+siddhant1@users.noreply.github.com>
Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
SaaiAravindhRaja pushed a commit to SaaiAravindhRaja/OpenMetadata that referenced this pull request Apr 12, 2026
* * fixed kafka sink linage

* fix kafka pipline linage topic to table matching

* chore(ui): add sidebar navigation (open-metadata#26498)

* chore(ui): add sidebar navigation

* fix

* Migrated CustomControls and BulkImportVersionSummary (open-metadata#26410)

* Migrated CustomControls and BulkImportVersionSummary

* migrated styled button component and playwright fixes

* fixed unit test

* fixed the bulkimportsummary modal and custom control tab item

* fixed the sonar issue

* addressed pr comments

* fixed playwright

* fixed playwright and lint issues

* addressed gitar comment

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* refactor: update toggle component and related fields to use UT_SWITCH type (open-metadata#26425)

* refactor: update toggle component and related fields to use UT_SWITCH type

* addressed gitar comment

* fixed unit tests

* addressed PR comment

* Chore(deps): Bump undici in /openmetadata-ui/src/main/resources/ui (open-metadata#26497)

Bumps [undici](https://github.com/nodejs/undici) from 6.23.0 to 6.24.1.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](nodejs/undici@v6.23.0...v6.24.1)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 6.24.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(ui): fix UserTeamSelectableList closing when clicking inside popover in modal (open-metadata#26502)

* Remove unused `/api/v1/domains?limit=10000&fields=parent` call on every page load (open-metadata#26485)

* Initial plan

* Remove unused domain list API call on every page load

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

* feat: migrate DeleteModal from MUI to untitledUI components (open-metadata#26461)

* feat: migrate DeleteModal from MUI to untitledUI components

* fixed the import

* comment addressed

* fix: remove unnecessary span element from DeleteModal

* refactor: remove DeleteModal component and its associated files

* addressed gitar comment

* Added unit tests

* MINOR: Informix Schema update (open-metadata#26483)

* MINOR: Informix Schema update

* fix

* Update generated TypeScript types

* Add supportdatabase

* Update generated TypeScript types

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>

* ui: make search placeholder in service main tabs dynamic (open-metadata#26511)

* Revert "feat: migrate DeleteModal from MUI to untitledUI components (open-metadata#26461)" (open-metadata#26519)

This reverts commit 8327d22.

* Revert "refactor: update toggle component and related fields to use UT_SWITCH…" (open-metadata#26520)

This reverts commit 59fe777.

* fix(ui): Test suite add-test-case modal and table UX (open-metadata#26512)

* fix(ui): Test suite add-test-case modal and table UX

* fix footer padding

* Add eslint-plugin-playwright enforcement with CI check (open-metadata#26494)

* Add eslint-plugin-playwright enforcement with CI check

Add eslint-plugin-playwright to catch common Playwright anti-patterns
automatically. 13 rules configured in two tiers:

- Error (blocks CI): no-networkidle, no-page-pause, no-focused-test
- Warn (tracks debt): missing-playwright-await, no-wait-for-timeout,
  no-force-option, no-element-handle, no-eval, no-skipped-test,
  prefer-web-first-assertions, no-useless-await, no-wait-for-selector,
  valid-expect

Changes:
- Install eslint-plugin-playwright, configure rules in eslint.config.mjs
- Add yarn lint:playwright script
- Repurpose ui-checkstyle.yml workflow to run Playwright lint on PRs
- Fix last networkidle usage in ClassificationConditionalRendering
- Remove stale eslint-disable comments for undefined rules
- Update PLAYWRIGHT_DEVELOPER_HANDBOOK with ESLint Enforcement section
- Update playwright, writing-playwright-tests, and playwright-validation
  skills to reference lint check

Current: 0 errors, 1657 warnings (CI passes cleanly)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix CodeQL cache poisoning alert: use pull_request instead of pull_request_target

Switch from pull_request_target to pull_request since this workflow
only runs yarn lint:playwright — no secrets or write permissions needed.
This eliminates the untrusted code execution risk flagged by CodeQL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix all 1657 Playwright ESLint warnings across 190 files

Resolve every playwright lint rule violation to enforce test quality:
- no-wait-for-selector (1290): replace page.waitForSelector() with locator.waitFor()
- no-wait-for-timeout (130): replace hardcoded waits with event-driven alternatives
- prefer-web-first-assertions (61): use toHaveText/toBeVisible/toHaveValue
- no-force-option (51): remove { force: true } bypassing actionability checks
- missing-playwright-await (40): add await to fire-and-forget assertions
- no-skipped-test (28): acknowledge skipped tests with eslint-disable reasons
- no-eval (11): replace page.$eval with locator APIs
- no-element-handle (10): replace page.$() with page.locator()
- no-useless-await (8): remove await from synchronous operations
- valid-expect (4): add matchers to bare expect() calls
- no-networkidle (2): replace networkidle with domcontentloaded

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove unnecessary ANTLR install and add --frozen-lockfile to CI

- Remove Install Antlr4 CLI step from playwright-lint job (not needed
  for ESLint, saves 10-30s and avoids external network dependency)
- Add --frozen-lockfile to yarn install for reproducible CI builds

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore ANTLR CLI install — required by yarn postinstall script

yarn install triggers build-check → js-antlr which needs antlr4 CLI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix trivially-true assertion in special character search test

Remove `(await tableRows.count()) >= 0` which is always true since
count() never returns negative. The assertion now properly validates
that either the table or empty state is visible after searching.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violations: add .first() to multi-element waitFor calls

The no-wait-for-selector conversion (waitForSelector → locator.waitFor)
introduced strict mode violations. Playwright's Locator API throws when
waitFor() matches multiple elements, unlike the old waitForSelector API.

Key patterns fixed:
- getByTestId('loader').waitFor() — multiple loaders on page
- getByTestId('select-owner-tabs').getByTestId('loader') — tab loaders
- locator('.ant-skeleton-active/content').waitFor() — multiple skeletons
- locator('table/thead th').waitFor() — multiple tables
- getByTestId('side-panel-classification') — multiple panels
- locator('.ant-select-dropdown:visible') — multiple dropdowns
- locator('.ant-popover').waitFor() — multiple popovers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix strict mode violation in incidentManager assignee/owner-link

The getByTestId('assignee').getByTestId('owner-link') locator resolves
to 15 elements in the incident manager table (one per row). Adding
.first() matches the original waitForSelector semantics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix test failures from lint conversions: proper waits and retry logic

- GlossaryPermissions: add waitForAllLoadersToDisappear after page nav
- GlossaryP3Tests: restore waits for special char search + error state
- GlossaryStatusFilterLargeDataset: restore waitForTimeout for filter
  state settling (no reliable DOM element to wait for)
- AutoPilot: use waitForAllLoadersToDisappear instead of .first() to
  ensure ALL loaders are gone, increase banner timeout to 60s
- importUtils: use expect().toHaveCount(0) for scoped multi-element
  loader/skeleton waits instead of waitForSelector or .first()
- ColumnBulkOperations: increase response timeout, add retry polling
  for empty state verification
- DomainDataProductsWidgets: add poll-based wait for DOM update after
  asset removal instead of removed waitForTimeout
- customizeLandingPage: scope widget click to dialog to avoid matching
  stale elements on the page behind the modal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restore force:true where genuinely needed with eslint-disable comments

The no-force-option ESLint rule removal broke tests where force was required
for Ant Select comboboxes (selected item overlay covers input), popover triggers
(partially obstructed by animation), data grid buttons (covered by overlay),
and drag-and-drop (row hover overlays). Each restoration includes an
eslint-disable-line comment explaining why force is necessary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Stabilize 4 flaky tests with retry wrappers and proper waits

- DataContractsSemanticRules: Wrap reload + status assertions in
  expect().toPass() to handle async backend contract validation
- DataContracts: Increase timeout on dynamically rendered row filter
  and column text assertions that contain UUID-suffixed names
- Customproperties-part2: Wrap user search-and-select loop in
  expect().toPass() retry to handle search dropdown rendering races
- UserProfileOnlineStatus: Add profile render wait, navigation
  completion guard, and increased badge visibility timeout

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Replace .first().waitFor loader pattern with waitForAllLoadersToDisappear

The pattern `page.getByTestId('loader').first().waitFor({ state: 'detached' })`
only waits for the FIRST loader element to detach. When multiple loaders exist
(nested components, tabs, popovers), this causes tests to proceed before the
page is fully loaded — the root cause of ~18 flaky tests.

Replaced ~500 occurrences across 103 files with the correct
`waitForAllLoadersToDisappear(page)` which uses `expect(loaders).toHaveCount(0)`
to wait for ALL loaders to disappear.

Scoped loader waits (e.g., within select-owner-tabs, test-case-container,
tags-container) are intentionally preserved since they correctly target a
specific container's single loader.

Also fixes UserProfileOnlineStatus.spec.ts:
- Added createOrFetchUser helper for idempotent user creation on retries
- Added afterAll cleanup for test users
- Replaced waitForURL with redirectToHomePage for navigation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>

* [Fix-25563] - Issue in Search Entity By Key  (open-metadata#26482)

* Fix Issue Entity Not found , entity missing from search

* Fix Tests

* Fix Broken relationship issue for upstreamEntityRelationship

* Add Exponential Retry

* Remove Entity Not found from query

* Address Reciew Comments

* Address more review

* Fix Missing Columns Index

* Fix Join add stats

* Column stats and Merged Main

* Add separate custom bulk processor for column

* Fix reindex job falsely killed after 1 hour by orphan monitor

  The OrphanJobMonitor uses job.updatedAt as a liveness signal, but
  updatedAt was only set during state transitions (READY, RUNNING),
  never refreshed during processing. After 10 minutes the job appeared
  orphaned; at the 1-hour recovery window it was force-failed.

  Fix: touch updatedAt alongside the lock refresh (every 60s) so the
  staleness check stays satisfied while the coordinator is alive.

* Set Failure callback for columnBulkProcessor

* fix review comments

* address review comments from claude

* Review Mocked tests

* Address review comments. Both fixed:

                           1. retryOnConflict(3) restored on all 4 call sites — updateEntity and upsertDocument in both ElasticSearchEntityManager and OpenSearchEntityManager. This handles shard-level
                           version conflicts independently from the 429 retry logic in SearchRetryUtil.
                           2. drainPendingColumnFutures race fixed in both bulk sinks — replaced iterate + clear() with poll() loop, which atomically removes each element from the deque so no
                           concurrently-added futures can be lost

* Get proper error message from Elastic and OpenSearch

* Fix Breaking Test

* MINOR: Port py-tests improvements to py-tests-postgres workflow (open-metadata#26517)

Apply the same structural improvements from the py-tests workflow:
- Add integration test sharding (shard-1/shard-2) for parallelism
- Replace `make run_python_tests` with nox integration-tests session
- Add explicit timeout-minutes (180) and descriptive job name
- Remove unnecessary fetch-depth: 0 from checkout
- Normalize indentation to 2-space with proper YAML style

* Use context to enhance recognition by default (open-metadata#25856)

* Implement custom context enhancement for Presidio recognizers

Presidio's default context enhancement relies heavily on NLP and often fails
when analyzing individual values rather than full text. This implements a
custom context enhancement that:

- Boosts recognizer scores to MAX when context keywords match
- Applies a minimum score threshold (0.3) before enhancement
- Skips already-enhanced results to prevent double-boosting
- Introduces a decorator pattern for composing recognizer enhancements
- Adds eager_us_bank_recognizer with higher base scores for better results

The enhancement works by checking if any context words from the recognizer
match the provided context list, then boosting the confidence score to
maximum and setting the IS_SCORE_ENHANCED_BY_CONTEXT_KEY metadata flag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Integrate context enhancement into recognizer factory

Updates PresidioRecognizerFactory to apply the new decorator pattern:

- All recognizers now use enhance_using_context decorator
- Confidence threshold filtering applied via filter_enhanced_results_below_threshold
- Decorators composed using decorate_recognizer for clean application
- Context passed to PatternRecognizer during creation

This ensures all enabled recognizers benefit from custom context enhancement
while maintaining backward compatibility with confidence thresholds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add migration to update PII tag recognizers with enhanced configs

Adds a v1.1.22 migration that updates existing PII tags with improved
recognizer configurations featuring context keywords and optimized patterns.

Changes:
- Add patchRecognizers method in CollectionDAO for updating tag recognizers
- Implement setRecognizersForSensitiveTags in MigrationUtil to load and apply
  recognizer configs from piiTagsWithRecognizers.json
- Update piiTagsWithRecognizers.json with context keywords for better
  classification accuracy
- Execute migration as post-DDL script for both MySQL and PostgreSQL

This migration ensures existing deployments benefit from the improved context
enhancement logic without manual reconfiguration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix potential NPE and type mismatch in migration utility

Addresses code review feedback:

1. Fix potential NullPointerException in setRecognizersForSensitiveTags
   - Use Boolean.TRUE.equals() instead of auto-unboxing for nullable Boolean
   - Prevents NPE when autoClassificationEnabled is absent from JSON
   - Follows existing pattern from v1120/MigrationUtil.java

2. Fix Boolean vs boolean type mismatch in updateTagRecognizers
   - Change isForceMigration parameter from boxed Boolean to primitive boolean
   - Matches caller signature and eliminates latent NPE risk
   - Maintains consistency across method signatures

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add context to NHSRecognizer

* Fix typing

* Fix broken unit tests

* Fix broken integration test

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Chore(UI): Remove tags customization option (open-metadata#26514)

* Add regenerate-bot-tokens operation for JWT key rotation (open-metadata#26477)

* Add regenerate-bot-tokens operation for JWT key rotation

Add a new `regenerate-bot-tokens` subcommand to OpenMetadataOperations
that regenerates JWT tokens for all bot users. This is needed when
rotating JWT signing keys or changing the cluster name (e.g., during
disaster recovery from POV to PROD).

The command:
- Iterates through all non-deleted bots with pagination
- Regenerates JWT tokens only for bots using JWT authentication
- Skips bots without associated users or non-JWT auth mechanisms
- Displays results in an ASCII table with status per bot

Closes open-metadata/openmetadata-collate#3157

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix review: add --expiry CLI parameter and null-guard paging

- Token expiry is now a CLI parameter (--expiry) defaulting to Unlimited,
  so operators can choose OneHour, One, Seven, Thirty, Sixty, Ninety, or
  Unlimited when regenerating
- Add null guard on getPaging() to prevent NPE on empty result sets

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Document regenerate-bot-tokens expiry values in help output

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Move bot token regeneration logic to server API

Move the token regeneration logic from the ops CLI into a new REST
endpoint PUT /v1/users/regenerateBotTokens (admin-only). The ops
command now calls the server API via HTTP, following the same pattern
as deploy-pipelines.

This keeps business logic in the server and the ops CLI as a thin
HTTP client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert server API approach, use direct DB access for token regeneration

After JWT key rotation, all existing bot tokens are invalid — including
the ingestion-bot token needed to authenticate against the server API.
This command must operate directly on the database to recover from
that state. Added a doc comment explaining the reasoning.

Also reverts the UserResource endpoint added in the previous commit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix compilation: use fully-qualified ResultList to match existing pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix botUser always null: fetch each bot individually via getByName

listAll/listAfter use setFieldsInBulk which only calls registered
fieldFetchers (tags, owners, etc.) — not the entity-specific setFields
override. BotRepository.setFields populates botUser via a relationship
lookup, but this is never called in bulk listing. Fetch each bot
individually with getByName which triggers setFields properly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add integration tests for regenerate-bot-tokens operation

Tests cover token regeneration producing new tokens, respecting
expiry parameters, and handling multiple bots independently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix flaky test: use different expiries to guarantee distinct tokens

Unlimited tokens generated in the same second produce identical JWTs
(same iat, null exp). Changed test to use different expiry values so
the tokens are guaranteed to differ.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* format

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* Fix open-metadata#2730: Add tempLineageTables field to lineage details (open-metadata#26487)

* Fix open-metadata#2730: Lineage Add Temp Node Support

* missing sql file

* Update generated TypeScript types

* Update generated TypeScript types

* address guitar commens

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Feature: Fabric's lineage, usage and profiler support (open-metadata#26373)

* Feature: Fabric's lineage, usage and profiler support

* Python Checkstyle

* Update generated TypeScript types

* fix: narrow Fabric fallback from except Exception to except ProgrammingError

* Addressed comments

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* chore(ui): Add Test Case list filters while creating a bundle suite (open-metadata#26293)

* fix: Add Test Case list filters (Status, Test type, Table, Column) and E2E coverage

- Add filters to AddTestCaseList: Status (single), Test type (single), Table (multi), Column (multi)
- Use SearchDropdown for all four filters with hideCounts
- Client-side filtering for Table/Column; API refetch for Status/Test type
- Add AddTestCaseListFilters component and constants
- Add AddTestCaseList.utils (getTableFilterOptions, getColumnFilterOptions, filterTestCasesByTableAndColumn, getSelectedOptionsFromKeys)
- Unit tests: AddTestCaseList.component.test.tsx (filter tests), AddTestCaseList.utils.test.ts
- E2E: TestSuite.spec.ts - flatten steps, verify/apply/clear filters in create flow

Fixes https://github.com/open-metadata/openmetadata-collate/issues/3045

Made-with: Cursor

* fix lint

* address gitar-bot comments

* fix sonar issues

* address gitar-bot comments and sonar issues

* minor fix

* fix gitar-bot comments

* use backend filter for table type

* feat: added columnName parameter

* feat: adde columnname parameter

* feat: added colunmName argument

* convert column filter to server side filter

* address gitar-bot comments

* minor fix

* fix failing test

* address sonar issues

* Revert yarn.lock changes in openmetadata-ui-core-components

Made-with: Cursor

* address comment

* show select all filter in all test case lists

* address copilot comments

* add hideTableFilter and columnFilter props

* address comments

* hide table filter for for add pipeline under table

* fix failing test

* fix failing tests

---------

Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>

* added unit tests for kafkaconnect pipeline

* Add gitar fix

* MINOR - Column name Length patch (open-metadata#26530)

* chore(bulkEndpoint): create bulk bundle suite body object

* patch(columnName): remove maxLength for column name

* patch(columnName): remove files from commit

* patch(columnName): remove unused code

* fix unit tests

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>

* update alert props (open-metadata#26544)

* Fix: flaky domain rename data products count assertion (open-metadata#26542)

* Fix: flaky domain rename data products count assertion

After a domain rename, the search index update for subdomain-level data
products can lag behind the DB update due to async reindexing by the
governance-bot. The `verifyDataProductsCount` helper was doing a one-shot
`textContent()` + `toBe()` assertion that failed when the search index
hadn't converged yet.

Changes:
- Update `verifyDataProductsCount` to accept optional `{ apiContext, domainFqn }`
  that uses `expect.poll()` to poll the data product search API until
  `hits.total.value` matches the expected count (following the same pattern
  used in glossary.ts, alert.ts, and other polling utilities)
- Switch from one-shot `toBe()` to auto-retrying `toHaveText()` for the UI check
- Pass polling options in the post-rename assertion in Domains.spec.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add HTTP status check in search API poll

Check response.ok() before parsing JSON in the expect.poll callback
to avoid silently treating 4xx/5xx errors as count=0.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove custom timeout from toHaveText assertion

The expect.poll already ensures the search index has converged before
the UI check, so the default Playwright timeout is sufficient.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Refactor InputOutputPortsTab to use Accordion components and update styles (open-metadata#26450)

* Refactor InputOutputPortsTab to use Accordion components and update styles

* addressed PR comment

* fixed playwright test

* addressed gitar comment and fix sonar issues

* fixed overflow issue

* fixed ui issues

* minor fix

* addressed UI fixes

* py_format_check

* Added test for table format without topic prefix

* Removed commented code

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Rohit Jain <60229265+Rohit0301@users.noreply.github.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Akash Verma <138790903+akashverma0786@users.noreply.github.com>
Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Harsh Vador <58542468+harsh-vador@users.noreply.github.com>
Co-authored-by: Harshit Shah <harshit.shah@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: IceS2 <pablo.takara@getcollate.io>
Co-authored-by: Eugenio <eugenio.donaque@getcollate.io>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: TeddyCr <teddy.crepineau@gmail.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Sid <30566406+siddhant1@users.noreply.github.com>
Co-authored-by: Siddhant <siddhant@MacBook-Pro-290.local>
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants