Skip to content

perf(ui): deduplicate docStore persona requests via shared React Query key - #31300

Open
Rohit0301 wants to merge 5 commits into
mainfrom
duplicate-persona-onboarding-requests
Open

perf(ui): deduplicate docStore persona requests via shared React Query key#31300
Rohit0301 wants to merge 5 commits into
mainfrom
duplicate-persona-onboarding-requests

Conversation

@Rohit0301

Copy link
Copy Markdown
Contributor

Describe your changes:

I worked on eliminating three redundant identical GET requests to /api/v1/docStore/name/persona.* that fired on every /my-data navigation (543ms total wasted, all 404s) because MyDataPage and multiple useCustomPages consumers each fetched independently with no cache coordination.

Root cause: useCustomPages used a manual useState/useCallback/useEffect pattern, and MyDataPage had its own separate fetchDocument() + useEffect. Neither used React Query, so concurrent subscribers for the same persona FQN each fired an independent network request.

Fix:

  • Added rest/queries/docStoreQuery.ts — shared docStoreQueryKey(fqn) + docStoreQueryFn(fqn) following the existing tableQuery.ts pattern
  • Migrated useCustomPages from manual fetch to useQuery with the shared key; pageType filtering moves into the return value (no longer triggers a re-fetch on pageType change — it filters from the cached doc)
  • Replaced MyDataPage's fetchDocument() + useEffect with useQuery using the same key; layout and personaPreferences derived via useMemo

With React Query's in-flight deduplication, all concurrent subscribers to ['docStore', 'persona.X'] share exactly one network request.

Type of change:

  • Improvement

High-level design:

The existing rest/queries/ pattern (e.g. tableQuery.ts, dashboardQuery.ts) exports a canonical queryKey + queryFn pair so any consumer — detail page, sidebar widget, hover prefetch — hits the same normalised cache slot. docStoreQuery.ts adds the same plumbing for DocStore documents.

useCustomPages previously re-fetched the full persona document on every pageType change even though the document contains all page types. The new implementation fetches once per persona FQN and filters locally, reducing N fetches to 1 per persona.

MyDataPage previously had a duplicate fetch path independent of useCustomPages. Both now share the same React Query cache slot (['docStore', 'persona.X']), so on /my-data navigation the document is fetched exactly once regardless of how many consumers are mounted.

Tests:

Use cases covered

  • /my-data page loads with a persona selected — one GET to docStore/name/persona.* instead of three
  • Sidebar navigation (useSidebarItemsuseCustomPages('Navigation')) and page body share the cached response
  • Error case (404): navigation resets to [], customizedPage resets to null — same contract as before
  • Changing pageType between renders filters from the cache without a network round-trip
  • Switching selected personas triggers a new fetch for the new FQN

Unit tests

  • Updated useCustomPages.test.ts — wrapped with QueryClientProvider, updated "pageType changes" test to assert one fetch (not two), all other assertions preserved
  • Files updated: openmetadata-ui/src/main/resources/ui/src/hooks/useCustomPages.test.ts

Backend integration tests

  • Not applicable (no backend API changes).

Ingestion integration tests

  • Not applicable (no ingestion changes).

Playwright (UI) tests

  • Not applicable — no visible UI behaviour change; this is a network-layer deduplication fix.

Manual testing performed

  1. Navigate to /my-data with a persona assigned
  2. Open browser DevTools → Network tab, filter by docStore
  3. Confirm exactly 1 GET to /api/v1/docStore/name/persona.* fires (was 3 before this change)
  4. Verify page layout and sidebar navigation render correctly
  5. Switch personas — confirm a new request fires for the new FQN

UI screen recording / screenshots:

Not applicable — no visual change; the fix is a network deduplication at the data-fetching layer.

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>
  • My PR is linked to a GitHub issue via Fixes #<issue-number> above.
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed. — N/A
  • For UI changes: I attached a screen recording and/or screenshots above. — N/A (no visual change)
  • I have added tests (unit / integration / Playwright as applicable) and listed them above.

Three identical GET /api/v1/docStore/name/persona.* requests fired on
every /my-data navigation because MyDataPage and multiple useCustomPages
consumers each fetched independently with no cache coordination.

Introduce docStoreQuery.ts (shared queryKey + queryFn), migrate
useCustomPages to useQuery, and rewrite MyDataPage's manual fetch/effect
to useQuery with the same key. React Query's in-flight deduplication
collapses N concurrent subscribers to one network request.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@Rohit0301
Rohit0301 requested a review from a team as a code owner August 10, 2026 14:52
@github-actions

Copy link
Copy Markdown
Contributor

❌ PR checklist incomplete

This PR cannot be merged until the following are addressed on its linked issue:

  • No GitHub issue is linked. Link an issue in the Development section of the PR (or add Fixes #12345 to the description). For a same-org cross-repo issue, add Fixes open-metadata/<repo>#123 to the description.

The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically.

Maintainers can bypass this check by adding the skip-pr-checks label.

@Rohit0301 Rohit0301 self-assigned this Aug 10, 2026
@Rohit0301 Rohit0301 added the safe to test Add this label to run secure Github workflows on PRs label Aug 10, 2026
@github-actions github-actions Bot added the UI UI specific issues label Aug 10, 2026
…lper

Both useCustomPages and MyDataPage were independently building the
persona docStore FQN string. Extract to personaDocFqn() in
docStoreQuery.ts so the cache key derivation has a single definition
and consumers can't silently drift apart.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

@Rohit0301 Rohit0301 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call — addressed in dcb5759. Extracted the FQN construction to personaDocFqn() in docStoreQuery.ts so both useCustomPages and MyDataPage derive the cache key from a single definition. Both consumers now import and call personaDocFqn(selectedPersona) and the inline template literals are gone.

@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

🔴 Playwright Results — workflow failed

Validated commit 960fc5c33ca416c928315e17b042ea30c3dc28bd in Playwright run 31411042665, attempt 1.

✅ 876 passed · ❌ 1 failed · 🟡 1 flaky · ⏭️ 1 skipped · 🧰 0 lifecycle flaky

Performance

Blocking targets: ✅ met · Optimization targets: 🟡 in progress

Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting.

🕒 Full workflow signal wall (to summary) 58m 7s

⏱️ Max setup 3m 12s · max shard execution 19m 36s · max shard-job elapsed before upload 25m 2s · reporting 6s

🌐 198.87 requests/attempt · 2.44 app boots/UI scenario · 38.69% common-shard skew

Optimization targets still in progress:

  • Common shard skew was 38.69% (convergence target: at most 15%).
  • Application boot ratio was 2.44 per UI scenario (2244 boots / 921 scenarios; convergence target: at most 1).
Shard Passed Failed Flaky Skipped Lifecycle failed Lifecycle flaky
🟡 Shard chromium-01 137 0 1 0 0 0
✅ Shard chromium-02 134 0 0 0 0 0
🔴 Shard chromium-03 155 1 0 1 0 0
✅ Shard chromium-04 136 0 0 0 0 0
✅ Shard chromium-05 154 0 0 0 0 0
✅ Shard data-asset-rules-01 61 0 0 0 0 0
✅ Shard domain-isolation-01 14 0 0 0 0 0
✅ Shard global-state-01 34 0 0 0 0 0
✅ Shard import-export-01 9 0 0 0 0 0
✅ Shard ingestion-01 1 0 0 0 0 0
✅ Shard reindex-01 2 0 0 0 0 0
✅ Shard search-01 10 0 0 0 0 0
✅ Shard search-rbac-01 29 0 0 0 0 0

Genuine Failures (failed on all attempts)

Features/Glossary/GlossaryAdvancedOperations.spec.tsshould change domain on glossary (shard chromium-03)
�[31mTest timeout of 60000ms exceeded.�[39m
🟡 1 flaky test(s) (passed on retry)
  • Pages/Entity.spec.tsUser as Owner Add, Update and Remove (shard chromium-01, 1 retry)

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

useCustomPages now uses useQuery internally, so components that call it
require a QueryClientProvider ancestor. Fix each test with the
appropriate strategy:

- GlossaryV1, GlossaryDetails, LeftSidebar: add jest.mock for
  useCustomPages (same pattern used by 20+ other component tests)
- MyDataPage: add QueryClientProvider wrapper via a renderMyDataPage()
  helper + fresh QueryClient per test to avoid cache pollution

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gitar-bot

gitar-bot Bot commented Aug 10, 2026

Copy link
Copy Markdown
Code Review ✅ Approved 1 resolved / 1 findings

Centralizes persona FQN construction in a shared helper and migrates docStore requests to React Query to eliminate duplicate network calls, addressing the duplicate persona FQN construction finding.

✅ 1 resolved
Quality: Persona FQN construction duplicated across two consumers

📄 openmetadata-ui/src/main/resources/ui/src/pages/MyDataPage/MyDataPage.component.tsx:88-90 📄 openmetadata-ui/src/main/resources/ui/src/hooks/useCustomPages.ts:26-28 📄 openmetadata-ui/src/main/resources/ui/src/rest/queries/docStoreQuery.ts:28-31
Both MyDataPage.component.tsx (lines 88-90) and useCustomPages.ts (lines 26-28) rebuild the persona FQN with the identical ${EntityType.PERSONA}${FQN_SEPARATOR_CHAR}${...} expression. Since the whole point of docStoreQuery.ts is a single normalized cache slot, the key derivation should also be centralized there (e.g. a personaDocFqn(persona) helper) so both consumers can't drift apart and produce mismatched keys that would silently break the intended request deduplication.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ UI Checkstyle passed — lint findings in changed files

🔍 ESLint findings in this PR's files — 0 error(s), 30 warning(s)

Errors block the build. Warnings do not yet — they are rules whose backlog is still
being worked down, listed so this PR does not add to it. See docs/ui-code-quality-gate.md.

0 error(s), 30 warning(s) across 5 changed file(s).

Count Rule
16 i18next/no-literal-string
10 sonarjs/no-duplicate-string
1 jsx-a11y/anchor-is-valid
1 react-hooks/exhaustive-deps
1 jsx-a11y/click-events-have-key-events
1 jsx-a11y/no-static-element-interactions
All findings
Location Rule Message
🟡 src/components/Glossary/GlossaryDetails/GlossaryDetails.test.tsx:24:39 i18next/no-literal-string disallow literal string:

GlossaryTermTab.component

🟡 src/components/Glossary/GlossaryDetails/GlossaryDetails.test.tsx:27:39 i18next/no-literal-string disallow literal string:

GlossaryHeader.component

🟡 src/components/Glossary/GlossaryDetails/GlossaryDetails.test.tsx:47:36 i18next/no-literal-string disallow literal string:

testActivityFeedTab

🟡 src/components/Glossary/GlossaryDetails/GlossaryDetails.test.tsx:52:43 i18next/no-literal-string disallow literal string:
Description
🟡 src/components/Glossary/GlossaryDetails/GlossaryDetails.test.tsx:95:55 i18next/no-literal-string disallow literal string:
GenericTab
🟡 src/components/Glossary/GlossaryV1.test.tsx:77:56 jsx-a11y/anchor-is-valid The href attribute is required for an anchor to be keyboard accessible. Provide a valid, navigable address as the href value. If you cannot provide an href, but
🟡 src/components/Glossary/GlossaryV1.test.tsx:83:38 i18next/no-literal-string disallow literal string: <>Glossary-Details component</>
🟡 src/components/Glossary/GlossaryV1.test.tsx:87:38 i18next/no-literal-string disallow literal string: <>Glossary-Term component</>
🟡 src/components/Glossary/GlossaryV1.test.tsx:91:38 i18next/no-literal-string disallow literal string: <>TitleBreadcrumb</>
🟡 src/components/Glossary/GlossaryV1.test.tsx:95:34 i18next/no-literal-string disallow literal string:
Breadcrumb
🟡 src/components/Glossary/GlossaryV1.test.tsx:103:39 i18next/no-literal-string disallow literal string:

FeedEditor

🟡 src/hooks/useCustomPages.test.ts:48:25 sonarjs/no-duplicate-string Define a constant instead of duplicating this literal 3 times.
🟡 src/hooks/useCustomPages.test.ts:97:55 sonarjs/no-duplicate-string Define a constant instead of duplicating this literal 3 times.
🟡 src/pages/MyDataPage/MyDataPage.component.tsx:169:6 react-hooks/exhaustive-deps React Hook useEffect has missing dependencies: 'isWelcomeVisible', 'updateWelcomeScreen', and 'usernameExistsInCookie'. Either include them or remove the depend
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:49:50 i18next/no-literal-string disallow literal string:
Loader
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:61:50 i18next/no-literal-string disallow literal string:
MyDataPageSkeleton
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:82:9 jsx-a11y/click-events-have-key-events Visible, non-interactive elements with click handlers must have at least one keyboard listener.
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:82:9 jsx-a11y/no-static-element-interactions Avoid non-native interactive elements. If using native HTML is not possible, add an appropriate role and support for tabbing, mouse, keyboard, and touch inputs
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:82:32 i18next/no-literal-string disallow literal string:
WelcomeScreen
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:93:58 i18next/no-literal-string disallow literal string:
CustomiseLandingPageHeader
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:153:11 sonarjs/no-duplicate-string Define a constant instead of duplicating this literal 7 times.
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:171:45 i18next/no-literal-string disallow literal string: <>LimitWrapper{children}</>
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:202:49 i18next/no-literal-string disallow literal string:
Link
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:271:26 sonarjs/no-duplicate-string Define a constant instead of duplicating this literal 6 times.
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:288:31 sonarjs/no-duplicate-string Define a constant instead of duplicating this literal 4 times.
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:291:31 sonarjs/no-duplicate-string Define a constant instead of duplicating this literal 4 times.
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:298:33 sonarjs/no-duplicate-string Define a constant instead of duplicating this literal 4 times.
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:299:33 sonarjs/no-duplicate-string Define a constant instead of duplicating this literal 4 times.
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:300:33 sonarjs/no-duplicate-string Define a constant instead of duplicating this literal 4 times.
🟡 src/pages/MyDataPage/MyDataPage.test.tsx:363:31 sonarjs/no-duplicate-string Define a constant instead of duplicating this literal 5 times.

Fix locally (fast - only checks files changed in this branch):

make ui-checkstyle-changed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs UI UI specific issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant