Skip to content

Conversation

david-leifker
Copy link
Collaborator

@david-leifker david-leifker commented Jul 15, 2025

Why?

The graph index growth is significantly increased when generating edges between queries and schemaField and dataset entities. These edges are not currently required in the product and their value in the graph index is not sufficient to warrant them at this time.

Changes

  • Removed graph relation for queries
  • Bulk delete implementation which runs as an async task
  • Cleanup step for query graph edges using bulk delete
  • Consolidated non-blocking upgrade beans

@github-actions github-actions bot added product PR or Issue related to the DataHub UI/UX devops PR or Issue related to DataHub backend & deployment labels Jul 15, 2025
Copy link

codecov bot commented Jul 15, 2025

❌ Unsupported file format

Upload processing failed due to unsupported file format. Please review the parser error message:
Error parsing JUnit XML in /home/runner/work/datahub/datahub/metadata-io/build/test-results/test/TEST-com.linkedin.metadata.graph.search.elasticsearch.SearchGraphServiceElasticSearchTest.xml at 683:1058

Caused by:
RuntimeError: Error converting computed name to ValidatedString

Caused by:
    string is too long</code></pre>

For more help, visit our troubleshooting guide.


:x: 2 Tests Failed:

Tests completed Failed Passed Skipped
6067 2 6065 71
View the full list of 2 :snowflake: flaky tests
tests.integration.powerbi.test_m_parser::test_parse_m_query9

Flake rate in main: 8.00% (Passed 23 times, Failed 2 times)

Stack Traces | 60s run time
@pytest.mark.integration
    def test_parse_m_query9():
        expression: str = M_QUERIES[8]
>       parse_tree: Tree = parser._parse_expression(expression)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.../integration/powerbi/test_m_parser.py:188: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../powerbi/m_query/parser.py:54: in _parse_expression
    parse_tree: Tree = lark_parser.parse(expression)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
venv/lib/python3.11.../site-packages/lark/lark.py:645: in parse
    return self.parser.parse(text, start=start, on_error=on_error)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
venv/lib/python3.11.../site-packages/lark/parser_frontends.py:96: in parse
    return self.parser.parse(stream, chosen_start, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
venv/lib/python3.11.../lark/parsers/earley.py:266: in parse
    to_scan = self._parse(lexer, columns, to_scan, start_symbol)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
venv/lib/python3.11.../lark/parsers/xearley.py:144: in _parse
    self.predict_and_complete(i, to_scan, columns, transitives)
venv/lib/python3.11.../lark/parsers/earley.py:154: in predict_and_complete
    elif new_item not in column:
         ^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = additive_expression ::= * multiplicative_expression NEWLINE AMPERSAND additive_expression (1194)
other = additive_expression ::= * multiplicative_expression NEWLINE AMPERSAND additive_expression (1194)

>   def __eq__(self, other):
E   stopit.utils.TimeoutException

venv/lib/python3.11.../lark/parsers/earley_common.py:28: TimeoutException
glossary sidebar navigation test cypress/e2e/glossaryV2/v2_glossary_navigation.js::cypress/e2e/glossaryV2/v2_glossary_navigation.js

Flake rate in main: 92.50% (Passed 3 times, Failed 37 times)

Stack Traces | 104s run time
2025-07-15T18:15:02.312Z
Timed out retrying after 10000ms: Expected to find content: 'CypressGlosssaryNavigationTerm' but never did.

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Copy link

alwaysmeticulous bot commented Jul 15, 2025

🔴 Meticulous spotted visual differences in 18 of 1453 screens tested: view and approve differences detected.

Meticulous evaluated ~8 hours of user flows against your PR.

Last updated for commit 716593d. This comment will update as new commits are pushed.

Copy link

codecov bot commented Jul 15, 2025

Bundle Report

Changes will decrease total bundle size by 174.46kB (-0.79%) ⬇️. This is within the configured threshold ✅

Detailed changes
Bundle name Size Change
datahub-react-web-esm 21.99MB -174.46kB (-0.79%) ⬇️

Affected Assets, Files, and Routes:

view changes for bundle: datahub-react-web-esm

Assets Changed:

Asset Name Size Change Total Size Change (%)
assets/index-*.js -164.35kB 18.36MB -0.89%
assets/index-*.css 479 bytes 609.33kB 0.08%
assets/web-*.es5.min-94f16f1b.js (New) 3.24kB 3.24kB 100.0% 🚀
assets/saclogo-*.svg (New) 2.03kB 2.03kB 100.0% 🚀
assets/add-*.svg (New) 2.01kB 2.01kB 100.0% 🚀
assets/lookerlogo-*.svg (New) 1.92kB 1.92kB 100.0% 🚀
assets/ermodelrelationshipIcon-*.svg (New) 1.83kB 1.83kB 100.0% 🚀
assets/editIconBlack-*.svg (New) 1.16kB 1.16kB 100.0% 🚀
assets/incident-*.svg (New) 628 bytes 628 bytes 100.0% 🚀
assets/incident-*.svg (New) 779 bytes 779 bytes 100.0% 🚀
assets/incident-*.svg (New) 989 bytes 989 bytes 100.0% 🚀
assets/incident-*.svg (New) 927 bytes 927 bytes 100.0% 🚀
assets/Arrow-*.svg (New) 794 bytes 794 bytes 100.0% 🚀
assets/shield-*.svg (New) 599 bytes 599 bytes 100.0% 🚀
assets/shield-*.svg (New) 714 bytes 714 bytes 100.0% 🚀
assets/presetlogo-*.svg (New) 543 bytes 543 bytes 100.0% 🚀
assets/restricted-*.svg (New) 421 bytes 421 bytes 100.0% 🚀
assets/close_dark-*.svg (New) 383 bytes 383 bytes 100.0% 🚀
assets/focus-*.svg (New) 266 bytes 266 bytes 100.0% 🚀
assets/sample-*.png (Deleted) -29.82kB 0 bytes -100.0% 🗑️

@datahub-cyborg datahub-cyborg bot added pending-submitter-response Issue/request has been reviewed but requires a response from the submitter and removed needs-review Label for PRs that need review from a maintainer. labels Jul 15, 2025
@datahub-cyborg datahub-cyborg bot added pending-submitter-merge and removed pending-submitter-response Issue/request has been reviewed but requires a response from the submitter labels Jul 15, 2025
* remove graph relation for queries
* bulk delete implementation
* cleanup step for query graph edges
@david-leifker david-leifker force-pushed the query-graph-edge-refactor branch from 716593d to 3093c15 Compare July 15, 2025 21:54
@david-leifker david-leifker merged commit f12f8e3 into master Jul 15, 2025
72 of 74 checks passed
@david-leifker david-leifker deleted the query-graph-edge-refactor branch July 15, 2025 22:40
kartikey-visa pushed a commit to kartikey-visa/datahub that referenced this pull request Jul 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devops PR or Issue related to DataHub backend & deployment pending-submitter-merge product PR or Issue related to the DataHub UI/UX

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants