Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BugFix]support table function on compute nodes #34577

Conversation

abc982627271
Copy link
Contributor

@abc982627271 abc982627271 commented Nov 8, 2023

Fixes #issue

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.2
    • 3.1
    • 3.0
    • 2.5

Signed-off-by: abc982627271 <liuxuefen@starrocks.com>

Collections.shuffle(nodeIds);
ComputeNode node = GlobalStateMgr.getCurrentSystemInfo().getBackendOrComputeNode(nodeIds.get(0));
address = new TNetworkAddress(node.getHost(), node.getBrpcPort());

PGetFileSchemaResult result;
try {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code reviews can vary in depth and focus depending on the context of the code change and what the reviewers are looking for. Here’s a summary of potential suggestions for this particular diff snippet review:

  1. Consistent Naming:

    • The renaming from Backend to ComputeNode might be implying that Backend has been either replaced or extended by ComputeNode. It's crucial to verify everywhere in the codebase that this change is consistently applied and documented if it is part of a larger refactoring.
  2. Exception Message Clarity:

    • The new exception message "Failed to send proxy request. No alive backends" could potentially be made more clear, as you're not just looking for "backends" anymore due to the shared-nothing addition but also "compute nodes." Adjusting the messages to reflect the current state of the system could reduce debugging time in the future.
  3. Feature Toggles or Runtime Configuration:

    • Adding conditional logic based on RunMode.getCurrentRunMode() seems like it might be introducing runtime configuration control flow, which can be both powerful and risky. Ensure there is enough test coverage for both branches (shared-nothing and otherwise) and that these runtime changes are well-documented.
  4. Error Handling and Retry Logic:

    • The adjustment adds new exception paths which could benefit from more sophisticated error handling. Depending on how critical this getFileSchema function is within the system, adding retry logic or better fallbacks might be worthwhile.
  5. Thread Safety and Concurrency:

    • Collections.shuffle(nodeIds); implies a mutation of the nodeIds list. If nodeIds is shared across threads, this operation could introduce race conditions. Ensure that either nodeIds is confined to a single thread or is otherwise safely published between threads.
  6. Code Duplication:

    • There appears to be some code duplication between the cases for handling when nodeIds is empty and when looking for backends or compute nodes. If similar logic is used in multiple places, consider abstracting it into a method to promote reuse and simplify future updates.
  7. Unit Testing:

    • With logic being changed/added (especially around conditions like RunMode.getCurrentRunMode()), unit tests need to be updated or added to cover the new scenarios to ensure that the behavior is as expected.
  8. Performance Implications:

    • By adding additional backend and compute node checks (as well as potentially making requests to each), there can be performance implications. Verify that this does not significantly impact the performance, especially in large deployments.
  9. Documentation and Comments:

    • There aren't any comments in the provided diff. Contextual comments explaining why decisions were made (such as the introduction of different node types and run modes) can be incredibly valuable, particularly when unusual or non-obvious logic is introduced.
  10. Dependency on Global State:

    • The overall logic relies heavily on the global state (GlobalStateMgr). Be aware that such reliance makes unit testing more difficult and may increase coupling, which can lead to issues with maintainability.

Without the broader context or understanding of prior and future pieces of work, these are top-level suggestions. Changes connected with databases, distributed systems, or resilient service design typically involve deeper considerations, which would be reviewed as part of a more thorough code base analysis.

@sduzh sduzh requested review from kevincai, wyb and sduzh November 9, 2023 02:07
Signed-off-by: abc982627271 <liuxuefen@starrocks.com>
sduzh
sduzh previously approved these changes Nov 9, 2023
Signed-off-by: abc982627271 <liuxuefen@starrocks.com>
Signed-off-by: abc982627271 <liuxuefen@starrocks.com>
Signed-off-by: abc982627271 <liuxuefen@starrocks.com>
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

warning The version of Java (11.0.21) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more here

Copy link

[BE Incremental Coverage Report]

pass : 0 / 0 (0%)

Copy link

[FE Incremental Coverage Report]

pass : 10 / 10 (100.00%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 com/starrocks/catalog/TableFunctionTable.java 10 10 100.00% []

@sduzh sduzh merged commit 24a6d0e into StarRocks:main Nov 14, 2023
Copy link

@Mergifyio backport branch-3.2

@github-actions github-actions bot removed the 3.2 label Nov 14, 2023
Copy link

@Mergifyio backport branch-3.1

@github-actions github-actions bot removed the 3.1 label Nov 14, 2023
Copy link
Contributor

mergify bot commented Nov 14, 2023

backport branch-3.2

✅ Backports have been created

Copy link
Contributor

mergify bot commented Nov 14, 2023

backport branch-3.1

✅ Backports have been created

mergify bot pushed a commit that referenced this pull request Nov 14, 2023
Signed-off-by: abc982627271 <liuxuefen@starrocks.com>
(cherry picked from commit 24a6d0e)
mergify bot pushed a commit that referenced this pull request Nov 14, 2023
Signed-off-by: abc982627271 <liuxuefen@starrocks.com>
(cherry picked from commit 24a6d0e)

# Conflicts:
#	fe/fe-core/src/test/java/com/starrocks/catalog/TableFunctionTableTest.java
abc982627271 added a commit to abc982627271/starrocks that referenced this pull request Nov 14, 2023
Signed-off-by: abc982627271 <liuxuefen@starrocks.com>
wanpengfei-git pushed a commit that referenced this pull request Nov 14, 2023
Signed-off-by: abc982627271 <liuxuefen@starrocks.com>
(cherry picked from commit 24a6d0e)
abc982627271 added a commit to abc982627271/starrocks that referenced this pull request Nov 14, 2023
Signed-off-by: abc982627271 <liuxuefen@starrocks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants