Skip to content

Sync EvaluteExpr execution #11801

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 21, 2024
Merged

Conversation

Hor911
Copy link
Collaborator

@Hor911 Hor911 commented Nov 20, 2024

During query compilation special subroutine called "Expression Evaluation" may be triggered (most common case is Constant Folding in query optimization phase).

It was implemented with async API and future waiting. In some rare cases (erros or failures) future was not completed and compilation hangs forever

This path removes future from call stack and replaces it with direct sync call. It requires change of internal API, it is one of the reason why this pattern was not choosed at the time of the development.

The fix was checked with load test (enabled in this PR) and by @dcherednik in manual runs.

Copy link

github-actions bot commented Nov 20, 2024

2024-11-20 14:22:46 UTC Pre-commit check linux-x86_64-release-asan for 4a8e559 has started.
2024-11-20 14:22:58 UTC Artifacts will be uploaded here
2024-11-20 14:26:08 UTC ya make is running...
🟡 2024-11-20 15:40:41 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
9114 9045 0 18 4 47

🟢 2024-11-20 15:41:29 UTC Build successful.
🟢 2024-11-20 15:41:58 UTC ydbd size 4.9 GiB changed* by -44.3 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: 2dd1cac merge: 4a8e559 diff diff %
ydbd size 5 266 044 608 Bytes 5 265 999 200 Bytes -44.3 KiB -0.001%
ydbd stripped size 1 351 951 824 Bytes 1 351 935 408 Bytes -16.0 KiB -0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Nov 20, 2024

2024-11-20 14:23:47 UTC Pre-commit check linux-x86_64-relwithdebinfo for 4a8e559 has started.
2024-11-20 14:23:50 UTC Artifacts will be uploaded here
2024-11-20 14:26:52 UTC ya make is running...
🟢 2024-11-20 15:31:03 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40495 34826 0 0 5558 111

🟢 2024-11-20 15:34:25 UTC Build successful.
🟢 2024-11-20 15:34:41 UTC ydbd size 2.5 GiB changed* by -41.4 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: 1c15035 merge: 4a8e559 diff diff %
ydbd size 2 686 799 328 Bytes 2 686 756 960 Bytes -41.4 KiB -0.002%
ydbd stripped size 481 350 576 Bytes 481 348 464 Bytes -2.1 KiB -0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

dcherednik
dcherednik previously approved these changes Nov 20, 2024
Copy link

github-actions bot commented Nov 20, 2024

2024-11-20 20:29:18 UTC Pre-commit check linux-x86_64-relwithdebinfo for a26f1ef has started.
2024-11-20 20:29:50 UTC Artifacts will be uploaded here
2024-11-20 20:33:14 UTC ya make is running...
🟡 2024-11-20 21:30:50 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40496 34825 0 1 5558 112

2024-11-20 21:34:15 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-11-20 21:45:46 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
111 (only retried tests) 5 0 0 0 106

🟢 2024-11-20 21:45:53 UTC Build successful.
🟢 2024-11-20 21:46:10 UTC ydbd size 2.5 GiB changed* by -37.5 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: f76323e merge: a26f1ef diff diff %
ydbd size 2 689 511 440 Bytes 2 689 473 056 Bytes -37.5 KiB -0.001%
ydbd stripped size 482 297 296 Bytes 482 291 024 Bytes -6.1 KiB -0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Nov 20, 2024

2024-11-20 20:31:27 UTC Pre-commit check linux-x86_64-release-asan for a26f1ef has started.
2024-11-20 20:31:38 UTC Artifacts will be uploaded here
2024-11-20 20:34:37 UTC ya make is running...
🟡 2024-11-20 21:39:50 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
9114 9048 0 12 7 47

🟢 2024-11-20 21:40:38 UTC Build successful.
🟢 2024-11-20 21:41:04 UTC ydbd size 4.9 GiB changed* by -44.4 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: f76323e merge: a26f1ef diff diff %
ydbd size 5 280 000 688 Bytes 5 279 955 248 Bytes -44.4 KiB -0.001%
ydbd stripped size 1 358 852 080 Bytes 1 358 835 664 Bytes -16.0 KiB -0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@Hor911 Hor911 merged commit 1b00f6a into ydb-platform:main Nov 21, 2024
10 checks passed
dahbka-lis pushed a commit to dahbka-lis/ydb that referenced this pull request Nov 25, 2024
dahbka-lis pushed a commit to dahbka-lis/ydb that referenced this pull request Nov 25, 2024
Hor911 added a commit to Hor911/ydb that referenced this pull request Feb 11, 2025
Hor911 added a commit to Hor911/ydb that referenced this pull request Feb 11, 2025
Hor911 added a commit to Hor911/ydb that referenced this pull request Feb 11, 2025
gridnevvvit pushed a commit that referenced this pull request Feb 14, 2025
gridnevvvit pushed a commit that referenced this pull request Feb 14, 2025
gridnevvvit pushed a commit that referenced this pull request Feb 14, 2025
gridnevvvit pushed a commit to gridnevvvit/ydb that referenced this pull request Feb 20, 2025
gridnevvvit added a commit that referenced this pull request Feb 20, 2025
Co-authored-by: Hor911 <hor911@ydb.tech>
zinal pushed a commit to zinal/ydb that referenced this pull request Mar 13, 2025
zinal pushed a commit to zinal/ydb that referenced this pull request Jun 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants