Skip to content

Fix usage single UDF object from many threads in DQ. #7436

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 17 commits into from
Aug 7, 2024
Merged

Fix usage single UDF object from many threads in DQ. #7436

merged 17 commits into from
Aug 7, 2024

Conversation

@Tony-Romanov
Copy link
Collaborator

@Tony-Romanov Tony-Romanov commented Aug 3, 2024

No description provided.

@Tony-Romanov Tony-Romanov added area/yql YQL query language issues bugfix labels Aug 3, 2024
@github-actions github-actions bot removed the bugfix label Aug 3, 2024
@github-actions
Copy link

github-actions bot commented Aug 3, 2024 •
edited
Loading

⚪ 2024-08-03 08:01:20 UTC Pre-commit check for eb0b994 has started.
⚪ 2024-08-03 08:04:34 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-03 08:32:56 UTC Build successful.

@github-actions
Copy link

github-actions bot commented Aug 3, 2024 •
edited
Loading

⚪ 2024-08-03 08:01:32 UTC Pre-commit check for eb0b994 has started.
⚪ 2024-08-03 08:04:53 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-08-03 10:56:46 UTC Too many tests failed, NOT going to retry
🔴 2024-08-03 11:06:56 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76425 60226 0 2598 13588 13

🟢 2024-08-03 11:17:02 UTC Build successful.
🟡 2024-08-03 11:17:55 UTC ydbd size 8.1 GiB changed* by +219.1 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 35bc65d merge: eb0b994 diff diff %
ydbd size 8 664 247 152 Bytes 8 664 471 512 Bytes +219.1 KiB +0.003%
ydbd stripped size 471 417 864 Bytes 471 442 824 Bytes +24.4 KiB +0.005%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Aug 3, 2024 •
edited
Loading

⚪ 2024-08-03 08:01:37 UTC Pre-commit check for eb0b994 has started.
⚪ 2024-08-03 08:04:53 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-03 10:36:12 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14926 14569 0 313 31 13

🟢 2024-08-03 10:37:54 UTC Build successful.
🟡 2024-08-03 10:38:28 UTC ydbd size 5.4 GiB changed* by +268.9 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 35bc65d merge: eb0b994 diff diff %
ydbd size 5 824 733 680 Bytes 5 825 009 008 Bytes +268.9 KiB +0.005%
ydbd stripped size 1 463 037 136 Bytes 1 463 127 696 Bytes +88.4 KiB +0.006%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Aug 3, 2024 •
edited
Loading

⚪ 2024-08-03 12:36:21 UTC Pre-commit check for 2df36e4 has started.
⚪ 2024-08-03 12:40:02 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-03 14:57:42 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14923 14571 0 305 32 15

🟢 2024-08-03 14:59:26 UTC Build successful.
🟡 2024-08-03 14:59:59 UTC ydbd size 5.4 GiB changed* by +268.9 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: e9534cc merge: 2df36e4 diff diff %
ydbd size 5 824 733 680 Bytes 5 825 009 008 Bytes +268.9 KiB +0.005%
ydbd stripped size 1 463 037 136 Bytes 1 463 127 696 Bytes +88.4 KiB +0.006%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Aug 3, 2024 •
edited
Loading

⚪ 2024-08-03 12:38:05 UTC Pre-commit check for 2df36e4 has started.
⚪ 2024-08-03 12:41:23 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-08-03 15:31:53 UTC Too many tests failed, NOT going to retry
🔴 2024-08-03 15:38:42 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76425 60221 0 2602 13588 14

🟢 2024-08-03 15:48:49 UTC Build successful.
🟡 2024-08-03 15:49:36 UTC ydbd size 8.1 GiB changed* by +219.1 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: e9534cc merge: 2df36e4 diff diff %
ydbd size 8 664 247 152 Bytes 8 664 471 512 Bytes +219.1 KiB +0.003%
ydbd stripped size 471 417 864 Bytes 471 442 824 Bytes +24.4 KiB +0.005%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Aug 3, 2024 •
edited
Loading

⚪ 2024-08-03 12:38:34 UTC Pre-commit check for 2df36e4 has started.
⚪ 2024-08-03 12:41:55 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-03 13:13:05 UTC Build successful.

@github-actions
Copy link

github-actions bot commented Aug 5, 2024 •
edited
Loading

⚪ 2024-08-05 10:28:59 UTC Pre-commit check for 9b0435d has started.
⚪ 2024-08-05 10:32:18 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-05 10:40:25 UTC Build failed. see the logs.

@github-actions
Copy link

github-actions bot commented Aug 5, 2024 •
edited
Loading

⚪ 2024-08-05 10:29:05 UTC Pre-commit check for 9b0435d has started.
⚪ 2024-08-05 10:32:22 UTC Check linux-x86_64-release-clang14 is running...
🔴 2024-08-05 10:43:30 UTC Build failed. see the logs.

@github-actions
Copy link

github-actions bot commented Aug 5, 2024 •
edited
Loading

⚪ 2024-08-05 10:29:05 UTC Pre-commit check for 9b0435d has started.
⚪ 2024-08-05 10:32:16 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-08-05 10:38:43 UTC Build failed. see the logs.

@github-actions
Copy link

github-actions bot commented Aug 5, 2024 •
edited
Loading

⚪ 2024-08-05 12:20:08 UTC Pre-commit check for 6829cec has started.
⚪ 2024-08-05 12:23:24 UTC Check linux-x86_64-release-clang14 is running...
🔴 2024-08-05 12:27:00 UTC Build failed. see the logs.

@github-actions
Copy link

github-actions bot commented Aug 5, 2024 •
edited
Loading

⚪ 2024-08-05 12:20:41 UTC Pre-commit check for 6829cec has started.
⚪ 2024-08-05 12:23:51 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-05 12:29:04 UTC Build failed. see the logs.

@github-actions
Copy link

github-actions bot commented Aug 5, 2024 •
edited
Loading

⚪ 2024-08-05 12:20:44 UTC Pre-commit check for 6829cec has started.
⚪ 2024-08-05 12:23:54 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-08-05 12:30:55 UTC Build failed. see the logs.

@github-actions
Copy link

github-actions bot commented Aug 5, 2024 •
edited
Loading

⚪ 2024-08-05 13:45:17 UTC Pre-commit check for 8430335 has started.
⚪ 2024-08-05 13:48:31 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-05 16:34:26 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14932 14825 0 54 37 16

🟢 2024-08-05 16:36:05 UTC Build successful.
🟡 2024-08-05 16:36:36 UTC ydbd size 5.4 GiB changed* by +268.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: e4c34ae merge: 8430335 diff diff %
ydbd size 5 828 208 184 Bytes 5 828 483 368 Bytes +268.7 KiB +0.005%
ydbd stripped size 1 463 788 400 Bytes 1 463 878 960 Bytes +88.4 KiB +0.006%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Aug 5, 2024 •
edited
Loading

⚪ 2024-08-05 13:45:32 UTC Pre-commit check for 8430335 has started.
⚪ 2024-08-05 13:48:44 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-08-05 16:41:18 UTC Too many tests failed, NOT going to retry
🔴 2024-08-05 16:44:58 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76431 62241 0 586 13589 15

🟢 2024-08-05 16:55:21 UTC Build successful.
🟡 2024-08-05 16:56:03 UTC ydbd size 8.1 GiB changed* by +225.5 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: e4c34ae merge: 8430335 diff diff %
ydbd size 8 668 415 856 Bytes 8 668 646 720 Bytes +225.5 KiB +0.003%
ydbd stripped size 471 630 824 Bytes 471 660 072 Bytes +28.6 KiB +0.006%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Aug 5, 2024 •
edited
Loading

⚪ 2024-08-05 13:47:25 UTC Pre-commit check for 8430335 has started.
⚪ 2024-08-05 13:51:12 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-05 14:38:22 UTC Build successful.

@github-actions
Copy link

github-actions bot commented Aug 6, 2024

⚪ 2024-08-06 05:47:29 UTC Pre-commit check for 9acf5e7 has started.

@github-actions
Copy link

github-actions bot commented Aug 6, 2024

⚪ 2024-08-06 05:47:33 UTC Pre-commit check for 9acf5e7 has started.

@github-actions
Copy link

github-actions bot commented Aug 6, 2024 •
edited
Loading

⚪ 2024-08-06 07:12:34 UTC Pre-commit check for a2f7c7e has started.
⚪ 2024-08-06 07:15:53 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-06 09:53:41 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76429 62770 0 57 13589 13

⚪ 2024-08-06 10:03:21 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-06 10:10:52 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
74 (only retried tests) 12 0 54 0 8

⚪ 2024-08-06 10:17:58 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-06 10:26:15 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
63 (only retried tests) 1 0 54 0 8

🟢 2024-08-06 10:33:36 UTC Build successful.
🟡 2024-08-06 10:34:16 UTC ydbd size 8.1 GiB changed* by +222.4 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 99add96 merge: a2f7c7e diff diff %
ydbd size 8 671 677 264 Bytes 8 671 905 032 Bytes +222.4 KiB +0.003%
ydbd stripped size 471 804 296 Bytes 471 829 512 Bytes +24.6 KiB +0.005%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Aug 6, 2024 •
edited
Loading

⚪ 2024-08-06 07:14:44 UTC Pre-commit check for a2f7c7e has started.
⚪ 2024-08-06 07:17:56 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-06 07:43:41 UTC Build successful.

@github-actions
Copy link

github-actions bot commented Aug 6, 2024 •
edited
Loading

⚪ 2024-08-06 07:14:53 UTC Pre-commit check for a2f7c7e has started.
⚪ 2024-08-06 07:18:14 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-06 09:55:54 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14931 14873 0 8 33 17

🟢 2024-08-06 09:57:36 UTC Build successful.
🟡 2024-08-06 09:58:23 UTC ydbd size 5.4 GiB changed* by +268.3 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 99add96 merge: a2f7c7e diff diff %
ydbd size 5 830 489 984 Bytes 5 830 764 736 Bytes +268.3 KiB +0.005%
ydbd stripped size 1 464 451 824 Bytes 1 464 542 512 Bytes +88.6 KiB +0.006%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Aug 6, 2024 •
edited
Loading

⚪ 2024-08-06 12:49:41 UTC Pre-commit check for 471976f has started.
⚪ 2024-08-06 12:52:55 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-06 15:11:38 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76451 62845 0 2 13592 12

⚪ 2024-08-06 15:21:29 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟢 2024-08-06 15:29:57 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
16 (only retried tests) 8 0 0 0 8

🟢 2024-08-06 15:37:03 UTC Build successful.
🟡 2024-08-06 15:37:37 UTC ydbd size 8.1 GiB changed* by +222.9 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 58580f8 merge: 471976f diff diff %
ydbd size 8 672 472 720 Bytes 8 672 701 016 Bytes +222.9 KiB +0.003%
ydbd stripped size 471 954 664 Bytes 471 980 008 Bytes +24.8 KiB +0.005%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Aug 6, 2024 •
edited
Loading

⚪ 2024-08-06 12:49:41 UTC Pre-commit check for 471976f has started.
⚪ 2024-08-06 12:53:00 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-06 13:14:39 UTC Build successful.

@github-actions
Copy link

github-actions bot commented Aug 6, 2024 •
edited
Loading

⚪ 2024-08-06 12:49:52 UTC Pre-commit check for 471976f has started.
⚪ 2024-08-06 12:53:02 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-06 15:13:21 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14929 14874 0 7 34 14

🟢 2024-08-06 15:15:28 UTC Build successful.
🟡 2024-08-06 15:15:57 UTC ydbd size 5.4 GiB changed* by +269.9 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 58580f8 merge: 471976f diff diff %
ydbd size 5 831 268 696 Bytes 5 831 545 032 Bytes +269.9 KiB +0.005%
ydbd stripped size 1 464 775 536 Bytes 1 464 866 800 Bytes +89.1 KiB +0.006%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@Tony-Romanov Tony-Romanov marked this pull request as ready for review August 7, 2024 05:29
@Tony-Romanov Tony-Romanov requested review from a team as code owners August 7, 2024 05:29
@Tony-Romanov Tony-Romanov enabled auto-merge (squash) August 7, 2024 05:29
@Tony-Romanov Tony-Romanov linked an issue Aug 7, 2024 that may be closed by this pull request
TPC-DS q16, q94 sometimes fails with assert UnRef(): requirement Refs_ > 0 failed inside wide_combine (with FinalizeByKey) #6042
Closed
@Tony-Romanov Tony-Romanov merged commit a9e2b92 into ydb-platform:main Aug 7, 2024
@Tony-Romanov Tony-Romanov deleted the udf_fix branch August 7, 2024 11:16
@StekPerepolnen StekPerepolnen mentioned this pull request Aug 8, 2024
Merged
This was referenced Aug 8, 2024
Merged
Merged
lll-phill-lll pushed a commit to lll-phill-lll/ydb that referenced this pull request Sep 9, 2024
@Tony-Romanov @lll-phill-lll
Fix usage single UDF object from many threads in DQ. (ydb-platform#7436)
6d46214
lll-phill-lll pushed a commit to lll-phill-lll/ydb that referenced this pull request Sep 10, 2024
@Tony-Romanov @lll-phill-lll
Fix usage single UDF object from many threads in DQ. (ydb-platform#7436)
57d6885
lll-phill-lll pushed a commit to lll-phill-lll/ydb that referenced this pull request Sep 11, 2024
@Tony-Romanov @lll-phill-lll
Fix usage single UDF object from many threads in DQ. (ydb-platform#7436)
2a6530a
lll-phill-lll pushed a commit that referenced this pull request Sep 12, 2024
@Tony-Romanov @lll-phill-lll
Fix usage single UDF object from many threads in DQ. (#7436)
1d84ea1
lll-phill-lll pushed a commit to lll-phill-lll/ydb that referenced this pull request Sep 13, 2024
@Tony-Romanov @lll-phill-lll
Fix usage single UDF object from many threads in DQ. (ydb-platform#7436)
60571d1
lll-phill-lll pushed a commit to lll-phill-lll/ydb that referenced this pull request Sep 13, 2024
@Tony-Romanov @lll-phill-lll
Fix usage single UDF object from many threads in DQ. (ydb-platform#7436)
617e942
lll-phill-lll pushed a commit to lll-phill-lll/ydb that referenced this pull request Sep 15, 2024
@Tony-Romanov @lll-phill-lll
Fix usage single UDF object from many threads in DQ. (ydb-platform#7436)
6268acb
@lll-phill-lll lll-phill-lll mentioned this pull request Sep 18, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rvu1024 rvu1024 rvu1024 approved these changes

Assignees

No one assigned

Labels

area/yql YQL query language issues

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

TPC-DS q16, q94 sometimes fails with assert UnRef(): requirement Refs_ > 0 failed inside wide_combine (with FinalizeByKey)

2 participants

@Tony-Romanov @rvu1024