Skip to content

Don't skip null values on COUNT(*) in column shards #8024

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Aug 22, 2024
Merged

Don't skip null values on COUNT(*) in column shards #8024

merged 10 commits into from
Aug 22, 2024

Conversation

swalrus1
Copy link
Collaborator

Fix COUNT(*) WHERE value IS NULL. Result was always zero, now it works as expected.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 19, 2024 •
edited
Loading

⚪ 2024-08-19 16:48:49 UTC Pre-commit check for 32c3887 has started.
⚪ 2024-08-19 16:51:35 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-19 17:03:52 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 19, 2024 •
edited
Loading

⚪ 2024-08-19 16:50:11 UTC Pre-commit check for 32c3887 has started.
⚪ 2024-08-19 16:53:42 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-19 19:02:45 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
10294 10184 0 37 26 47

🟢 2024-08-19 19:03:58 UTC Build successful.
🟢 2024-08-19 19:04:25 UTC ydbd size 5.5 GiB changed* by +28.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 9f309c4 merge: 32c3887 diff diff %
ydbd size 5 883 817 464 Bytes 5 883 847 064 Bytes +28.9 KiB +0.001%
ydbd stripped size 1 478 040 816 Bytes 1 478 048 304 Bytes +7.3 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 19, 2024 •
edited
Loading

⚪ 2024-08-19 16:51:00 UTC Pre-commit check for 32c3887 has started.
⚪ 2024-08-19 16:55:42 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-19 18:19:46 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
39153 33749 0 34 5318 52

⚪ 2024-08-19 18:23:34 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-19 18:31:14 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
105 (only retried tests) 43 0 25 0 37

⚪ 2024-08-19 18:31:24 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-19 18:38:28 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
83 (only retried tests) 21 0 25 0 37

🟢 2024-08-19 18:38:36 UTC Build successful.
🟢 2024-08-19 18:39:20 UTC ydbd size 8.1 GiB changed* by +27.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 9f309c4 merge: 32c3887 diff diff %
ydbd size 8 743 498 016 Bytes 8 743 526 584 Bytes +27.9 KiB +0.000%
ydbd stripped size 475 706 408 Bytes 475 708 648 Bytes +2.2 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 19, 2024 •
edited
Loading

⚪ 2024-08-19 19:36:12 UTC Pre-commit check for 875ff9a has started.
⚪ 2024-08-19 19:38:56 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-19 19:50:22 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 19, 2024 •
edited
Loading

⚪ 2024-08-19 19:36:30 UTC Pre-commit check for 875ff9a has started.
⚪ 2024-08-19 19:39:21 UTC Check linux-x86_64-release-asan is running...

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 19, 2024 •
edited
Loading

⚪ 2024-08-19 19:36:43 UTC Pre-commit check for 875ff9a has started.
⚪ 2024-08-19 19:39:31 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-19 21:07:39 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
39156 33764 0 36 5313 43

⚪ 2024-08-19 21:11:03 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-19 21:18:25 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
97 (only retried tests) 34 0 26 0 37

⚪ 2024-08-19 21:18:33 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-19 21:25:33 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
85 (only retried tests) 22 0 26 0 37

🟢 2024-08-19 21:25:40 UTC Build successful.
🟢 2024-08-19 21:26:19 UTC ydbd size 8.1 GiB changed* by +27.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: d3dd3a7 merge: 875ff9a diff diff %
ydbd size 8 743 498 032 Bytes 8 743 526 624 Bytes +27.9 KiB +0.000%
ydbd stripped size 475 706 408 Bytes 475 708 712 Bytes +2.2 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@swalrus1 swalrus1 linked an issue Aug 20, 2024 that may be closed by this pull request
"COUNT(*) WHERE field IS NULL" doesn't count NULL values in Column table #8036
Closed
@swalrus1 swalrus1 marked this pull request as draft August 20, 2024 07:43
@swalrus1 swalrus1 force-pushed the fix-select-star-where-column-is-null branch from 3d27d58 to 0715cae Compare August 20, 2024 11:49
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 20, 2024 •
edited
Loading

⚪ 2024-08-20 11:54:08 UTC Pre-commit check for 729ce4b has started.
⚪ 2024-08-20 11:57:00 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-20 12:09:02 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 20, 2024 •
edited
Loading

⚪ 2024-08-20 11:54:49 UTC Pre-commit check for 729ce4b has started.
⚪ 2024-08-20 11:57:36 UTC Check linux-x86_64-release-asan is running...
⚫ 2024-08-20 12:21:34 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 20, 2024 •
edited
Loading

⚪ 2024-08-20 11:55:12 UTC Pre-commit check for 729ce4b has started.
⚪ 2024-08-20 11:58:53 UTC Check linux-x86_64-relwithdebinfo is running...
⚫ 2024-08-20 12:21:38 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 20, 2024 •
edited
Loading

⚪ 2024-08-20 12:23:11 UTC Pre-commit check for 7ba36b7 has started.
⚪ 2024-08-20 12:26:40 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-20 12:29:12 UTC Build failed. see the logs.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 20, 2024 •
edited
Loading

⚪ 2024-08-20 12:25:16 UTC Pre-commit check for 7ba36b7 has started.
⚪ 2024-08-20 12:28:35 UTC Check linux-x86_64-release-clang14 is running...
🔴 2024-08-20 12:31:54 UTC Build failed. see the logs.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 20, 2024 •
edited
Loading

⚪ 2024-08-20 12:25:31 UTC Pre-commit check for 7ba36b7 has started.
⚪ 2024-08-20 12:28:16 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-08-20 12:32:15 UTC Build failed. see the logs.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 20, 2024 •
edited
Loading

⚪ 2024-08-20 13:14:09 UTC Pre-commit check for b83cdc2 has started.
⚪ 2024-08-20 13:17:03 UTC Check linux-x86_64-release-clang14 is running...
🔴 2024-08-20 13:20:13 UTC Build failed. see the logs.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 20, 2024 •
edited
Loading

⚪ 2024-08-20 13:14:28 UTC Pre-commit check for b83cdc2 has started.
⚪ 2024-08-20 13:17:38 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-20 13:20:27 UTC Build failed. see the logs.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 20, 2024 •
edited
Loading

⚪ 2024-08-20 13:16:21 UTC Pre-commit check for b83cdc2 has started.
⚪ 2024-08-20 13:19:07 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-08-20 13:23:10 UTC Build failed. see the logs.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 20, 2024 •
edited
Loading

⚪ 2024-08-20 13:27:20 UTC Pre-commit check for 9827a99 has started.
⚪ 2024-08-20 13:30:06 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-20 13:33:10 UTC Build failed. see the logs.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 20, 2024 •
edited
Loading

⚪ 2024-08-20 13:27:20 UTC Pre-commit check for 9827a99 has started.
⚪ 2024-08-20 13:30:08 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-08-20 13:33:50 UTC Build failed. see the logs.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 20, 2024 •
edited
Loading

⚪ 2024-08-20 13:35:15 UTC Pre-commit check for 9827a99 has started.
⚪ 2024-08-20 13:38:00 UTC Check linux-x86_64-release-clang14 is running...
🔴 2024-08-20 13:40:48 UTC Build failed. see the logs.

@swalrus1 swalrus1 force-pushed the fix-select-star-where-column-is-null branch from dff3376 to 8caed33 Compare August 20, 2024 13:47
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 20, 2024 •
edited
Loading

⚪ 2024-08-20 13:50:31 UTC Pre-commit check for b0f9e2c has started.
⚪ 2024-08-20 13:53:17 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-20 14:04:39 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 20, 2024 •
edited
Loading

⚪ 2024-08-20 13:50:33 UTC Pre-commit check for b0f9e2c has started.
⚪ 2024-08-20 13:55:58 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-20 15:29:33 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
39153 33806 0 13 5314 20

⚪ 2024-08-20 15:32:53 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-20 15:40:13 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
37 (only retried tests) 17 0 11 1 8

⚪ 2024-08-20 15:40:20 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-20 15:47:15 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
24 (only retried tests) 5 0 11 0 8

🟢 2024-08-20 15:47:22 UTC Build successful.
🟢 2024-08-20 15:47:57 UTC ydbd size 8.1 GiB changed* by +38.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 767ff64 merge: b0f9e2c diff diff %
ydbd size 8 748 356 632 Bytes 8 748 396 216 Bytes +38.7 KiB +0.000%
ydbd stripped size 475 797 864 Bytes 475 800 040 Bytes +2.1 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 20, 2024 •
edited
Loading

⚪ 2024-08-20 13:50:35 UTC Pre-commit check for b0f9e2c has started.
⚪ 2024-08-20 13:53:25 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-20 15:30:27 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
10084 10010 0 30 27 17

🟢 2024-08-20 15:31:22 UTC Build successful.
🟢 2024-08-20 15:31:52 UTC ydbd size 5.5 GiB changed* by +39.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 767ff64 merge: b0f9e2c diff diff %
ydbd size 5 887 494 088 Bytes 5 887 534 976 Bytes +39.9 KiB +0.001%
ydbd stripped size 1 478 366 640 Bytes 1 478 378 512 Bytes +11.6 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 22, 2024 •
edited
Loading

⚪ 2024-08-22 08:04:30 UTC Pre-commit check for e60bbd7 has started.
⚪ 2024-08-22 08:07:18 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-22 08:19:16 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 22, 2024 •
edited
Loading

⚪ 2024-08-22 08:04:51 UTC Pre-commit check for e60bbd7 has started.
⚪ 2024-08-22 08:07:43 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-22 09:47:49 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
10156 10074 0 33 31 18

🟢 2024-08-22 09:48:45 UTC Build successful.
🟡 2024-08-22 09:49:17 UTC ydbd size 5.5 GiB changed* by +430.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 892d691 merge: e60bbd7 diff diff %
ydbd size 5 888 764 656 Bytes 5 889 205 728 Bytes +430.7 KiB +0.007%
ydbd stripped size 1 478 697 328 Bytes 1 478 818 672 Bytes +118.5 KiB +0.008%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 22, 2024 •
edited
Loading

⚪ 2024-08-22 08:05:02 UTC Pre-commit check for e60bbd7 has started.
⚪ 2024-08-22 08:07:54 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-22 09:39:00 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
39225 33874 0 12 5325 14

⚪ 2024-08-22 09:42:21 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟢 2024-08-22 09:52:14 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
32 (only retried tests) 23 0 0 1 8

🟢 2024-08-22 09:52:21 UTC Build successful.
🟡 2024-08-22 09:52:58 UTC ydbd size 8.1 GiB changed* by +313.8 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: f2788b5 merge: e60bbd7 diff diff %
ydbd size 8 750 016 472 Bytes 8 750 337 816 Bytes +313.8 KiB +0.004%
ydbd stripped size 475 913 224 Bytes 475 931 912 Bytes +18.2 KiB +0.004%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@swalrus1 swalrus1 merged commit a5de48c into ydb-platform:main Aug 22, 2024
10 of 12 checks passed
@swalrus1 swalrus1 deleted the fix-select-star-where-column-is-null branch August 22, 2024 10:58
@StekPerepolnen StekPerepolnen mentioned this pull request Aug 22, 2024
Merged
This was referenced Aug 23, 2024
Merged
Merged
Merged
rekby pushed a commit to rekby-forks/ydb that referenced this pull request Aug 28, 2024
@swalrus1 @rekby
Don't skip null values on COUNT(*) in column shards (ydb-platform#8024)
8a590a9
@niksaveliev niksaveliev mentioned this pull request Aug 29, 2024
Merged
This was referenced Aug 30, 2024
Merged
Merged
stanislav-shchetinin pushed a commit to stanislav-shchetinin/ydb that referenced this pull request Aug 30, 2024
@swalrus1 @stanislav-shchetinin
Don't skip null values on COUNT(*) in column shards (ydb-platform#8024)
29650e3
@siarheivesialou siarheivesialou mentioned this pull request Sep 4, 2024
Merged
@niksaveliev niksaveliev mentioned this pull request Sep 9, 2024
Merged
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 9, 2024
@swalrus1 @zverevgeny
Don't skip null values on COUNT(*) in column shards (ydb-platform#8024)
b24b76c
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 10, 2024
@swalrus1 @zverevgeny
Don't skip null values on COUNT(*) in column shards (ydb-platform#8024)
17a2578
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 11, 2024
@swalrus1 @zverevgeny
Don't skip null values on COUNT(*) in column shards (ydb-platform#8024)
36b0aaf
@niksaveliev niksaveliev mentioned this pull request Sep 12, 2024
Merged
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 13, 2024
@swalrus1 @zverevgeny
Don't skip null values on COUNT(*) in column shards (ydb-platform#8024)
a88ded8
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 14, 2024
@swalrus1 @zverevgeny
Don't skip null values on COUNT(*) in column shards (ydb-platform#8024)
8f92649
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 15, 2024
@swalrus1 @zverevgeny
Don't skip null values on COUNT(*) in column shards (ydb-platform#8024)
d4df63b
@zverevgeny zverevgeny mentioned this pull request Sep 16, 2024
Closed
This was referenced Sep 17, 2024
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ivanmorozov333 ivanmorozov333 ivanmorozov333 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

"COUNT(*) WHERE field IS NULL" doesn't count NULL values in Column table
2 participants
@swalrus1 @ivanmorozov333