YQ-4085 FQ: Add partition id check in row dispatcher #14178

kardymonds · 2025-02-04T08:07:54Z

Changelog entry

...

Changelog category

Not for changelog (changelog entry is not required)

Additional information

...

github-actions · 2025-02-04T08:12:04Z

⚪ 2025-02-04 08:12:04 UTC Pre-commit check linux-x86_64-relwithdebinfo for 2ff2173 has started.
⚪ 2025-02-04 08:12:17 UTC Artifacts will be uploaded here
⚪ 2025-02-04 08:15:26 UTC ya make is running...
🟡 2025-02-04 09:08:30 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
25051	22493	0	3	2425	130

⚪ 2025-02-04 09:10:53 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-02-04 09:23:18 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
199 (only retried tests)	74	0	0	0	125

🟢 2025-02-04 09:23:25 UTC Build successful.
🟢 2025-02-04 09:23:50 UTC ydbd size 2.1 GiB changed* by +6.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `3defd52`	merge: `2ff2173`	diff	diff %
ydbd size	2 224 081 456 Bytes	2 224 087 656 Bytes	+6.1 KiB	+0.000%
ydbd stripped size	472 677 304 Bytes	472 678 840 Bytes	+1.5 KiB	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-02-04T08:12:46Z

⚪ 2025-02-04 08:12:46 UTC Pre-commit check linux-x86_64-release-asan for 2ff2173 has started.
⚪ 2025-02-04 08:12:58 UTC Artifacts will be uploaded here
⚪ 2025-02-04 08:15:59 UTC ya make is running...
🟡 2025-02-04 09:16:15 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
11136	11070	0	24	11	31

⚪ 2025-02-04 09:17:18 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-02-04 09:29:44 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
104 (only retried tests)	71	0	2	3	28

⚪ 2025-02-04 09:29:53 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-02-04 09:41:49 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
55 (only retried tests)	23	0	1	5	26

🟢 2025-02-04 09:41:56 UTC Build successful.
🟢 2025-02-04 09:42:27 UTC ydbd size 3.6 GiB changed* by +9.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `3defd52`	merge: `2ff2173`	diff	diff %
ydbd size	3 867 472 256 Bytes	3 867 481 632 Bytes	+9.2 KiB	+0.000%
ydbd stripped size	1 355 262 864 Bytes	1 355 265 520 Bytes	+2.6 KiB	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

ydb/core/fq/libs/row_dispatcher/row_dispatcher.cpp

yumkam · 2025-02-04T09:55:57Z

ydb/core/fq/libs/row_dispatcher/row_dispatcher.cpp

-        Y_ENSURE(sessionInfo.Consumers.count(consumer->ReadActorId));
+        if (!sessionInfo.Consumers.contains(consumer->ReadActorId)) {
+            LOG_ROW_DISPATCHER_ERROR("Wrong readActorId " << consumer->ReadActorId << ", no such consumer");
+        }


Nit: if (!erase())

.erase возвращает false если не нашёл, можно просто его проверять, минус строчка (и минус лукап)

ydb/library/yql/providers/pq/async_io/dq_pq_read_actor_base.cpp

github-actions · 2025-02-04T10:07:58Z

⚪ 2025-02-04 10:07:57 UTC Pre-commit check linux-x86_64-release-asan for 6d01e69 has started.
⚪ 2025-02-04 10:08:01 UTC Artifacts will be uploaded here
⚪ 2025-02-04 10:10:55 UTC ya make is running...
🟡 2025-02-04 11:09:18 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
11136	11072	0	22	8	34

⚪ 2025-02-04 11:10:34 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-02-04 11:32:13 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
102 (only retried tests)	71	0	1	2	28

⚪ 2025-02-04 11:32:21 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-02-04 11:43:49 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
54 (only retried tests)	24	0	1	3	26

🟢 2025-02-04 11:43:56 UTC Build successful.
🟢 2025-02-04 11:44:28 UTC ydbd size 3.6 GiB changed* by +9.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `3defd52`	merge: `6d01e69`	diff	diff %
ydbd size	3 867 472 256 Bytes	3 867 481 456 Bytes	+9.0 KiB	+0.000%
ydbd stripped size	1 355 262 864 Bytes	1 355 265 648 Bytes	+2.7 KiB	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-02-04T10:08:38Z

⚪ 2025-02-04 10:08:37 UTC Pre-commit check linux-x86_64-relwithdebinfo for 6d01e69 has started.
⚪ 2025-02-04 10:08:47 UTC Artifacts will be uploaded here
⚪ 2025-02-04 10:11:41 UTC ya make is running...
🟡 2025-02-04 11:07:34 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
25051	22484	0	6	2436	125

⚪ 2025-02-04 11:09:44 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-02-04 11:22:47 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
208 (only retried tests)	86	0	0	0	122

🟢 2025-02-04 11:22:54 UTC Build successful.
🟢 2025-02-04 11:23:17 UTC ydbd size 2.1 GiB changed* by +6.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `3defd52`	merge: `6d01e69`	diff	diff %
ydbd size	2 224 081 456 Bytes	2 224 088 016 Bytes	+6.4 KiB	+0.000%
ydbd stripped size	472 677 304 Bytes	472 678 968 Bytes	+1.6 KiB	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

yumkam · 2025-02-04T10:37:06Z

ydb/core/fq/libs/row_dispatcher/row_dispatcher.cpp

-        Y_ENSURE(sessionInfo.Consumers.count(consumer->ReadActorId));
+        if (!sessionInfo.Consumers.contains(consumer->ReadActorId)) {
+            LOG_ROW_DISPATCHER_ERROR("Wrong readActorId " << consumer->ReadActorId << ", no such consumer");
+        }


.erase возвращает false если не нашёл, можно просто его проверять, минус строчка (и минус лукап)

yumkam · 2025-02-04T10:42:31Z

ydb/core/fq/libs/row_dispatcher/row_dispatcher.cpp

-    partition.PendingNewDataArrived = true;
-    it->second->Counters.NewDataArrived++;
-    it->second->EventsQueue.Send(ev->Release().Release(), it->second->Generation);
+    auto consumerInfoPtr = it->second; 


nit: можно передвинуть на пару строчек выше и упростить LWPROBE/TRACE

Поправлю в любом следующем pr

)

…le (#14191)

)

add partition id check

6713976

kardymonds requested a review from a team as a code owner February 4, 2025 08:07

github-actions bot added the not-for-changelog label Feb 4, 2025

remove comment

0c79fce

yumkam reviewed Feb 4, 2025

View reviewed changes

fix if

744e16b

yumkam approved these changes Feb 4, 2025

View reviewed changes

kardymonds merged commit 1f034d6 into ydb-platform:main Feb 4, 2025
12 checks passed

kardymonds added a commit to kardymonds/ydb that referenced this pull request Feb 4, 2025

YQ-4085 FQ: Add partition id check in row dispatcher (ydb-platform#14178

e27bd75

)

kardymonds added a commit that referenced this pull request Feb 4, 2025

YQ-4085 FQ: Add partition id check in row dispatcher #14178 / to stab…

36884b0

…le (#14191)

GrigoriyPA mentioned this pull request Feb 11, 2025

YQ-4103 fixed overflow in kqp write actor CalculateNextAttemptDelay #14415

Merged

lberserq pushed a commit to lberserq/ydb that referenced this pull request Feb 14, 2025

YQ-4085 FQ: Add partition id check in row dispatcher (ydb-platform#14178

0c74b81

)

GrigoriyPA mentioned this pull request Feb 17, 2025

merge to yq stable YQ remove anonymous namespace from CA #14656

Closed

GrigoriyPA mentioned this pull request Mar 6, 2025

merge to yq stable YQ added YDB_AUTH_TICKET_HEADER for fq proxy ydb requests #15432

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

YQ-4085 FQ: Add partition id check in row dispatcher #14178

YQ-4085 FQ: Add partition id check in row dispatcher #14178

Uh oh!

kardymonds commented Feb 4, 2025

Uh oh!

github-actions bot commented Feb 4, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Feb 4, 2025 •

edited

Loading

Uh oh!

Uh oh!

yumkam Feb 4, 2025

Uh oh!

yumkam Feb 4, 2025

Uh oh!

Uh oh!

github-actions bot commented Feb 4, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Feb 4, 2025 •

edited

Loading

Uh oh!

yumkam Feb 4, 2025

Uh oh!

yumkam Feb 4, 2025

Uh oh!

kardymonds Feb 4, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

YQ-4085 FQ: Add partition id check in row dispatcher #14178

YQ-4085 FQ: Add partition id check in row dispatcher #14178

Uh oh!

Conversation

kardymonds commented Feb 4, 2025

Changelog entry

Changelog category

Additional information

Uh oh!

github-actions bot commented Feb 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

yumkam Feb 4, 2025

Choose a reason for hiding this comment

Uh oh!

yumkam Feb 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Feb 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yumkam Feb 4, 2025

Choose a reason for hiding this comment

Uh oh!

yumkam Feb 4, 2025

Choose a reason for hiding this comment

Uh oh!

kardymonds Feb 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions bot commented Feb 4, 2025 •

edited

Loading

github-actions bot commented Feb 4, 2025 •

edited

Loading

github-actions bot commented Feb 4, 2025 •

edited

Loading

github-actions bot commented Feb 4, 2025 •

edited

Loading