Skip to content

Forward cache bugfix index pages queue verify #3134

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Mar 28, 2024

Conversation

kunga
Copy link
Member

@kunga kunga commented Mar 25, 2024

Changelog entry

...

Changelog category

  • Not for changelog (changelog entry is not required)

Additional information

Fix "Cache line head don't want to do fetch but should" verify: we may request index page from Queues[0] while handling another queue (for example a history one)

Copy link

github-actions bot commented Mar 25, 2024

2024-03-25 13:31:53 UTC Pre-commit check for 4c49378 has started.
2024-03-25 13:31:55 UTC Build linux-x86_64-release-asan is running...
🟢 2024-03-25 13:34:22 UTC Build successful.
2024-03-25 13:34:35 UTC Tests are running...
🔴 2024-03-25 13:54:42 UTC Test run completed, no test results found for commit 659302d. Please check build logs.
2024-03-25 13:54:45 UTC Check cancelled

Copy link

github-actions bot commented Mar 25, 2024

2024-03-25 13:31:59 UTC Pre-commit check for 4c49378 has started.
2024-03-25 13:32:01 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-03-25 13:34:16 UTC Build successful.
2024-03-25 13:34:29 UTC Tests are running...
🔴 2024-03-25 13:54:33 UTC Test run completed, no test results found for commit 659302d. Please check build logs.
2024-03-25 13:54:36 UTC Check cancelled

Copy link

github-actions bot commented Mar 25, 2024

2024-03-25 13:32:09 UTC Pre-commit check for 4c49378 has started.
2024-03-25 13:32:14 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-03-25 13:34:31 UTC Build successful.

@@ -20,10 +20,10 @@ namespace NTable {
namespace {
using namespace NTest;

struct TFramesWrap : public NTest::TSteps<TFramesWrap>, protected NFwd::IPageLoadingQueue {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Хотел написать сюда честный тест на TEnv, но оказалось что тот TPartStore который создается TMake не кастуется к NTable::TPartStore

Поэтому пришлось писать тесты через Executor

Странная фигня, наверно надо бы это как-то исправить

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Тут я соглашусь, я когда впервые в локальную базу пришёл тоже удивился - в тестах очень многие такие структуры повторяются, в итоге тестируется не код локальной базы, а продублированный код тестов. Видимо оно у нас недостаточно generic, чтобы можно было только работу со страницами/блобами замокать, а всё остальное оставить. Идея красивая, иметь возможность запускать локальную базу поверх чего угодно, не только блоб стораджа, но на практике скорее не нужно, и абстракция всё-равно протекает по факту. :-/

Copy link

github-actions bot commented Mar 25, 2024

2024-03-25 13:58:07 UTC Pre-commit check for 47fc644 has started.
2024-03-25 13:58:10 UTC Build linux-x86_64-release-asan is running...
🟢 2024-03-25 14:00:34 UTC Build successful.
2024-03-25 14:00:49 UTC Tests are running...
🔴 2024-03-25 15:31:12 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14424 14122 0 47 229 26

Copy link

github-actions bot commented Mar 25, 2024

2024-03-25 13:58:28 UTC Pre-commit check for 47fc644 has started.
2024-03-25 13:58:31 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-03-25 14:00:43 UTC Build successful.

Copy link

github-actions bot commented Mar 25, 2024

2024-03-25 13:58:53 UTC Pre-commit check for 47fc644 has started.
2024-03-25 13:58:55 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-03-25 14:01:03 UTC Build successful.
2024-03-25 14:01:16 UTC Tests are running...
🔴 2024-03-25 15:12:32 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
69037 57931 0 6 11031 69

@kunga kunga requested a review from snaury March 25, 2024 13:59
@kunga kunga changed the title Forward cache bugfix index pages queue Forward cache bugfix index pages queue verify Mar 25, 2024
@kunga kunga self-assigned this Mar 25, 2024
@kunga kunga mentioned this pull request Mar 27, 2024
63 tasks
@@ -352,15 +353,15 @@ namespace NFwd {
}
}

TEgg MakeCache(const TPart *part, NPage::TGroupId groupId, TIntrusiveConstPtr<TSlices> bounds) noexcept
TEgg MakeCache(const TPart *part, NPage::TGroupId groupId, TIntrusiveConstPtr<TSlices> slices) noexcept
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Кстати оно в сканах называлось bounds, потому что я хотел подчеркнуть, что грузить что-либо за границами этих строк строго запрещено, там блобов может уже не существовать (шардированный компакшен умел компактить половину sst и удалять блобы от скомпакченных данных). Но в целом я не против переименования.

@@ -20,10 +20,10 @@ namespace NTable {
namespace {
using namespace NTest;

struct TFramesWrap : public NTest::TSteps<TFramesWrap>, protected NFwd::IPageLoadingQueue {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Тут я соглашусь, я когда впервые в локальную базу пришёл тоже удивился - в тестах очень многие такие структуры повторяются, в итоге тестируется не код локальной базы, а продублированный код тестов. Видимо оно у нас недостаточно generic, чтобы можно было только работу со страницами/блобами замокать, а всё остальное оставить. Идея красивая, иметь возможность запускать локальную базу поверх чего угодно, не только блоб стораджа, но на практике скорее не нужно, и абстракция всё-равно протекает по факту. :-/

@kunga kunga requested a review from snaury March 28, 2024 15:17
Copy link

github-actions bot commented Mar 28, 2024

2024-03-28 15:17:58 UTC Pre-commit check for 64b9fda has started.
2024-03-28 15:18:00 UTC Build linux-x86_64-release-asan is running...
🟢 2024-03-28 15:20:00 UTC Build successful.
2024-03-28 15:21:32 UTC Tests are running...
🔴 2024-03-28 16:47:05 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
9078 8841 0 29 167 41

Copy link

github-actions bot commented Mar 28, 2024

2024-03-28 15:19:29 UTC Pre-commit check for 64b9fda has started.
2024-03-28 15:19:30 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-03-28 15:25:10 UTC Build failed. see the build logs.

Copy link

github-actions bot commented Mar 28, 2024

2024-03-28 15:20:53 UTC Pre-commit check for 64b9fda has started.
2024-03-28 15:20:56 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-03-28 15:23:06 UTC Build successful.
2024-03-28 15:24:49 UTC Tests are running...
🟢 2024-03-28 16:29:36 UTC Tests successful.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
10225 10144 0 0 46 35

@kunga kunga merged commit 9573cb4 into ydb-platform:main Mar 28, 2024
This was referenced Mar 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants