Skip to content

messages for shadow partition #1084

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Jan 26, 2024
Merged

messages for shadow partition #1084

merged 7 commits into from
Jan 26, 2024

Conversation

Alek5andr-Kotov
Copy link
Collaborator

Сообщения для служебной партиции. Позволяют получить информацию о записанных данных

alexnick88
alexnick88 reviewed Jan 17, 2024
View reviewed changes
ydb/core/persqueue/events/internal.h Outdated
struct TSuccess {
TString SourceId;
ui64 MinSeqNo, MaxSeqNo;
NKikimrPQ::TPartitionKeyRange KeyRange;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

тут список ключей для ренейма, а не рендж. А лучше то, что в body лежит сразу

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

исправил
1518b41

ydb/core/persqueue/events/internal.h Outdated
};

struct TEvGetWriteInfoResponse : public TEventLocal<TEvGetWriteInfoResponse, EvGetWriteInfoResponse> {
struct TError {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

тут надо сообщение об ошике, и его в Propose отдавать в итоге ( или в причине аборта транзакции)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

добавил текст сообщения об ошибке

ydb/core/persqueue/events/internal.h Outdated
};

struct TSuccess {
TString SourceId;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

тут вектор. Сорсидов может быть много.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

исправил
1518b41

alexnick88
alexnick88 reviewed Jan 23, 2024
View reviewed changes
ydb/core/persqueue/events/internal.h
ui32 Cookie; // ShadowPartitionId
};

struct TEvGetWriteInfoResponse : public TEventLocal<TEvGetWriteInfoResponse, EvGetWriteInfoResponse> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

split in 2 messages - writeinforesponse and writeinfoerror

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
7f74ed7

ydb/core/persqueue/events/internal.h Outdated
struct TSuccess {
THashMap<TString, NPQ::TSeqNoRange> SeqNo; // SourceId -> (MinSeqNo, MaxSeqNo)
std::deque<NPQ::TDataKey> BodyKeys;
std::deque<NPQ::TBatch> Head;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change to list of messages (MsgWrite)

Copy link
Collaborator Author

@Alek5andr-Kotov Alek5andr-Kotov Jan 25, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
00dee5e

ydb/core/persqueue/events/internal.h
@@ -65,6 +66,18 @@ namespace NPQ {
{}
};

struct TDataKey {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Change caching proxy logic to process move commands too.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added it to the task of transferring data

alexnick88
alexnick88 previously approved these changes Jan 26, 2024
View reviewed changes
@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 26, 2024 •
edited
Loading

⚪ 2024-01-26 09:36:55 UTC Pre-commit check for 9d469b9 has started.
⚪ 2024-01-26 09:36:58 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-01-26 09:41:35 UTC Build successful.
⚪ 2024-01-26 09:41:52 UTC Tests are running...
🔴 2024-01-26 11:18:09 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
60159 50886 0 1 9245 27

@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 26, 2024 •
edited
Loading

⚪ 2024-01-26 09:37:09 UTC Pre-commit check for 9d469b9 has started.
⚪ 2024-01-26 09:37:11 UTC Build linux-x86_64-release-asan is running...
🟢 2024-01-26 09:42:22 UTC Build successful.
⚪ 2024-01-26 09:42:37 UTC Tests are running...
🔴 2024-01-26 11:28:43 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15957 15852 0 26 56 23

alexnick88
alexnick88 approved these changes Jan 26, 2024
View reviewed changes
ydb/core/persqueue/events/internal.h Outdated
@@ -1033,7 +1033,7 @@ struct TEvPQ {
TEvGetWriteInfoResponse(ui32 cookie,
THashMap<TString, NPQ::TSeqNoRange> seqNo,
std::deque<NPQ::TDataKey> bodyKeys,
std::deque<NPQ::TBatch> head) :
TVector<NPQ::TClientBlob> head) :
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TVector && - будет лишнее копирование

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

исправил
8fc84c4

@Alek5andr-Kotov Alek5andr-Kotov merged commit a358916 into main Jan 26, 2024
This was referenced Jan 26, 2024
Merged
Merged
Merged
Merged
Merged
Merged
Merged
@ildar-khisambeev ildar-khisambeev mentioned this pull request Jan 30, 2024
Merged
This was referenced Jan 30, 2024
Merged
Merged
Merged
Merged
@Jornydb Jornydb mentioned this pull request Jan 31, 2024
Merged
@GrigoriyPA GrigoriyPA mentioned this pull request Feb 7, 2024
Merged
This was referenced Feb 7, 2024
Merged
Merged
Merged
@ildar-khisambeev ildar-khisambeev mentioned this pull request Feb 8, 2024
Merged
@vitstn vitstn mentioned this pull request Feb 8, 2024
Merged
@niksaveliev niksaveliev mentioned this pull request Feb 9, 2024
Merged
@GrigoriyPA GrigoriyPA mentioned this pull request Feb 9, 2024
Merged
@pavelvelikhov pavelvelikhov mentioned this pull request Feb 9, 2024
Merged
@spuchin spuchin deleted the LOGBROKER-8878 branch February 9, 2024 17:59
@niksaveliev niksaveliev mentioned this pull request Feb 12, 2024
Merged
@starlinskiy starlinskiy mentioned this pull request Feb 12, 2024
Closed
@vitstn vitstn mentioned this pull request Feb 12, 2024
Merged
@GrigoriyPA GrigoriyPA mentioned this pull request Feb 12, 2024
Merged
@vitstn vitstn mentioned this pull request Feb 12, 2024
Merged
@starlinskiy starlinskiy mentioned this pull request Feb 12, 2024
Merged
@pavelvelikhov pavelvelikhov mentioned this pull request Feb 13, 2024
Merged
This was referenced Feb 13, 2024
Merged
Merged
Merged
@pavelvelikhov pavelvelikhov mentioned this pull request Feb 15, 2024
Merged
@niksaveliev niksaveliev mentioned this pull request Feb 16, 2024
Merged
@vitstn vitstn mentioned this pull request Feb 16, 2024
Merged
This was referenced Feb 17, 2024
Merged
Merged
Merged
Merged
Merged
@aakulaga-ydb aakulaga-ydb mentioned this pull request Feb 24, 2024
Merged
@GrigoriyPA GrigoriyPA mentioned this pull request Feb 26, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alexnick88 alexnick88 alexnick88 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Alek5andr-Kotov @alexnick88