Skip to content

Add start and end times to Operations API #5426

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Jun 17, 2024
Merged

Add start and end times to Operations API #5426

merged 3 commits into from
Jun 17, 2024

Conversation

@pixcc
Copy link
Member

@pixcc pixcc commented Jun 11, 2024 •
edited
Loading

Changelog entry

Add start and end times for long-running operations (export, import).

Changelog category

  • New feature

Additional information

$ ydb export s3
┌───────────────────────────────────────────┬───────┬─────────┬───────────┬───────────┬────────┐
│ id                                        │ ready │ status  │ progress  │ endpoint  │ bucket │
├───────────────────────────────────────────┼───────┼─────────┼───────────┼───────────┼────────┤
│ ydb://export/6?id=844425031076972&kind=s3 │ false │ SUCCESS │ Preparing │ localhost │ 1      │
├╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴┴╴╴╴╴╴╴╴┴╴╴╴╴╴╴╴╴╴┴╴╴╴╴╴╴╴╴╴╴╴┴╴╴╴╴╴╴╴╴╴╴╴┴╴╴╴╴╴╴╴╴┤
│ StorageClass: NOT_SET                                                                        │
│ Items:                                                                                       │
│   - source: /Root/pixcc-slice-db/users                                                       │
│     destination: 1                                                                           │
│ Description:                                                                                 │
│ Number of retries: 10                                                                        │
│ Start time: 2024-06-10T23:55:57Z                                                             │
└──────────────────────────────────────────────────────────────────────────────────────────────┘

$ ydb operation cancel "ydb://export/6?id=844425031076972&kind=s3"

$ ydb operation get "ydb://export/6?id=844425031076972&kind=s3"
┌───────────────────────────────────────────┬───────┬───────────┬───────────┬───────────┬────────┐
│ id                                        │ ready │ status    │ progress  │ endpoint  │ bucket │
├───────────────────────────────────────────┼───────┼───────────┼───────────┼───────────┼────────┤
│ ydb://export/6?id=844425031076972&kind=s3 │ true  │ CANCELLED │ Cancelled │ localhost │ 1      │
├╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴┴╴╴╴╴╴╴╴┴╴╴╴╴╴╴╴╴╴╴╴┴╴╴╴╴╴╴╴╴╴╴╴┴╴╴╴╴╴╴╴╴╴╴╴┴╴╴╴╴╴╴╴╴┤
│ StorageClass: NOT_SET                                                                          │
│ Issues:                                                                                        │
│   - <main>: Error: Cancelled manually                                                          │
│ Items:                                                                                         │
│   - source: /Root/pixcc-slice-db/users                                                         │
│     destination: 1                                                                             │
│ Description:                                                                                   │
│ Number of retries: 10                                                                          │
│ Start time: 2024-06-10T23:55:57Z                                                               │
│ End time: 2024-06-10T23:57:30Z                                                                 │
└────────────────────────────────────────────────────────────────────────────────────────────────┘

@pixcc pixcc requested a review from CyberROFL June 11, 2024 01:08
@github-actions
Copy link

github-actions bot commented Jun 11, 2024 •
edited
Loading

⚪ 2024-06-11 01:11:42 UTC Pre-commit check for b65e1d9 has started.
⚪ 2024-06-11 01:14:48 UTC Build linux-x86_64-release-clang14 is running...
⚫ 2024-06-11 01:51:53 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Jun 11, 2024 •
edited
Loading

⚪ 2024-06-11 01:11:49 UTC Pre-commit check for b65e1d9 has started.
⚪ 2024-06-11 01:14:56 UTC Build linux-x86_64-release-asan is running...
⚫ 2024-06-11 01:51:53 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Jun 11, 2024 •
edited
Loading

⚪ 2024-06-11 01:11:53 UTC Pre-commit check for b65e1d9 has started.
⚪ 2024-06-11 01:15:00 UTC Build linux-x86_64-relwithdebinfo is running...
⚫ 2024-06-11 01:51:53 UTC Check cancelled

@pixcc pixcc linked an issue Jun 11, 2024 that may be closed by this pull request
Start time & end time of long-running operations (api) #5427
Closed
@github-actions
Copy link

github-actions bot commented Jun 11, 2024 •
edited
Loading

⚪ 2024-06-11 01:53:13 UTC Pre-commit check for 5f2894e has started.
⚪ 2024-06-11 01:56:03 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-06-11 02:01:26 UTC Build successful.

@github-actions
Copy link

github-actions bot commented Jun 11, 2024 •
edited
Loading

⚪ 2024-06-11 01:53:13 UTC Pre-commit check for 5f2894e has started.
⚪ 2024-06-11 01:56:14 UTC Build linux-x86_64-release-asan is running...
🟢 2024-06-11 02:05:45 UTC Build successful.
⚪ 2024-06-11 02:06:01 UTC Tests are running...
🔴 2024-06-11 04:04:40 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15377 15292 0 26 47 12

@github-actions
Copy link

github-actions bot commented Jun 11, 2024 •
edited
Loading

⚪ 2024-06-11 01:55:23 UTC Pre-commit check for 5f2894e has started.
⚪ 2024-06-11 01:58:23 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-06-11 02:04:46 UTC Build successful.
⚪ 2024-06-11 02:05:00 UTC Tests are running...
🔴 2024-06-11 04:12:54 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
75039 62173 0 3 12853 10

@pixcc pixcc requested a review from spuchin June 11, 2024 08:23
spuchin
spuchin reviewed Jun 11, 2024
View reviewed changes
ydb/public/api/protos/ydb_operation.proto
// - Export;
// - Import.
google.protobuf.Timestamp start_time = 8;
google.protobuf.Timestamp end_time = 9;
Copy link
Collaborator

@spuchin spuchin Jun 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Если уж ты выше переименовал в 'completed', почему тогда не complete_time?

Copy link
Member Author

@pixcc pixcc Jun 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

end_time как будто сильно более популярнее. Особенно в связке с start_time.

В нашем публичном API для прогресса айтемов экспорта/импорта уже есть ровно такая связка.
https://github.com/ydb-platform/ydb/blob/main/ydb/public/api/protos/ydb_export.proto#L28

@pixcc pixcc requested a review from spuchin June 11, 2024 11:23
@github-actions
Copy link

github-actions bot commented Jun 11, 2024 •
edited
Loading

⚪ 2024-06-11 11:43:32 UTC Pre-commit check for b1d1c69 has started.
⚪ 2024-06-11 11:46:24 UTC Build linux-x86_64-release-asan is running...
🟢 2024-06-11 12:28:04 UTC Build successful.
⚪ 2024-06-11 12:28:14 UTC Tests are running...
🔴 2024-06-11 14:11:03 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15342 15257 0 25 47 13

@github-actions
Copy link

github-actions bot commented Jun 11, 2024 •
edited
Loading

⚪ 2024-06-11 11:47:43 UTC Pre-commit check for b1d1c69 has started.
⚪ 2024-06-11 11:50:36 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-06-11 12:32:59 UTC Build successful.

@github-actions
Copy link

github-actions bot commented Jun 11, 2024 •
edited
Loading

⚪ 2024-06-11 12:00:36 UTC Pre-commit check for b1d1c69 has started.
⚪ 2024-06-11 12:03:28 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-06-11 12:45:57 UTC Build successful.
⚪ 2024-06-11 12:46:08 UTC Tests are running...
🟢 2024-06-11 14:53:36 UTC Tests successful.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
75042 62180 0 0 12852 10

@pixcc pixcc merged commit 0dc827a into ydb-platform:main Jun 17, 2024
This was referenced Jun 18, 2024
Merged
Merged
Merged
This was referenced Jun 23, 2024
Merged
Merged
Merged
pixcc added a commit to pixcc/ydb that referenced this pull request Sep 4, 2024
@pixcc
Add start and end times to Operations API (ydb-platform#5426)
cede5ba
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CyberROFL CyberROFL CyberROFL approved these changes

@fomichev3000 fomichev3000 fomichev3000 approved these changes

@spuchin spuchin Awaiting requested review from spuchin

Assignees

No one assigned

Labels

new-feature

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Start time & end time of long-running operations (api)

4 participants

@pixcc @CyberROFL @fomichev3000 @spuchin