Fix busywait on adding to full async input buffer #14522
Conversation
DoExecute->PollAsyncInput->source.PollAsyncInput->ContinueExecution->DoExecute...
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
yumkam
changed the title
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
(cherry picked from commit 7cb4eb1)
|
🟢 |
yumkam
force-pushed
the
fix-busywait-full-input-buffers
branch
from
4e6644d to
fd18923
Compare
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
yumkam
added
the
rebase-and-check
github-actions
bot
removed
the
rebase-and-check
|
⚪ Test history | Ya make output | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
| LastPollResult = PollAsyncInput(); | ||
|
|
||
| if (LastPollResult && *LastPollResult != EResumeSource::CAPollAsyncNoSpace) { | ||
| ContinueExecute(*std::exchange(LastPollResult, {})); |
There was a problem hiding this comment.
Зачем еще раз запрашивать новый цикл? Он же сюда же в итоге придет. Данные сорсов синхронно пушаться в TaskRunner (TaskRunnerActor->AsyncInputPush), поэтому можно/нужно сразу вызывать AskContinueRun().
Т.е. тут даже можно прокинуть EResumeSource и анализировать: если пришли по CANewAsyncInput, то делаем (PollAsyncInput()); если пришли по другой причине, то безусловано делаем AskContinueRun().
Это всё неточно(
There was a problem hiding this comment.
Если
- нам не выделили cpu quota;
- есть непереполненные входы;
то мы можем продолжать их вычитывать, не дожидаясь квоты
There was a problem hiding this comment.
Я не вполне уверен, что от всей этой асинхронности много толку, с высокой вероятностью данные всё равно будут оседать в каких-то буферах, но я предпочёл заткнуть очевидную дырку ( busywait) и как можно меньше менять работу остальной части
| @@ -1185,6 +1190,9 @@ class TDqAsyncComputeActor : public TDqComputeActorBase<TDqAsyncComputeActor, TC | |||
| CA_LOG_T("AsyncCheckRunStatus: TakeInputChannelDataRequests: " << TakeInputChannelDataRequests.size()); | |||
| return; | |||
| } | |||
| if (ProcessOutputsState.LastRunStatus == ERunStatus::PendingInput && LastPollResult) { | |||
There was a problem hiding this comment.
Как я понял тут именно CAPollAsyncNoSpace.
Еще вариант (теоретический) не использовать вообще CAPollAsyncNoSpace, а по TEvNewAsyncInputDataArrived сохранять какой то флажок, а не опрашивать в цикле
There was a problem hiding this comment.
Да, тут ожидается только NoSpace; других вариантов, кмк, тут уже быть не может.
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
(cherry picked from commit b2df7c8)
(cherry picked from commit b2df7c8)
(cherry picked from commit b2df7c8)
DoExecute->PollAsyncInput->source.PollAsyncInput->ContinueExecution->DoExecute...
Changelog entry
...
Changelog category
Additional information
There are two scenario where busywait was triggered:
...
Checks passed: