YQ-2634: S3 runtime file listing #925
Conversation
|
Note This is an automated comment that will be appended during run. 🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit c43f84b.
🔴 linux-x86_64-release-asan: some tests FAILED for commit c43f84b.
|
ydb/library/yql/providers/s3/provider/yql_s3_dq_integration.cpp
Outdated
Show resolved
Hide resolved
MetaGigachad
force-pushed
the
YQ-2634
branch
2 times, most recently
from
29d5a6c to
509c96e
Compare
|
⚪
|
|
⚪
|
|
⚪
|
|
⚪
|
|
⚪ |
|
⚪ |
|
⚪
|
|
⚪
|
|
⚪
|
|
⚪
|
|
dq tests broken by #1503 |
ydb/library/yql/providers/s3/provider/yql_s3_dq_integration.cpp
Outdated
Show resolved
Hide resolved
| @@ -120,6 +120,11 @@ class TRetryEventsQueue { | |||
| } | |||
| return false; | |||
| } | |||
|
|
|||
| bool HasPendingEvents() { | |||
| RemoveConfirmedEvents(MyConfirmedSeqNo); | |||
There was a problem hiding this comment.
bad naming maybe
Why do you modify smth in method called Has... ?
| static_assert(EvEnd < EventSpaceEnd(NKikimr::TKikimrEvents::ES_S3_FILE_QUEUE), | ||
| "expect EvEnd < EventSpaceEnd(TEvents::ES_S3_FILE_QUEUE)"); | ||
|
|
||
| struct TEvUpdateConsumersCount : |
There was a problem hiding this comment.
Why did you replace local events with non-local ones here?
* Base feature implementation * Added interconnect retries for FileQueue * Added best effort FileQueue pass away * Global ids for PB events, review fixes * Added missing peer dirs * Move FileQueue creation to FillSourceSettings * Added FileQueueBatchSize pragrma, batch also limited by requested amount * Round-robin stage in FileQueue, new s3 settings * FileQueue processes one batch at a time, removing amount from batch request * Runtime listing fixes * Clean up * Added parametrization in partitioning integration tests * Poison fix * Fixed partitioning columns, fixed retries, added extra tests * Unit test imports fixes * FileQueue consumers count correction * Review fixes
* Base feature implementation * Added interconnect retries for FileQueue * Added best effort FileQueue pass away * Global ids for PB events, review fixes * Added missing peer dirs * Move FileQueue creation to FillSourceSettings * Added FileQueueBatchSize pragrma, batch also limited by requested amount * Round-robin stage in FileQueue, new s3 settings * FileQueue processes one batch at a time, removing amount from batch request * Runtime listing fixes * Clean up * Added parametrization in partitioning integration tests * Poison fix * Fixed partitioning columns, fixed retries, added extra tests * Unit test imports fixes * FileQueue consumers count correction * Review fixes
No description provided.