Skip to content

Commit ec56ec6

Browse files
committed
Set Coordination component for export partition completion handler
The previous commit covered the four `BackgroundSchedulePool` / query-handler entry points that talk to ZooKeeper from the partition- export feature, but missed the completion path. The part-export completion callback is invoked from a `MergeTreeBackgroundExecutor` thread, which does not inherit any thread-local component set by the scheduling task. As a result, when `enforce_keeper_component_tracking` is on, every Keeper request issued from `ExportPartitionTaskScheduler::handlePartExportCompletion` (and its callees `handlePartExportSuccess`, `handlePartExportFailure`, `tryToMovePartToProcessed`, `areAllPartsProcessed`) raises `LOGICAL_ERROR: Current component is empty`, aborting the server in debug builds. Observed on PR #1718 (commit `e239d42`): - Stateless tests (amd_debug, sequential) - Stateless tests (amd_asan, db disk, distributed plan, sequential) - Stateless tests (amd_debug, distributed plan, s3 storage, sequential) - Stateless tests (arm_binary, sequential) - Stateless tests (arm_asan, azure, sequential) - Many `test_export_replicated_mt_partition_to_*` integration tests All show the same fatal stack ending in `tryToMovePartToProcessed` -> `zkutil::ZooKeeper::tryGet`. Wrap `handlePartExportCompletion` with `Coordination::setCurrentComponent` to cover all four sub-paths in one place. Also wrap `ExportPartFromPartitionExportTask::executeStep`, which is the same threading model (a `MergeTreeBackgroundExecutor` task issuing `zk->tryCreate`) and would fail identically when `export_merge_tree_partition_lock_inside_the_task` is enabled. CI: #1718
1 parent e239d42 commit ec56ec6

2 files changed

Lines changed: 8 additions & 0 deletions

File tree

src/Storages/MergeTree/ExportPartFromPartitionExportTask.cpp

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
#include <Storages/MergeTree/ExportPartFromPartitionExportTask.h>
22
#include <Common/ProfileEvents.h>
3+
#include <Common/ZooKeeper/ZooKeeperCommon.h>
34

45
namespace ProfileEvents
56
{
@@ -23,6 +24,9 @@ ExportPartFromPartitionExportTask::ExportPartFromPartitionExportTask(
2324

2425
bool ExportPartFromPartitionExportTask::executeStep()
2526
{
27+
/// Runs on a MergeTreeBackgroundExecutor thread, so it does not inherit any component set by the scheduling task.
28+
auto component_guard = Coordination::setCurrentComponent("ExportPartFromPartitionExportTask::executeStep");
29+
2630
const auto zk = storage.getZooKeeper();
2731
const auto part_name = manifest.data_part->name;
2832

src/Storages/MergeTree/ExportPartitionTaskScheduler.cpp

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
#include <Interpreters/DatabaseCatalog.h>
55
#include <Common/Exception.h>
66
#include <Common/ZooKeeper/Types.h>
7+
#include <Common/ZooKeeper/ZooKeeperCommon.h>
78
#include <Common/ProfileEvents.h>
89
#include "Storages/MergeTree/ExportPartitionUtils.h"
910
#include "Storages/MergeTree/MergeTreePartExportManifest.h"
@@ -271,6 +272,9 @@ void ExportPartitionTaskScheduler::handlePartExportCompletion(
271272
const StoragePtr & destination_storage,
272273
const MergeTreePartExportManifest::CompletionCallbackResult & result)
273274
{
275+
/// Invoked from MergeTreeBackgroundExecutor threads, so the component is not inherited from selectPartsToExport.
276+
auto component_guard = Coordination::setCurrentComponent("ExportPartitionTaskScheduler::handlePartExportCompletion");
277+
274278
const auto export_path = fs::path(storage.zookeeper_path) / "exports" / export_key;
275279
const auto processing_parts_path = export_path / "processing";
276280
const auto processed_part_path = export_path / "processed" / part_name;

0 commit comments

Comments
 (0)