[TransferEngine] Make the RDMA worker idle spin window configurable

[changcui]

Description

WorkerPool::transferWorker busy-polls for a hard-coded 100 ms (kWaitPeriodInNano) after its device instance becomes idle, and only then parks on the pool's condition variable. The idle check is pool-wide (processed_slice_count_ == submitted_slice_count_). As a result, every worker of an instance keeps polling while any slice on that instance is in flight, and each polling worker occupies a whole CPU core.

When transfers arrive more often than every 100 ms but are far from saturating the NICs, the workers never park. We hit this in a PD-disaggregated deployment. An 8-GPU decode node pulled KV cache over 8 HCAs with MC_WORKERS_PER_CTX=4, which is 256 workers across 8 processes. Those workers kept about 90 cores busy, although the actual post and poll work needed only a small fraction of that. The node exceeded its container CPU quota, and CFS throttling then stalled the inference processes.

This PR adds MC_RDMA_WORKER_IDLE_SPIN_US to set the window:

• The default is 100000 (100 ms), so behavior is unchanged unless the variable is set.
• The range is 0-10000000. 0 parks a worker as soon as its instance is idle.
• Parsing is strict, as with MC_CONN_PAUSE_TTL_MS. Non-numeric, negative, out-of-range or suffixed values are ignored with a WARNING and the default is kept. An accepted value is logged at INFO.
• The parking protocol is unchanged: the double check under cond_mutex_, the notify_all from submitters while parked_worker_count_ > 0, and the 1 s wait_for backstop. A worker with outstanding completions still polls.
• The UB transport (ub_context.cpp) has the same constant but a different idle loop, and is left unchanged.

Trade-off: with a shorter window, a parked worker pays a thread wake-up when the next transfer arrives. Deployments that care most about latency and have spare cores can keep the default.

Module

• Transfer Engine (mooncake-transfer-engine)
• Mooncake Store (mooncake-store)
• Mooncake Conductor (mooncake-conductor)
• Reshard (mooncake-reshard)
• Mooncake EP (mooncake-ep)
• Mooncake PG (mooncake-pg)
• Integration (mooncake-integration)
• P2P Store (mooncake-p2p-store)
• Python Wheel (mooncake-wheel)
• Common (mooncake-common)
• Mooncake RL (mooncake-rl)
• CI/CD
• Docs
• Other

Type of Change

• Bug fix
• New feature
• Refactor
• Breaking change
• Documentation update
• Performance improvement
• Other

How Has This Been Tested?

Test commands:

# config.cpp + tests/config_test.cpp with GCC 13.1, glog 0.7.1, googletest 1.15.2
g++ -std=c++20 -DGLOG_USE_GLOG_EXPORT -DMOONCAKE_GLOG_HAS_IS_INITIALIZED=1 \
    -I<deps>/include -Imooncake-transfer-engine/include -Imooncake-common/include \
    mooncake-transfer-engine/src/config.cpp mooncake-transfer-engine/tests/config_test.cpp \
    -L<deps>/lib -lgtest_main -lgtest -lglog -pthread -o config_test
./config_test

# Compile check for the worker loop change
g++ -std=c++20 -fsyntax-only -Wall -Wextra -DGLOG_USE_GLOG_EXPORT -DMOONCAKE_GLOG_HAS_IS_INITIALIZED=1 \
    -I<deps>/include -Imooncake-transfer-engine/include -Imooncake-common/include \
    mooncake-transfer-engine/src/transport/rdma_transport/worker_pool.cpp

./scripts/code_format.sh --check --changed-lines --base origin/main
pre-commit run --files $(git diff --name-only --diff-filter=ACMR origin/main...HEAD)

Test results:

• Unit tests pass: config_test 117/117, including the 9 new RdmaWorkerIdleSpinEnvTest cases.
• Integration tests pass (if applicable): not run, because the build environment has no RDMA hardware.
• Manual testing done (describe below)

worker_pool.cpp compiles with no warnings. The format check and the pre-commit hooks pass, and the docs HTML build has no warnings. The change has not yet been measured on RDMA hardware.

Checklist

• I have performed a self-review of my own code
• I have formatted my code using ./scripts/code_format.sh
• I have run pre-commit on the files changed in this PR and all hooks pass
• I have updated the documentation (if applicable)
• I have added tests to prove my changes are effective
• For changes >500 LOC: I have filed an RFC issue (not applicable: +111/-2)

AI Assistance Disclosure

• No AI tools were used
• AI tools were used (specify below)

Claude Code (Claude Opus 5.5) helped trace the CPU usage to this idle spin, drafted the code change, tests, docs and this description, and ran the checks listed above.

🤖 Generated with Claude Code