feat(transfer_engine_bench): Add multi-GPU support

[staryxchen]

Summary

This PR enhances the mooncake transfer engine benchmark to support multi-GPU configurations and improves memory allocation logic for better performance demonstration.

Change

• Added support for using all available GPUs: When gpu_id=-1, the benchmark now automatically detects and utilizes all available GPUs
• Dynamic buffer allocation: The number of buffers now scales with the number of GPUs when using VRAM
• Enhanced logging: Added detailed logging for GPU memory allocation to aid in debugging and monitoring

Usage

• Single GPU: --gpu_id=0 (existing behavior)
• Multi-GPU: --gpu_id=-1 (new feature - uses all available GPUs)

Testing Result

The changes maintain full backward compatibility with existing single-GPU configurations while enabling multi-GPU scenarios for enhanced performance testing. I have test the performance with the below command:

• target:

./transfer_engine_bench --mode=target --metadata_server=redis://${redis_ip}:${redis_port} --local_server_name=${local_ip} --use_vram=true --block_size=$((1*1024*1024)) --threads=16 --auto_discovery --gpu_id=-1

• initiator:

./transfer_engine_bench --metadata_server=redis://${redis_ip}:${redis_port} --local_server_name=${local_ip} segment_id=${target_ip} --block_size=$((1*1024*1024)) --operation=write --duration=30 --threads=16 --auto_discovery --use_vram=true --gpu_id=-1

And the result show that When allocating memory from eight H20 GPUs simultaneously (which is the purpose of this PR), the Mooncake Transfer Engine can fully utilize the bandwidth of eight 400Gbps RDMA NIC

[alogfans]

LGTM