[Misc] Disable pin_memory in AsyncMetricsCollector for spec decode tensor allocation

[esmeetu]

I encountered a memory issue when using tensor parallelism with speculative decoding. On rank 0, it consumes approximately 1.4GB multiplied by the TP count, which is quite large for a single GPU with limited memory.

Since the _aggregate_num_accepted_tokens and _aggregate_num_emitted_tokens metrics are non-blocking, I believe setting pin_memory to False is a better tradeoff to significantly reduce memory usage.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[WoosukKwon]

Hi @esmeetu, good to see you again!

I don't quite understand your problem here. IIUC, these tensors are CPU tensors, so shouldn't take any GPU memory?

[davidsyoung]

For what it's worth, I was actually also seeing this issue - when setting a speculative decoding config, no matter what I would also see the same behaviour that 1.4GB per TP would be allocated to CUDA0.

This PR fixes that. Thank you!

[esmeetu]

Hi @esmeetu, good to see you again!

I don't quite understand your problem here. IIUC, these tensors are CPU tensors, so shouldn't take any GPU memory?

@WoosukKwon Yeah, busy year. Glad to get your review again.: )

I have same confusion with you, but i think it might like that pytorch precache a buffer in gpu space for the coming data transfer.
Also You can try to define a new tensor with pin_memory=True in a fresh env to reproduce this issue.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @esmeetu.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork