[FEATURE]: more general router

[PeaBrane]

Feature request

The core (backend) of the KvRouter is the KvIndexer written in a way that is incredibly general. In particular, the WorkerId type need not be restricted to i64, and in fact can be abstracted as a generic trait with minimal code change.

The messy bits are in the frontend components + LLM engine facing parts, including: metric publishing per dp-worker (vllm-project/vllm#18925), metric subscription / aggregation that is dp-aware (#1181), and handing said metrics off to the core router components (#1285 )

Describe the problem you're encountering

N/A

Describe alternatives you've tried

No response

[grahamking]

Could you add why you'd want a more general router, some example use cases?

[PeaBrane]

@grahamking Sorry I completely missed this comment.

But the immediate motivation is to support intra-worker DP routing, when each DP worker cannot be easily registered as an individual dynamo endpoint. In which case we need to route to (worker_id, dp_rank) instead of just a worker_id

Eventually (with support from vllm1) we may be able to identify each dp rank as a unique endpoint, then this may not be needed. But either way, I think it does make sense to have a generic KvIndexer at least for future-proofing.

cc: @alec-flowers