[Feature]: Set RoPE scaling parameters dynamically

[LilianJim]

🚀 The feature, motivation and pitch

As it was implemented in #555, specifying RoPE parameters is only available from the model's config.json, and I haven't found a way to set it dynamically in my code. Is there currently a way of doing this?

Related to #910.

Alternatives

Right now, unless providing a modified config.json (which is very inconvenient in my setup), I haven't found an alternative. I've tried monkey patching vllm.transformers_utils.config.get_config function to no avail (Ray uses it in a way I don't understand).

Additional context

For context - specific to my setup - I'm using 2 GPUs to run a quantized Llama-3-70B (casperhansen's). Thus vLLM is using Ray.