Support KV cache quantization, especially for vLLM inference

[xin3he]

• Capture and dump the kv scale Add Static FP8 KV Support #737
• Update the config to align with LLMC

[xin3he]

From what I got from Tai, you already supported KV cache quantization in FP8 vLLM inference,
@yiliu30 Do we need any change in the AutoRound part?

[yiliu30]

From what I got from Tai, you already supported KV cache quantization in FP8 vLLM inference, @yiliu30 Do we need any change in the AutoRound part?

Yes, this is already supported, please refer to #737.
We'll need to update the config to align with llm compressor when saving the model. Please keep this issue open for now.