[SpecDec] Fix inputs for speculation and revise past KV trimming#5449

yuanheng-zhao merged 1 commit intohpcaitech:feat/speculative-decodinghpcaitech/ColossalAI:feat/speculative-decodingfrom

yuanheng-zhao:feat/spec-dec/fixyuanheng-zhao/ColossalAI:feat/spec-dec/fixCopy head branch name to clipboard