Skip to content

How to use cpu memory cache kv cache? #309

Open
@zeroorhero

Description

@zeroorhero

Follow the tutorial(https://www.youtube.com/watch?v=SeImiPDVMCw) at this link to use CPU for storing KV cache, but found that the connector name is incorrect. Can you provide a tutorial on using CPU to store KV cache? Also, is SSD storage for KV cache currently supported?

Metadata

Metadata

Assignees

Labels

documentationImprovements or additions to documentation

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions