-
Notifications
You must be signed in to change notification settings - Fork 57
Closed
Labels
Description
Your current environment
Can it support unloading to Dram only? The following configuration reports an error about missing the storage_backends parameter during runtime. How should I configure this? The official documentation does not have relevant guidance. Could you provide an example?
"kv_connector_extra_config": {
"ucm_connectors": [
{
"ucm_connector_name": "UcmDramStore",
"ucm_connector_config": {
"max_cache_size": 53687091200,
"kv_block_size": 262144
}
}
],
"load_only_first_rank": false
}
}
How would you like to use ucm.
I want to run inference of a [specific model](put link here). I don't know how to integrate it with unified.