Skip to content

Commit

Permalink
fix: use path inside of speculator config (huggingface#1935)
Browse files Browse the repository at this point in the history
This PR access the path on the speculator similar to
`MLPSpeculatorHead.load` and `MedusaHeadV1.load`

these changes resolves this error locally when loading a `MedusaHeadV2`
```
TypeError: expected str, bytes or os.PathLike object, not dict
```
  • Loading branch information
drbh authored and yuanwu2017 committed Jul 17, 2024
1 parent b9469a1 commit 57ba035
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions server/text_generation_server/layers/medusa.py
Original file line number Diff line number Diff line change
Expand Up @@ -111,10 +111,10 @@ def __init__(self, config, prefix, weights):
from safetensors import safe_open
import json

speculator = config.speculator
speculator_path = config.speculator["path"]

medusa_config = str(Path(speculator) / "config.json")
filename = str(Path(speculator) / "medusa_lm_head.safetensors")
medusa_config = str(Path(speculator_path) / "config.json")
filename = str(Path(speculator_path) / "medusa_lm_head.safetensors")

with open(medusa_config, "r") as f:
medusa_config = json.load(f)
Expand Down

0 comments on commit 57ba035

Please sign in to comment.