Skip to content

Commit

Permalink
Fix formatting and better explain model conversion (Vahe1994#96)
Browse files Browse the repository at this point in the history
  • Loading branch information
justheuristic authored Jun 6, 2024
1 parent 05bdc6f commit de6c1bd
Showing 1 changed file with 7 additions and 5 deletions.
12 changes: 7 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -270,12 +270,14 @@ python lmeval.py \

### Preparing models for inference

To convert a model into a _Hugging Face_ compatible format, use `convert_to_hf.py` with corresponding arguments:
- `--model` - the original pretrained model (corresponds to `MODEL_PATH` of `main.py`, e.g. `meta-llama/Llama-2-7b-hf`).
- `--in_path` - the folder containing an initially quantized model (corresponds to `--save` of `main.py`).
- `--out_path` - the folder to save `transformers` model to.
To convert a model into a _Hugging Face_ compatible format, use `convert_to_hf.py model in_path out_path` with corresponding arguments:
- `model` - the original pretrained model (corresponds to `MODEL_PATH` of `main.py`, e.g. `meta-llama/Llama-2-7b-hf`).
- `in_path` - the folder containing an initially quantized model (corresponds to `--save` of `main.py`).
- `out_path` - the folder to save `transformers` model to.

The conversion automatically
You may also specify flags such as `--save_safetensors` to control the saved model format (see `--help` for details).

Example command: `python convert_to_hf.py meta-llama/Llama-2-7b-hf ./path/to/saved/quantization ./converted-llama2-7b-hf --save_safetensors`

## Contributing

Expand Down

0 comments on commit de6c1bd

Please sign in to comment.