This repository has been archived by the owner on Jun 24, 2024. It is now read-only.
This repository has been archived by the owner on Jun 24, 2024. It is now read-only.
Closed
Description
A couple of months ago, I decided to change the CLI from llm infer -a llama
to llm llama infer
, with the thinking being that:
- there may be model-specific commands
- it's more convenient if you know the model architecture
- it makes it explicit that the architecture is a required parameter
However, I think this was the wrong call in hindsight. The model-specific commands existed for a few days before being removed, and from a pure ergonomics perspective, having to move to the start of the command to change the architecture is more annoying than changing both the architecture and model in the same "area" of the command.
Additionally, #316 / GGUF should make the architecture argument entirely optional, allowing for llm infer -m llama-7b.gguf -p "What's up?"
to Just Work:tm:.