Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: diversify error mesages when loading a model fails #56

Open
flatsiedatsie opened this issue May 24, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@flatsiedatsie
Copy link
Contributor

flatsiedatsie commented May 24, 2024

Currently a model can fail to load for a number of different reasons. However, the error raised seems to always be a general "failed to load" error. It would be useful if different errors could be raised for:

  • 404 model file not found ( -> inform user to check the URL they provided)
  • Server isn't responding (HuggingFace is down again.. -> load from backup server)
  • Couldn't load model because there is no internet connection (-> suggest loading another model that is cached)
  • Failed to load from cache ( -> clear cache and retry downloading)
  • Couldn't load model because the file doesn't actually seem to be a valid .gguf file ( -> inform user to check the URL they provided)
  • Couldnt load model because it doesn't fit into memory ( -> recommend trying a smaller one)
  • Etc

This is because in my project users can enter their own URL to a .gguf file (or provide a list of shards), so failure can come in many forms.

I can see in the debug console that the worker has precise information (e.g. GET failed), that that precision isn't passed on yet.
Screenshot 2024-05-24 at 12 50 43

Screenshot 2024-05-24 at 12 39 39
@flatsiedatsie
Copy link
Contributor Author

flatsiedatsie commented May 24, 2024

Another exampleI just ran into while trying to load the new version of Phi 3 128K (Q2). I suspect this error is because of a Llama.cpp version mismatch?

Screenshot 2024-05-24 at 21 32 35

@ngxson ngxson added the enhancement New feature or request label Jun 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants