Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REFRACTOR] Cleanup backend and model_type usage #276

Merged
merged 8 commits into from
Jul 23, 2024

Conversation

LRL-ModelCloud
Copy link
Contributor

No description provided.

@LRL-ModelCloud LRL-ModelCloud marked this pull request as ready for review July 23, 2024 13:54
@Qubitium Qubitium changed the title [MISC] clearup backend and model_type usage [REFRACTOR] Cleanup backend and model_type usage Jul 23, 2024
@Qubitium
Copy link
Contributor

Qubitium commented Jul 23, 2024

@LRL-ModelCloud @ZX-ModelCloud I can't merge this yet. This introduces new backend property to BaseModel but the backend prop is not correctly updated to reflect the actual value during runtime.

For example, I load from_quantized model and set backend=auto, but since it is auto (not real), it will be converted to real backend after "auto" checking for compatible backend for this model. Once the backend is selected, the .backend prop value should reflect the real backend and not the incorrect auto.

@Qubitium Qubitium merged commit 9197738 into ModelCloud:main Jul 23, 2024
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants