Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: pass missing revision arg for lora adapter when loading multiple… #2510

Merged
merged 1 commit into from
Sep 12, 2024

Conversation

drbh
Copy link
Collaborator

@drbh drbh commented Sep 11, 2024

This PR correctly includes the missing adapter.revision param in load_module_map which should resolve a bug when loading multiple lora adapters and at least on has a revision

@Narsil Narsil merged commit 628334d into main Sep 12, 2024
10 of 11 checks passed
@Narsil Narsil deleted the fix-missing-lora-adapter-revision branch September 12, 2024 15:04
Narsil pushed a commit that referenced this pull request Sep 14, 2024
#2510)

fix: pass missing revision arg for lora adapter when loading multiple adapters
@nbroad1881
Copy link
Contributor

nbroad1881 commented Sep 17, 2024

A customer is still reporting that this doesn't work.

Here is what I tried on Inference Endpoints and the associated error (TGI image taken from 1 hour ago):

from huggingface_hub import create_inference_endpoint

endpoint = create_inference_endpoint(
    "phi-3-lora-revision-test",
    repository="microsoft/Phi-3-mini-4k-instruct",
    framework="pytorch",
    task="text-generation",
    accelerator="gpu",
    vendor="aws",
    region="us-east-1",
    type="protected",
    instance_size="x1",
    instance_type="nvidia-a10g",
    custom_image={
        "health_route": "/health",
        "env": {
            "MAX_BATCH_PREFILL_TOKENS": "2048",
            "MAX_INPUT_LENGTH": "1024",
            "MAX_TOTAL_TOKENS": "1512",
            "MODEL_ID": "/repository",
            "LORA_ADAPTERS": "grounded-ai/phi3-hallucination-judge@5f5f8c0483200db2ceb4db66adfae9ce77273bde"
        },
        "url": "ghcr.io/huggingface/text-generation-inference:sha-ce85efa",
    },
    token="token"
)

error

Exit code: 1. Reason:                                          │
│ /opt/conda/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py │
│ :160 in validate_repo_id                                                     │
│                                                                              │
│   157 │   │   )                                                              │
│   158 │                                                                      │
│   159 │   if not REPO_ID_REGEX.match(repo_id):                               │
│ ❱ 160 │   │   raise HFValidationError(                                       │
│   161 │   │   │   "Repo id must use alphanumeric chars or '-', '_', '.', '-- │
│   162 │   │   │   " forbidden, '-' and '.' cannot start or end the name, max │
│   163 │   │   │   f" '{repo_id}'."                                           │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ repo_id = 'grounded-ai/phi3-hallucination-judge@5f5f8c0483200db2ceb4db6… │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' 
and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 
96: 
'grounded-ai/phi3-hallucination-judge@5f5f8c0483200db2ceb4db66adfae9ce77273bde'.

yuanwu2017 pushed a commit to yuanwu2017/tgi-gaudi that referenced this pull request Sep 26, 2024
huggingface#2510)

fix: pass missing revision arg for lora adapter when loading multiple adapters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants