How to offline load deberta-ai4privacy model and spacy en-core-web-sm with llm-guard

[vamshigvk]

I work on an offline Windows machine without internet access. Please help me run llm-guard by copying these models to the offline machine. I tried using .cache folder of huggingface but llm-guard still tries to download the model from huggingface.co and times out.

[vamshigvk]

I already have en-core-web-sm installed on my machine, but it still tries to download it from raw.githubusercontent.com and gets refused to connect error as my machine has no internet.

[jrmueller13]

Hi @vamshigvk, if you want llm-guard to use the local model you need to change the config/scanners.yaml and provide the model_path. See this part of the documentation: Load Models from a Directory

Example config snippet for the Anonymize scanner:

input_scanners:
  - type: Anonymize
    params:
      use_faker: false
      use_onnx: true
      threshold: 0.75
      model_path: "./deberta-v3-base_finetuned_ai4privacy_v2/onnx"