HF_HUB_DISABLE_XET not disabling XET-based downloads

[devtobi]

Describe the bug

When using the huggingface_hub library version 0.34.1 disabling the XET-based downloads using the environment variable HF_HUB_DISABLE_XET is not working. Additionally the progress bar shown in the log shows completely wrong download speeds while the system itself downloads the model at a much higher speed. Then suddenly the progress bar jumps from a few percents almost instantly to 100%.

Using huggingface_hub in version 0.33.5 this problem does not occur.

Reproduction

Call the hf_hub_download() function while environment variable HF_HUB_DISABLE_XET is set to 1

Logs

Running with `HF_DEBUG=1` did not produce any relevant output.

System info

- huggingface_hub version: 0.34.1
- Platform: macOS-15.5-arm64-arm-64bit-Mach-O
- Python version: 3.13.5
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Running in Google Colab Enterprise ?: No
- Token path ?: /Users/devtobi/.cache/huggingface/token
- Has saved token ?: False
- Configured git credential helpers: osxkeychain
- FastAI: N/A
- Tensorflow: N/A
- Torch: N/A
- Jinja2: 3.1.6
- Graphviz: N/A
- keras: N/A
- Pydot: N/A
- Pillow: 11.2.1
- hf_transfer: N/A
- gradio: N/A
- tensorboard: N/A
- numpy: 2.3.0
- pydantic: N/A
- aiohttp: 3.12.13
- hf_xet: 1.1.4
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: /Users/devtobi/.cache/huggingface/hub
- HF_ASSETS_CACHE: /Users/devtobi/.cache/huggingface/assets
- HF_TOKEN_PATH: /Users/devtobi/.cache/huggingface/token
- HF_STORED_TOKENS_PATH: /Users/devtobi/.cache/huggingface/stored_tokens
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10

[hanouticelina]

Hi @devtobi,

Sorry about that. I can't reproduce the issue on my end. Could you provide more details on how you're setting the environment variable?
Note that the HF_HUB_DISABLE_XET variable needs to be set before importing hf_hub_download because the env variable is read during module initialization. If you set it after the import, it won t have any effect.

[devtobi]

Hi @devtobi,

Sorry about that. I can't reproduce the issue on my end. Could you provide more details on how you're setting the environment variable? Note that the HF_HUB_DISABLE_XET variable needs to be set before importing hf_hub_download because the env variable is read during module initialization. If you set it after the import, it won t have any effect.

I am loading the environment variables from a .env file using https://github.com/theskumar/python-dotenv by calling the load_dotenv function as the first part of my python script. (before calling the hf_hub_download function).
Since it worked like a charm prior to 0.34.x is there some way how I can check whether the variable is respected during the execution of the hf_hub_download function?

[chaitanya-dea]

I am also facing similar issue. I set the environment variable like below
os.environ["HF_HUB_DISABLE_XET"] = "1"
This worked earlier but now its not working.
Name: huggingface-hub
Version: 0.34.2

[hanouticelina]

Hello @devtobi and @chaitanya-dea,
would it be possible for each of you to provide a small script that reproduces the issue on your side?

[devtobi]

I created a minimal example explaining the use of dotenv I described above at https://github.com/devtobi/huggingface-xet-example.
I pointed the HF cache to a local repository directory to not interfere with your own cache.

When I execute the script:

• 0.33.5 -> xet folder is not created
• 0.34.3 -> xet folder is created and thus the environment variable is not respected

[jsulz]

Hey @hanouticelina I believe this is because of #3206

Prior to this PR, HF_HUB_DISABLE_XET was lazily evaluated, so the timing of the environment variable import wasn't as important. After that PR, HF_HUB_DISABLE_XET was computed at import. This is why the example from @devtobi shows huggingface_hub "ignoring" the import (edit: basically what you said here now that I read this thread more closely 😅 ).

If the imports in https://github.com/devtobi/huggingface-xet-example/blob/main/main.py were written like so:

from dotenv import load_dotenv
from os import getenv
# Load .env now before the huggingface_hub import
load_dotenv()

from huggingface_hub import hf_hub_download

Then the environment variable would be respected.

[devtobi]

Hey @hanouticelina I believe this is because of #3206

Prior to this PR, HF_HUB_DISABLE_XET was lazily evaluated, so the timing of the environment variable import wasn't as important. After that PR, HF_HUB_DISABLE_XET was computed at import. This is why the example from @devtobi shows huggingface_hub "ignoring" the import.

If the imports in devtobi/huggingface-xet-example@main/main.py were written like so:

from dotenv import load_dotenv
from os import getenv

Load .env now before the huggingface_hub import

load_dotenv()

from huggingface_hub import hf_hub_download
Then the environment variable would be respected.

This seems to make sense, I am not a python guru but I just don't like the looks of the code mixing import statements and method calls. However I am coming from a JavaScript/Java background so that might be my own kind of issue.. 😄

Anyways why was the change to computing on import necessary instead of lazily evaluating as before?