Adapt and test huggingface_hub v1.0.0

[Wauplin]

Test as part of huggingface/huggingface_hub#3340

The main changes in huggingface_hub impacting transformers are:

• Repository removed
• HfFolder removed
• migrated to httpx instead of requests

---

List of changes made in this PR:

• removed all imports that were commented with # for backward compatibility
• replaced requests exceptions by httpx ones in testing utils
• Trainer returns a CommitInfo (which is a subclass of str) => was already the case before but not type annotated
• allow_redirects => renamed follow_redirects in httpx
• proxies => argument is now ignored. Proxies must be set globally via env variable, not in script. If user passes a proxies, a warning is emitted and the process continues (ignoring the proxy)
• all requests.HTTPError are replaced by some huggingface_hub.HfHubHTTPError (which inherits from httpx.HTTPError)
• removed test_dynamic_saving_from_local_repo as it was commented as "# to be removed when huggingface_hub v1 comes out"
• removed all HfFolder occurrences from the tests => they were not relevant anyway
• migrated most "Repository clone from" by snapshot_download and "Repository push_to_hub" by upload_folder
• removed ModelFilter from utils/update_tiny_models.py (to be fair it has been removed since quite some time so this script is currently broken on main)
• switched a lot of requests.get into httpx.get in the code. However:
  • it'd be very beneficial to define a single client for them to benefit from connection pooling => that's out of scope for this PR
  • a LOT of requests calls is happening in ./models/ and test_models/ so I won't change them => it'd be good in transformers v5 to get rid of requests dependency entirely => also out of scope for this PR

---

TODO: (later?)

• adapt utils/create_dummy_models.py => currently using Repository
• remove back manual install of huggingface_hub 1.0.0.rc0 once tokenizers is out
• what to do with KerasCallback? => got deleted
• in v5, all use_auth_token logic must be removed (currently throws warnings)

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ydshieh]

python -m pytest -v tests/models/zoedepth/test_image_processing_zoedepth.py::ZoeDepthImageProcessingTest::test_slow_fast_equivalence

The following failure is related (see below)

 dummy_image = Image.open(

        httpx.get("http://images.cocodataset.org/val2017/000000039769.jpg", follow_redirects=True).content
    )

Is httpx works well with Image.open or in general with image?

The same failure happens many times in the job tests_processors

https://app.circleci.com/pipelines/github/huggingface/transformers/146796/workflows/8cffc93e-ca8e-4fd3-b01b-984033ee4ac1/jobs/1938513/parallel-runs/7?filterBy=FAILED

python -m pytest -v tests/models/zoedepth/test_image_processing_zoedepth.py::ZoeDepthImageProcessingTest::test_slow_fast_equivalence

============================================================================================ FAILURES =============================================================================================
_____________________________________________________________________ ZoeDepthImageProcessingTest.test_slow_fast_equivalence ______________________________________________________________________

self = <tests.models.zoedepth.test_image_processing_zoedepth.ZoeDepthImageProcessingTest testMethod=test_slow_fast_equivalence>

    @require_vision
    @require_torch
    def test_slow_fast_equivalence(self):
        if not self.test_slow_image_processor or not self.test_fast_image_processor:
            self.skipTest(reason="Skipping slow/fast equivalence test")

        if self.image_processing_class is None or self.fast_image_processing_class is None:
            self.skipTest(reason="Skipping slow/fast equivalence test as one of the image processors is not defined")

>       dummy_image = Image.open(
            httpx.get("http://images.cocodataset.org/val2017/000000039769.jpg", follow_redirects=True).content
        )

tests/test_image_processing_common.py:184: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.9/site-packages/PIL/Image.py:3466: in open
    filename = os.path.realpath(os.fspath(fp))
/usr/local/lib/python3.9/posixpath.py:392: in realpath
    path, ok = _joinrealpath(filename[:0], filename, strict, {})
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

path = b''
rest = b'\\)\xa4\x81\x17\xb5\xac\x97\xe9\x96\x98\nZh\x01U@*6\xb1\xdc\x1ew\x00\x0e\x9c\xbfLX\x12\x1ac\x08\xc9\x1d\xf0T\x86\x0b...f\x00\xd3Ff \x10v\xdb\xad\x85\xaf\xd0\x0e\x9b\xe2\x8d\xa7\xd4v``\xba\xbd\xd6s\xfe\xa3\xb3\xb6\xa5\xc3Z\xea\xbf\xff\xd9'
strict = False, seen = {}

    def _joinrealpath(path, rest, strict, seen):
        if isinstance(path, bytes):
            sep = b'/'
            curdir = b'.'
            pardir = b'..'
        else:
            sep = '/'
            curdir = '.'
            pardir = '..'
            getcwd = os.getcwd
        if strict is ALLOW_MISSING:
            ignored_error = FileNotFoundError
        elif strict:
            ignored_error = ()
        else:
            ignored_error = OSError

        maxlinks = None

        if isabs(rest):
            rest = rest[1:]
            path = sep

        while rest:
            name, _, rest = rest.partition(sep)
            if not name or name == curdir:
                # current dir
                continue
            if name == pardir:
                # parent dir
                if path:
                    path, name = split(path)
                    if name == pardir:
                        path = join(path, pardir, pardir)
                else:
                    path = pardir
                continue
            newpath = join(path, name)
            try:
>               st = os.lstat(newpath)
E               ValueError: lstat: embedded null character in path

/usr/local/lib/python3.9/posixpath.py:436: ValueError
===================================================================================== short test summary info =====================================================================================
FAILED tests/models/zoedepth/test_image_processing_zoedepth.py::ZoeDepthImageProcessingTest::test_slow_fast_equivalence - ValueError: lstat: embedded null character in path

[ydshieh]

python -m pytest -v tests/utils/test_tokenization_utils.py::TokenizerUtilTester::test_cached_files_are_used_when_internet_is_down

python -m pytest -v tests/utils/test_tokenization_utils.py::TokenizerUtilTester::test_cached_files_are_used_when_internet_is_down_missing_files

are also related to this PR.

============================================================================================ FAILURES =============================================================================================
______________________________________________________________ TokenizerUtilTester.test_cached_files_are_used_when_internet_is_down _______________________________________________________________

self = <tests.utils.test_tokenization_utils.TokenizerUtilTester testMethod=test_cached_files_are_used_when_internet_is_down>

    def test_cached_files_are_used_when_internet_is_down(self):
        # A mock response for an HTTP head request to emulate server down
        response_mock = mock.Mock()
        response_mock.status_code = 500
        response_mock.headers = {}
        response_mock.raise_for_status.side_effect = httpx.HTTPError("failed")
        response_mock.json.return_value = {}

        # Download this model to make sure it's in the cache.
        _ = BertTokenizer.from_pretrained("hf-internal-testing/tiny-random-bert")

        # Under the mock environment we get a 500 error when trying to reach the tokenizer.
        with mock.patch("httpx.Client.request", return_value=response_mock) as mock_head:
>           _ = BertTokenizer.from_pretrained("hf-internal-testing/tiny-random-bert")

tests/utils/test_tokenization_utils.py:60:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
src/transformers/tokenization_utils_base.py:2043: in from_pretrained
    for template in list_repo_templates(
src/transformers/utils/hub.py:166: in list_repo_templates
    return [
src/transformers/utils/hub.py:166: in <listcomp>
    return [
/usr/local/lib/python3.9/site-packages/huggingface_hub/hf_api.py:3083: in list_repo_tree
    for path_info in paginate(path=tree_url, headers=headers, params={"recursive": recursive, "expand": expand}):
/usr/local/lib/python3.9/site-packages/huggingface_hub/utils/_pagination.py:37: in paginate
    hf_raise_for_status(r)
/usr/local/lib/python3.9/site-packages/huggingface_hub/utils/_http.py:560: in hf_raise_for_status
    response.raise_for_status()
/usr/local/lib/python3.9/unittest/mock.py:1092: in __call__
    return self._mock_call(*args, **kwargs)
/usr/local/lib/python3.9/unittest/mock.py:1096: in _mock_call
    return self._execute_mock_call(*args, **kwargs)
/usr/local/lib/python3.9/unittest/mock.py:1151: in _execute_mock_call
    raise effect
src/transformers/utils/hub.py:477: in cached_files
    hf_hub_download(
/usr/local/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py:89: in _inner_fn
    return fn(*args, **kwargs)
/usr/local/lib/python3.9/site-packages/huggingface_hub/file_download.py:937: in hf_hub_download
    return _hf_hub_download_to_cache_dir(
/usr/local/lib/python3.9/site-packages/huggingface_hub/file_download.py:998: in _hf_hub_download_to_cache_dir
    (url_to_download, etag, commit_hash, expected_size, xet_file_data, head_call_error) = _get_metadata_or_catch_error(
/usr/local/lib/python3.9/site-packages/huggingface_hub/file_download.py:1453: in _get_metadata_or_catch_error
    metadata = get_hf_file_metadata(
/usr/local/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py:89: in _inner_fn
    return fn(*args, **kwargs)
/usr/local/lib/python3.9/site-packages/huggingface_hub/file_download.py:1379: in get_hf_file_metadata
    response = _httpx_follow_relative_redirects(method="HEAD", url=url, headers=hf_headers, timeout=timeout)
/usr/local/lib/python3.9/site-packages/huggingface_hub/file_download.py:266: in _httpx_follow_relative_redirects
    hf_raise_for_status(response)
/usr/local/lib/python3.9/site-packages/huggingface_hub/utils/_http.py:560: in hf_raise_for_status
    response.raise_for_status()
/usr/local/lib/python3.9/unittest/mock.py:1092: in __call__
    return self._mock_call(*args, **kwargs)
/usr/local/lib/python3.9/unittest/mock.py:1096: in _mock_call
    return self._execute_mock_call(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Mock name='mock.raise_for_status' id='140314207904624'>, args = (), kwargs = {}, effect = HTTPError('failed')

    def _execute_mock_call(self, /, *args, **kwargs):
        # separate from _increment_mock_call so that awaited functions are
        # executed separately from their call, also AsyncMock overrides this method

        effect = self.side_effect
        if effect is not None:
            if _is_exception(effect):
>               raise effect
E               httpx.HTTPError: failed

/usr/local/lib/python3.9/unittest/mock.py:1151: HTTPError
======================================================================================== warnings summary =========================================================================================
<frozen importlib._bootstrap>:228
  <frozen importlib._bootstrap>:228: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute

<frozen importlib._bootstrap>:228
  <frozen importlib._bootstrap>:228: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===================================================================================== short test summary info =====================================================================================
FAILED tests/utils/test_tokenization_utils.py::TokenizerUtilTester::test_cached_files_are_used_when_internet_is_down - httpx.HTTPError: failed
================================================================================== 1 failed, 2 warnings in 0.91s ==================================================================================

[ydshieh]

python -m pytest -v tests/extended/test_trainer_ext.py::TestTrainerExt::test_run_seq2seq_no_dist

This pass when I ssh into a CircleCI runner. Could ignore for now and will check later

[ydshieh]

There are some

AttributeError: module 'huggingface_hub.hf_api' has no attribute 'HfFolder'

for example

python -m pytest -v examples/pytorch/test_accelerate_examples.py::ExamplesTestsNoTrainer::test_run_swag_no_trainer

But I'm not able to reproduce when I ssh into a CircleCI runner. Could check later.

[ydshieh]

There are some failing tests (in the complete CI runs) that seems irrelevant at a first glance but actually related surprisingly.

HF_HOME=/mnt/cache RUN_SLOW=1 python3 -m pytest -v tests/generation/test_continuous_batching.py::ContinuousBatchingTest::test_continuous_batching_parity_gemma_eager

fails with 1.0 rc but pass with 0.35.

Add HF_TOKEN=hf_XXX works with 1.0 rc. Could you check this ?

same for

tests/models/gemma/test_modeling_gemma.py::GemmaIntegrationTest::test_model_2b_bf16_dola

[ydshieh]

tests/peft_integration/test_peft_integration.py::PeftIntegrationTester::test_peft_pipeline_no_warning

also fails with this PR

        from transformers import pipeline

        ADAPTER_PATH = "peft-internal-testing/tiny-OPTForCausalLM-lora"
        BASE_PATH = "hf-internal-testing/tiny-random-OPTForCausalLM"

        # Input text for testing
        text = "Who is a Elon Musk?"

        model = AutoModelForCausalLM.from_pretrained(
            BASE_PATH,
            device_map="auto",
        )
        tokenizer = AutoTokenizer.from_pretrained(BASE_PATH)

>       lora_model = PeftModel.from_pretrained(
            model,
            ADAPTER_PATH,
            device_map="auto",
        )

tests/peft_integration/test_peft_integration.py:865:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python3.10/dist-packages/peft/peft_model.py:441: in from_pretrained
    PeftConfig._get_peft_type(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'peft.config.PeftConfig'>, model_id = 'peft-internal-testing/tiny-OPTForCausalLM-lora', hf_hub_download_kwargs = {'cache_dir': None, 'revision': None, 'subfolder': None, 'token': None, ...}, subfolder = None, path = 'peft-internal-testing/tiny-OPTForCausalLM-lora'

    @classmethod
    def _get_peft_type(
        cls,
        model_id: str,
        **hf_hub_download_kwargs,
    ):
        subfolder = hf_hub_download_kwargs.get("subfolder", None)

        path = os.path.join(model_id, subfolder) if subfolder is not None else model_id

        if os.path.isfile(os.path.join(path, CONFIG_NAME)):
            config_file = os.path.join(path, CONFIG_NAME)
        else:
            try:
                config_file = hf_hub_download(
                    model_id,
                    CONFIG_NAME,
                    **hf_hub_download_kwargs,
                )
            except Exception:
>               raise ValueError(f"Can't find '{CONFIG_NAME}' at '{model_id}'")
E               ValueError: Can't find 'adapter_config.json' at 'peft-internal-testing/tiny-OPTForCausalLM-lora'

/usr/local/lib/python3.10/dist-packages/peft/config.py:268: ValueError

[ydshieh]

tests/pipelines/test_pipelines_text_to_audio.py::TextToAudioPipelineTests::test_csm_model_pt

same error as the first and second one

[ydshieh]

if you need to see the job run pages

https://github.com/huggingface/transformers/actions/runs/17923228517/job/50963407968

[ydshieh]

The CI (circleci) against python 3.10 is fine 🎉

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: auto

[ydshieh]

CI looks fine:

• On CircleCI, they pass both on python 3.9 and now 3.10
• on Github Actions (daily scheduled CI), there are only 13 new failing tests compared to a previous run (of the same commit) on the main branch, and they look irrelevant.

The PR is ready to be merged to main 🚀 Let's go

[Wauplin]

Yay! thanks for confirming! 🎉

I will work on the remaining TODOs in separate PRs.