Support huggingface_hub v0.x and v1.x

[Wauplin]

Related to huggingface/huggingface_hub#3340.

This PR adapts datasets to be compatible with both huggingface_hub v0.x and v1.x.

In practice nothing else should change (I've checked the codebase). The HfHubHTTPError is a base error defined in huggingface_hub that inherits from requests.HTTPError in v0.x and will inherit from httpx.HTTPError in v1.x. It has been introduced ~2 years ago so it's fine to use it right now (i.e. no need to wait for v1.x release or bump minimal version).

Most of the changes have been around the test suite to make sure that tests are passing with both requests and httpx backends. Mid-term it would be good to completely remove the requests dependency from datasets but that's an orthogonal topic.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Wauplin]

Bumping to 0.25.0 to be able to use update_repo_settings in tests. Honestly not a big breaking change IMO (dates back to Sept. 2024)

[Wauplin]

@lhoestq could you have a look at this PR please? It makes datasets compatible with the upcoming huggingface_hub v1.0 release while staying compatible with 0.x. Look at PR description for more details.

The CI is currently failing because of 429 rate limit errors but otherwise everything should be fine (made extensive local and ci tests to ensure that). Let me know if you notice anything weird. PR is ready to be merged "as-is" in my opinion.

[lhoestq]

lgtm !