Skip to content

Conversation

@wirthual
Copy link
Collaborator

@wirthual wirthual commented Jun 20, 2025

Attempt to reduce traffic to huggingface to get rid of errors like:

 huggingface_hub.errors.HfHubHTTPError: 429 Client Error: Too Many Requests for url: https://huggingface.co/api/models/mixedbread-ai/mxbai-rerank-xsmall-v1/tree/main?recursive=True&expand=False

Attempt is to cache the hf models to avoid redownload every run.

This adds some support for caching huggingface models for the CI pipeline:

  • Run the tests for python3.12 and cache the HF_HOME/hub folder during this run for each of the tests
  • After that, run the tests for python 3.9 - 3.11 and use the cache which was created in the step above

This should reduce the required download of the models.

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

Modifies GitHub Actions workflow to address HuggingFace API rate limiting issues by implementing model caching while reducing Python version test matrix.

  • Added caching for HuggingFace models in .github/workflows/test.yaml to prevent repeated downloads across CI runs
  • Reduced Python version testing from multiple versions (3.9-3.11) to only Python 3.12, which significantly decreases test coverage
  • Removed macOS-specific exclusions from test matrix as macOS testing was already commented out
  • These changes may help resolve the 429 Too Many Requests errors but introduce potential version compatibility blind spots

1 file reviewed, no comments
Edit PR Review Bot Settings | Greptile

@codecov-commenter
Copy link

codecov-commenter commented Jun 20, 2025

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.42%. Comparing base (644bb37) to head (b0fc189).

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #602      +/-   ##
==========================================
- Coverage   79.85%   79.42%   -0.43%     
==========================================
  Files          43       43              
  Lines        3489     3489              
==========================================
- Hits         2786     2771      -15     
- Misses        703      718      +15     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@wirthual
Copy link
Collaborator Author

This adds some support for caching huggingface models for the CI pipeline:

  • Run the tests for python3.12 and cache the HF_HOME/hub folder during this run for each of the tests
  • After that, run the tests for python 3.9 - 3.11 and use the cache which was created in the step above

This should reduce the required download of the models.

@wirthual wirthual requested a review from michaelfeil June 20, 2025 23:43
@wirthual wirthual merged commit 6b2dee3 into main Jun 21, 2025
36 checks passed
@wirthual wirthual deleted the add-cache branch June 21, 2025 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants