feat: implement retry mechanism for dataset downloads to handle rate limits

[samiuc]

• Updated to use the versioned Omnidoc Bench dataset to fix test failures caused by format changes in the original dataset. (https://huggingface.co/datasets/opendatalab/OmniDocBench/tree/main)
• Added exponential backoff to handle Hugging Face rate limiting errors (1000 requests per 5 minutes).

[github-actions]

✅ DCO Check Passed

Thanks @samiuc, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[cau-git]

@samiuc This looks nice!Just one question: Do we need the max_workers thing?

[samiuc]

This looks nice!Just one question: Do we need the max_workers thing?

Thanks! The max_workers parameter isn’t strictly required, it’s optional. It just helps control concurrency when multiple tests are running in parallel