Add feature to skip already-downloaded problems for improved speed #4
Description
Summary
Add a feature that checks which problems have already been downloaded and skips them when running the scraper. This will make the process faster by avoiding redundant downloads.
Details
- Implement a mechanism to track downloaded problems (e.g., by filename, database, or cache).
- Before downloading a problem, check if it has already been downloaded.
- If already present, skip the download and move to the next problem.
- Ensure this feature works efficiently for large numbers of problems.
Motivation
Currently, the scraper may attempt to download problems that have already been saved, resulting in slower performance. Skipping these will save time and resources.
Acceptance Criteria
- The scraper reliably skips already-downloaded problems.
- No duplicate downloads occur.
- Feature is documented in the README.