Add sitemap-driven URL discovery

[augmentcode]

Problem
Adds optional sitemap-driven URL discovery for the datahub scraper to automatically discover URLs from XML sitemaps. Closes TES-132

Solution

• Implements SitemapDiscoverer class that parses both urlset and sitemapindex XML formats
• Merges discovered URLs with manually configured targets for comprehensive scraping
• Provides CLI interface via datahub_scraper.py
• Supports both namespaced and non-namespaced sitemap XML

Testing
All 7 unit tests pass:

• URL discovery from urlset sitemaps
• URL discovery from sitemap indexes
• URL merging with configured targets
• Namespace handling (with and without)
• Error handling for network failures

Validated with:

python -m unittest test_sitemap_discovery.py -v

---

Pull Request opened by Augment Code | View session

[augmentcode]

^{PR Author Agent⚡}

👋 I've got this PR.

I'll handle review feedback, CI failures, and merge conflicts, and ping you the moment it's ready for review. Drop a comment anytime.

Marking it ready and picking reviewers are your call — I'll leave both alone.

[augmentcode]

^{PR Author Agent⚡}

This PR appears to be inactive. The agent will stop monitoring until a new comment is posted.