E10: bulk ranged WARC fetch by location

[tamnd]

Spec: notes/Spec/2115/07-enhancements.md, item E10.

Problem

columnar locations gives us filename, offset and length, and fetch - consumes them, but one at a time with no batching, no grouping by file and no concurrency shaped for the job. The gao recovery pass is tens of millions of ranged GETs, and issuing them in index order means random access across thousands of WARC files.

What to do

• Add ccrawl fetch --batch, which reads locations on stdin, sorts by filename and offset, and coalesces adjacent ranges within a gap threshold (default 1 MB) into single multi record GETs.
• Dispatch with a per file worker pool under the shared throttle.
• Add --order input|file so callers can choose between input order and file order output.
• Resume from a completed locations ledger, same shape as the publish ledgers.

Done when

• 1M locations fetched with at least 3x fewer HTTP requests than the one at a time path
• a SIGKILL mid run resumes without refetching
• record output is identical to the one at a time path

Estimate: 3 days. Depends on E7 for the S3 side to be worth anything.