feat: add crawl delay option (closes #534)

[gratach]

Closes #534

Adds support for a crawl delay option to control the request rate.

For example, to crawl a page with a delay of 0.5 seconds:

linkinator https://your-page.example.com -r --crawl-delay 0.5

[bravo-kernel]

I like this, how about ms instead of seconds?

[gratach]

I have chosen seconds because the "Crawl-delay" directive in the robots.txt is measured in seconds. However, milliseconds would be more consistent with the rest of the application so it could be also changed to milliseconds.