PSAT CLI run in loop when doing analysis of sitemaps or CSV files

[milindmore22]

Describe the bug
We encountered an issue where the PSAT CLI enters a never-ending loop when processing specific sitemaps or CSV files. This might be due to the CLI failing to detect redirections or handling different redirection HTTP statuses correctly.

My initial findings suggest this may be because of redirection used by websites, but even if they are redirection it should not run in endless loop.

To Reproduce
Steps to reproduce the behavior:

• Open terminal
• Sitemap
  • Use the command npm run cli -- -s https://css-tricks.com/post-sitemap10.xml -n 50
• CSV file: (attached below)
  • npm run cli -- -f /Users/milind/Downloads/http-status.csv
  • Observe if the CLI gets stuck in a loop and doesn't proceed to other URLs.

Expected behavior
The PSAT CLI should process the provided sitemap and CSV file, handle any redirects encountered, and continue processing other URLs in the queue.

Screenshots

Screen.Recording.2024-08-23.at.5.49.59.PM.mov

Desktop (please complete the following information):

• OS: macOS
• Browser : Chrome
• Version : 0.10.1

Additional context
CSV File
http-status.csv