sc-103647-add-sitemap-for-docs-pennylane-ai-catalyst

[ashishks0522]

Add sitemap generation for documentation

Summary

• Added sphinx-sitemap extension to automatically generate a sitemap for the PennyLane catalyst documentation, improving SEO and to reduce number of 404 pages showing up in Google Console.
• marks C++ API pages a no-index to prevent C++ pages from getting indexed in Google.

Changes

• Added sphinx_sitemap to Sphinx extensions in doc/conf.py
• Updated html_baseurl to use proper canonical URL with ReadTheDocs fallback
• Added sitemap_url_scheme configuration for standard sitemap format
• Added sphinx-sitemap dependency to:
  • doc/requirements.txt
  • pyproject.toml (docs optional dependencies)
• Added add_noindex_to_cpp_pages to exclude C++ API pages

Ticket

• sc-103647-add-sitemap-for-docs-pennylane-ai-catalyst

Testing

• Check the sitemap here which includes list of all pages.
• C++ API pages have noindex nofollow tags to exclude them from Google.

[github-actions]

Hello. You may have forgotten to update the changelog!
Please edit doc/releases/changelog-dev.md on your branch with:

• A one-to-two sentence description of the change. You may include a small working example for new features.
• A link back to this PR.
• Your name (or GitHub username) in the contributors section.

[doctorperceptron]

Should get someone from the Catalyst team (@josh146 or @mlxd?) to approve as well.