Use link rel="canonical" to doc.rust-lang.org in HTML docs

[geofft]

I'm fairly often finding old versions of Rust docs from other sites when I Google things. For instance, searching for 'rust macro_rules' finds me http://web.mit.edu/rust-lang_v0.9/doc/guide-macros.html (as the second result, after the Rust by Example page), and the rust-lang.org page is nowhere to be found. Googling 'ContravariantLifetime' finds me http://www.contrib.andrew.cmu.edu/~acrichto/doc/std/kinds/marker/struct.ContravariantLifetime.html .

One way to avoid this going forward would be for the documentation included in release tarballs to use a <link rel="canonical"> tag pointing to the corresponding doc.rust-lang.org URL for that documentation page. This would allow the pages to be viewed directly without redirects (preserving local use), but any search engine juice would be repointed at the official URL.

See the following pages for discussion on canonical links:

• https://support.google.com/webmasters/answer/139066#2
• https://developer.mozilla.org/en-US/docs/Web/HTTP/www_and_non-www_URLs#Using_link_with_the_attirbute_rel.3D.22canonical.22
• https://www.mattcutts.com/blog/canonical-link-tag/
• http://moz.com/blog/complete-guide-to-rel-canonical-how-to-and-why-not

The other option would be to include <meta name="robots" content="noindex"> in the documentation in tarballs, which would completely prevent them from getting indexed. This tag would of course need to not be in the doc.rust-lang.org version; I don't know if this would be difficult.

I could probably create a PR if that would be helpful, but I'm not at all familiar with the doc generation and release tarball creation process.

[steveklabnik]

We already have an issue about this... But yes, big 👍

[geofft]

I see #9461, #12466, and #14466... maybe I'm misreading, but those seem to be about using <link rel="canonical"> from within doc.rust-lang.org to improve internal SEO when pages appear with multiple URLs (e.g., with /master/ and without).

To clarify, my request is specifically about including it in the documentation distributed outside doc.rust-lang.org (i.e., in the release tarballs and such), so that sites that semi-unintentionally host a copy of the docs point search engines back to doc.rust-lang.org. I don't think that's mentioned in those issues.

Feel free to close if I'm misreading or if there's another issue that I missed that covers this case.

[steveklabnik]

Triage: no change.

[luser]

Duplicate of #9461 ?

[jsha]

I think this issue has largely resolved itself - doc.rust-lang.org now ranks rather high. For instance, [rust macro_rules], the starting example here, now shows doc.rust-lang.org as the top two results.