Add a doc_paragraphs_missing_punctuation lint (#15758)

This introduces a new lint that aims to check for missing terminal
punctuation in doc comments.

## Lint scope and motivation

It partially addresses
#8371 by currently
focusing on the case that is both the easiest to look for without
advanced NLP capacities and still very useful: missing punctuation at
the end of the last sentence of doc comments. This is particularly
useful when working in a team to enforce a style guide and make sure all
doc comments end with some kind of punctuation, which is often forgotten
in the case of short, single-sentence doc comments. The lint is biased
towards avoiding false positives so it can be easily adopted without
requiring an excessive number of `#[expect]` attributes.

It is currently only concerned with Latin languages, but adding support
for `。` for instance should be very easy.

Even if consistently ending doc comments (even very short ones) with
punctuation is somewhat of an opinion, it seems sufficiently
well-established, [at least in the
`std`](rust-lang/rust#91886), to warrant its own
lint, even if of course it would be allow-by-default.

## Lint category and possible evolution

Because it is unclear how useful and more complex it would be to also
look for missing punctuation at the end of *each* Markdown paragraphs, I
opted for the `nursery` category for now. My understanding of [the
stability
guarantees](https://github.com/rust-lang/rfcs/blob/master/text/2476-clippy-uno.md#stability-guarantees)
is that would it not be acceptable to add such capability later if it
was part of `pedantic` or `restriction` categories right away. I tried
to make sure the lint name allows this kind of iterative evolution.

## Testing

I ran the lint against the `core` library which led me to introduce a
few special cases (which seem common enough) to avoid false positives:
e.g., the last sentence being in parentheses. After excluding specific
modules which should likely be excluded (e.g., `core_arch`,
`intrinsincs`), it currently finds a bit more than 200 errors in the
`core` library.

This lint also helped find issues with a few doc comments, which [are
now fixed](rust-lang/rust#146136).

## How to review this PR

I suppose it is easier to have a look at the UI tests first before
checking the implementation.

---

*Please write a short comment explaining your change (or "none" for
internal only changes)*

changelog: add new lint [`doc_paragraphs_missing_punctuation`]

Author: Alexendoo

CHANGELOG.md

@@ -6314,6 +6314,7 @@ Released 2018-09-13
 [`doc_markdown`]: https://rust-lang.github.io/rust-clippy/master/index.html#doc_markdown
 [`doc_nested_refdefs`]: https://rust-lang.github.io/rust-clippy/master/index.html#doc_nested_refdefs
 [`doc_overindented_list_items`]: https://rust-lang.github.io/rust-clippy/master/index.html#doc_overindented_list_items
+[`doc_paragraphs_missing_punctuation`]: https://rust-lang.github.io/rust-clippy/master/index.html#doc_paragraphs_missing_punctuation
 [`doc_suspicious_footnotes`]: https://rust-lang.github.io/rust-clippy/master/index.html#doc_suspicious_footnotes
 [`double_comparisons`]: https://rust-lang.github.io/rust-clippy/master/index.html#double_comparisons
 [`double_ended_iterator_last`]: https://rust-lang.github.io/rust-clippy/master/index.html#double_ended_iterator_last

clippy_lints/src/declared_lints.rs

@@ -118,6 +118,7 @@ pub static LINTS: &[&::declare_clippy_lint::LintInfo] = &[
     crate::doc::DOC_MARKDOWN_INFO,
     crate::doc::DOC_NESTED_REFDEFS_INFO,
     crate::doc::DOC_OVERINDENTED_LIST_ITEMS_INFO,
+    crate::doc::DOC_PARAGRAPHS_MISSING_PUNCTUATION_INFO,
     crate::doc::DOC_SUSPICIOUS_FOOTNOTES_INFO,
     crate::doc::EMPTY_DOCS_INFO,
     crate::doc::MISSING_ERRORS_DOC_INFO,

clippy_lints/src/doc/doc_paragraphs_missing_punctuation.rs

@@ -0,0 +1,126 @@
+use rustc_errors::Applicability;
+use rustc_lint::LateContext;
+use rustc_resolve::rustdoc::main_body_opts;
+
+use rustc_resolve::rustdoc::pulldown_cmark::{Event, Options, Parser, Tag, TagEnd};
+
+use super::{DOC_PARAGRAPHS_MISSING_PUNCTUATION, Fragments};
+
+const MSG: &str = "doc paragraphs should end with a terminal punctuation mark";
+const PUNCTUATION_SUGGESTION: char = '.';
+
+pub fn check(cx: &LateContext<'_>, doc: &str, fragments: Fragments<'_>) {
+    for missing_punctuation in is_missing_punctuation(doc) {
+        match missing_punctuation {
+            MissingPunctuation::Fixable(offset) => {
+                // This ignores `#[doc]` attributes, which we do not handle.
+                if let Some(span) = fragments.span(cx, offset..offset) {
+                    clippy_utils::diagnostics::span_lint_and_sugg(
+                        cx,
+                        DOC_PARAGRAPHS_MISSING_PUNCTUATION,
+                        span,
+                        MSG,
+                        "end the paragraph with some punctuation",
+                        PUNCTUATION_SUGGESTION.to_string(),
+                        Applicability::MaybeIncorrect,
+                    );
+                }
+            },
+            MissingPunctuation::Unfixable(offset) => {
+                // This ignores `#[doc]` attributes, which we do not handle.
+                if let Some(span) = fragments.span(cx, offset..offset) {
+                    clippy_utils::diagnostics::span_lint_and_help(
+                        cx,
+                        DOC_PARAGRAPHS_MISSING_PUNCTUATION,
+                        span,
+                        MSG,
+                        None,
+                        "end the paragraph with some punctuation",
+                    );
+                }
+            },
+        }
+    }
+}
+
+#[must_use]
+/// If punctuation is missing, returns the offset where new punctuation should be inserted.
+fn is_missing_punctuation(doc_string: &str) -> Vec<MissingPunctuation> {
+    // The colon is not exactly a terminal punctuation mark, but this is required for paragraphs that
+    // introduce a table or a list for example.
+    const TERMINAL_PUNCTUATION_MARKS: &[char] = &['.', '?', '!', '…', ':'];
+
+    let mut no_report_depth = 0;
+    let mut missing_punctuation = Vec::new();
+    let mut current_paragraph = None;
+
+    for (event, offset) in
+        Parser::new_ext(doc_string, main_body_opts() - Options::ENABLE_SMART_PUNCTUATION).into_offset_iter()
+    {
+        match event {
+            Event::Start(
+                Tag::CodeBlock(..)
+                | Tag::FootnoteDefinition(_)
+                | Tag::Heading { .. }
+                | Tag::HtmlBlock
+                | Tag::List(..)
+                | Tag::Table(_),
+            ) => {
+                no_report_depth += 1;
+            },
+            Event::End(TagEnd::FootnoteDefinition) => {
+                no_report_depth -= 1;
+            },
+            Event::End(
+                TagEnd::CodeBlock | TagEnd::Heading(_) | TagEnd::HtmlBlock | TagEnd::List(_) | TagEnd::Table,
+            ) => {
+                no_report_depth -= 1;
+                current_paragraph = None;
+            },
+            Event::InlineHtml(_) | Event::Start(Tag::Image { .. }) | Event::End(TagEnd::Image) => {
+                current_paragraph = None;
+            },
+            Event::End(TagEnd::Paragraph) => {
+                if let Some(mp) = current_paragraph {
+                    missing_punctuation.push(mp);
+                }
+            },
+            Event::Code(..) | Event::Start(Tag::Link { .. }) | Event::End(TagEnd::Link)
+                if no_report_depth == 0 && !offset.is_empty() =>
+            {
+                if doc_string[..offset.end]
+                    .trim_end()
+                    .ends_with(TERMINAL_PUNCTUATION_MARKS)
+                {
+                    current_paragraph = None;
+                } else {
+                    current_paragraph = Some(MissingPunctuation::Fixable(offset.end));
+                }
+            },
+            Event::Text(..) if no_report_depth == 0 && !offset.is_empty() => {
+                let trimmed = doc_string[..offset.end].trim_end();
+                if trimmed.ends_with(TERMINAL_PUNCTUATION_MARKS) {
+                    current_paragraph = None;
+                } else if let Some(t) = trimmed.strip_suffix(|c| c == ')' || c == '"') {
+                    if t.ends_with(TERMINAL_PUNCTUATION_MARKS) {
+                        // Avoid false positives.
+                        current_paragraph = None;
+                    } else {
+                        current_paragraph = Some(MissingPunctuation::Unfixable(offset.end));
+                    }
+                } else {
+                    current_paragraph = Some(MissingPunctuation::Fixable(offset.end));
+                }
+            },
+            _ => {},
+        }
+    }
+
+    missing_punctuation
+}
+
+#[derive(Debug, Copy, Clone, PartialEq, Eq)]
+enum MissingPunctuation {
+    Fixable(usize),
+    Unfixable(usize),
+}

clippy_lints/src/doc/mod.rs

@@ -27,6 +27,7 @@ use url::Url;
 
 mod broken_link;
 mod doc_comment_double_space_linebreaks;
+mod doc_paragraphs_missing_punctuation;
 mod doc_suspicious_footnotes;
 mod include_in_doc_without_cfg;
 mod lazy_continuation;
@@ -670,6 +671,33 @@ declare_clippy_lint! {
     "looks like a link or footnote ref, but with no definition"
 }
 
+declare_clippy_lint! {
+    /// ### What it does
+    /// Checks for doc comments whose paragraphs do not end with a period or another punctuation mark.
+    /// Various Markdowns constructs are taken into account to avoid false positives.
+    ///
+    /// ### Why is this bad?
+    /// A project may wish to enforce consistent doc comments by making sure paragraphs end with a
+    /// punctuation mark.
+    ///
+    /// ### Example
+    /// ```no_run
+    /// /// Returns a random number
+    /// ///
+    /// /// It was chosen by a fair dice roll
+    /// ```
+    /// Use instead:
+    /// ```no_run
+    /// /// Returns a random number.
+    /// ///
+    /// /// It was chosen by a fair dice roll.
+    /// ```
+    #[clippy::version = "1.93.0"]
+    pub DOC_PARAGRAPHS_MISSING_PUNCTUATION,
+    restriction,
+    "missing terminal punctuation in doc comments"
+}
+
 pub struct Documentation {
     valid_idents: FxHashSet<String>,
     check_private_items: bool,
@@ -704,6 +732,7 @@ impl_lint_pass!(Documentation => [
     DOC_INCLUDE_WITHOUT_CFG,
     DOC_COMMENT_DOUBLE_SPACE_LINEBREAKS,
     DOC_SUSPICIOUS_FOOTNOTES,
+    DOC_PARAGRAPHS_MISSING_PUNCTUATION,
 ]);
 
 impl EarlyLintPass for Documentation {
@@ -875,6 +904,15 @@ fn check_attrs(cx: &LateContext<'_>, valid_idents: &FxHashSet<String>, attrs: &[
         },
     );
 
+    doc_paragraphs_missing_punctuation::check(
+        cx,
+        &doc,
+        Fragments {
+            doc: &doc,
+            fragments: &fragments,
+        },
+    );
+
     // NOTE: check_doc uses it own cb function,
     // to avoid causing duplicated diagnostics for the broken link checker.
     let mut full_fake_broken_link_callback = |bl: BrokenLink<'_>| -> Option<(CowStr<'_>, CowStr<'_>)> {

tests/ui/blanket_clippy_restriction_lints.rs

@@ -3,7 +3,7 @@
 
 #![warn(clippy::blanket_clippy_restriction_lints)]
 
-//! Test that the whole restriction group is not enabled
+//! Test that the whole restriction group is not enabled.
 #![warn(clippy::restriction)]
 //~^ blanket_clippy_restriction_lints
 #![deny(clippy::restriction)]

tests/ui/doc/doc_paragraphs_missing_punctuation.fixed

@@ -0,0 +1,172 @@
+#![feature(custom_inner_attributes)]
+#![rustfmt::skip]
+#![warn(clippy::doc_paragraphs_missing_punctuation)]
+
+/// Returns the Answer to the Ultimate Question of Life, the Universe, and Everything.
+//~^ doc_paragraphs_missing_punctuation
+fn answer() -> i32 {
+    42
+}
+
+/// The `Option` type.
+//~^ doc_paragraphs_missing_punctuation
+// Triggers even in the presence of another attribute.
+#[derive(Debug)]
+enum MyOption<T> {
+    /// No value.
+    //~^ doc_paragraphs_missing_punctuation
+    None,
+    /// Some value of type `T`.
+    Some(T),
+}
+
+// Triggers correctly even when interleaved with other attributes.
+/// A multiline
+#[derive(Debug)]
+/// doc comment:
+/// only the last line triggers the lint.
+//~^ doc_paragraphs_missing_punctuation
+enum Exceptions {
+    /// Question marks are fine?
+    QuestionMark,
+    /// Exclamation marks are fine!
+    ExclamationMark,
+    /// Ellipses are ok too…
+    Ellipsis,
+    /// HTML content is however not checked:
+    /// <em>Raw HTML is allowed as well</em>
+    RawHtml,
+    /// The raw HTML exception actually does the right thing to autolinks:
+    /// <https://spec.commonmark.org/0.31.2/#autolinks>.
+    //~^ doc_paragraphs_missing_punctuation
+    MarkdownAutolink,
+    /// This table introduction ends with a colon:
+    ///
+    /// | Exception      | Note  |
+    /// | -------------- | ----- |
+    /// | Markdown table | A-ok  |
+    MarkdownTable,
+    /// Here is a snippet.
+    //~^ doc_paragraphs_missing_punctuation
+    ///
+    /// ```
+    /// // Code blocks are no issues.
+    /// ```
+    CodeBlock,
+}
+
+// Check the lint can be expected on a whole enum at once.
+#[expect(clippy::doc_paragraphs_missing_punctuation)]
+enum Char {
+    /// U+0000
+    Null,
+    /// U+0001
+    StartOfHeading,
+}
+
+// Check the lint can be expected on a single variant without affecting others.
+enum Char2 {
+    #[expect(clippy::doc_paragraphs_missing_punctuation)]
+    /// U+0000
+    Null,
+    /// U+0001.
+    //~^ doc_paragraphs_missing_punctuation
+    StartOfHeading,
+}
+
+mod module {
+    //! Works on
+    //! inner attributes too.
+    //~^ doc_paragraphs_missing_punctuation
+}
+
+enum Trailers {
+    /// Sometimes the last sentence ends with parentheses (and that's ok).
+    ParensPassing,
+    /// (Sometimes the last sentence is in parentheses.)
+    SentenceInParensPassing,
+    /// **Sometimes the last sentence is in bold, and that's ok.**
+    DoubleStarPassing,
+    /// **But sometimes it is missing a period.**
+    //~^ doc_paragraphs_missing_punctuation
+    DoubleStarFailing,
+    /// _Sometimes the last sentence is in italics, and that's ok._
+    UnderscorePassing,
+    /// _But sometimes it is missing a period._
+    //~^ doc_paragraphs_missing_punctuation
+    UnderscoreFailing,
+    /// This comment ends with "a quote."
+    AmericanStyleQuotePassing,
+    /// This comment ends with "a quote".
+    BritishStyleQuotePassing,
+}
+
+/// Doc comments can end with an [inline link](#anchor).
+//~^ doc_paragraphs_missing_punctuation
+struct InlineLink;
+
+/// Some doc comments contain [link reference definitions][spec].
+//~^ doc_paragraphs_missing_punctuation
+///
+/// [spec]: https://spec.commonmark.org/0.31.2/#link-reference-definitions
+struct LinkRefDefinition;
+
+// List items do not always need to end with a period.
+enum UnorderedLists {
+    /// This list has an introductory sentence:
+    ///
+    /// - A list item
+    Dash,
+    /// + A list item
+    Plus,
+    /// * A list item
+    Star,
+}
+
+enum OrderedLists {
+    /// 1. A list item
+    Dot,
+    /// 42) A list item
+    Paren,
+}
+
+/// Doc comments with trailing blank lines are supported.
+//~^ doc_paragraphs_missing_punctuation
+///
+struct TrailingBlankLine;
+
+/// This doc comment has multiple paragraph.
+/// This first paragraph is missing punctuation.
+//~^ doc_paragraphs_missing_punctuation
+///
+/// The second one as well
+/// And it has multiple sentences.
+//~^ doc_paragraphs_missing_punctuation
+///
+/// Same for this third and last one.
+//~^ doc_paragraphs_missing_punctuation
+struct MultiParagraphDocComment;
+
+/// ```
+struct IncompleteBlockCode;
+
+/// This ends with a code `span`.
+//~^ doc_paragraphs_missing_punctuation
+struct CodeSpan;
+
+#[expect(clippy::empty_docs)]
+///
+struct EmptyDocComment;
+
+/**
+ * Block doc comments work.
+ *
+ */
+//~^^^ doc_paragraphs_missing_punctuation
+struct BlockDocComment;
+
+/// Sometimes a doc attribute is used for concatenation
+/// ```
+#[doc = ""]
+/// ```
+struct DocAttribute;