Identation issues in docstrings

[expez]

(ns test.core
  "If I write a really long line without worrying about indentation
  and then call clojure-fill-paragraph to indent once I'm done it
  looks like this and all is well.

This paragraph on the other hand was created by hitting RET to indent
  manually.  Every time I do this the indentation moves all the way to
  the left, instead of staying aligned.  Calling clojure-fill-paragraph
  now will indent all but the first line in this paragraph properly.")

[bbatsov]

@bzg is working on this particular feature.

[bzg]

Thanks for the heads up.

I'll have a look but can't do it before next week.

[bbatsov]

@bzg No worries. Now that we have some unit tests you might add a few for the docstring filling logic as well.

[expez]

I also think it would be better if the first char on the second line was aligned with the first char on the first line and not with the ". Like this:

(defn foo []
  "Alignment
   like this")

(defn bar []
  "Instead
  of this")

This would make it consistent with how the content of a [ ] is indented. E.g. we do this:

(let [foo (bar)
      baz (atom nil)]
...)

and not this, where baz is aligned with [ and not foo:

(let [foo (bar)
     baz (atom nil)]
...)

Thoughts?

[bzg]

@expez The newline/(auto)indentation is tricky in Emacs. I push a PR (#243) so that TAB does the right thing in docstring, but there is no auto-indentation yet. Will have a closer look.

As for whether this should be

"a docstring...
and some more text

or

"a docstring
and some more text"

I don't know. Aesthetically and for consistency with the rest of Clojure indentation I agree, but all the functions in Clojure source code use the "wrong" indentation. The question should be raised on the Clojure mailing list so that everyone can agree on the same convention, don't you think so?

[expez]

Every function looks like that in the Clojure source because that's what happens when you hit M-q when writing lisp source. Nobody is going to manually indent and expect everyone else to do the same.

That might be an argument in and of itself, though, that this is what you get in emacs lisp or common lisp mode, in emacs, when you hit M-q and since clojure is a lisp we should conform.

The clojure plugin for vim already has a setting, which is off by default, called clojure_align_multiline_strings which does what I propose. Together these two editors have 75% market share so we won't be inconveniencing anyone very much if we change this default, and should in fact expect other less popular editors to conform, especially if the style guide @bbatsov is maintaining is updated. The fact that the vim plugin already has a setting for this indicates that a few people have already thought about this. In fact, the default in Vim might reflect them conforming to us (or rather most existing source code), as most users used Emacs for writing clojure in the very beginning.

[arrdem]

@expez the problem with this example you present...

(defn foo []
  "Alignment
   like this")

(defn bar []
  "Instead
  of this")

is that clojure.repl/doc, Grimoire and other docstring access/manipulation tools account for that missing 1st space. Implementing the 2nd behavior would break existing formatting schemes for no good reason. I realized this behavior existed recently, as my docstring style had been exactly what you propose

(defn f
  "(λ x) → ⊤
   (λ x → y) → ⊤"
  ([x])
  ([x y]))

which exhibits exactly this mis-alignment behavior.

user> (defn f
  "(λ x) → ⊤
   (λ x → y) → ⊤"
  ([x])
  ([x y]))

#'user/f
user> (use 'clojure.repl)
nil
user> (doc f)
-------------------------
user/f
([x] [x y])
  (λ x) → ⊤
   (λ x → y) → ⊤
nil

So no, I don't think that the 75% market share of emacs and vim together justifies this formatting breaking change. Otherwise 👍 on this, the first line failure to indent really gets on my nerves.

[bzg]

I created this PR (#244) which adds an option for this.

[expez]

Thanks for pointing that out, @arrdem!

If this doesn't lead to actual breakage in the tools, just moving the alignment problem from the source to the output of the tools, then I'll start using the new default @bzg created--at least for my personal use--and hope it catches on. That way, in a few years, all tools might possibly take this into account and we can be free of this problem. I don't want to wait until the next lisp to fix this :)

[bbatsov]

@arrdem I guess we should simply patch doc to handle both styles. The existing default seems pretty odd to me.

[arrdem]

@bbatsov is your issue with the implicit two spaces, or that it should be an implicit three spaces or what? The standing behavior seems pretty reasonable to me. I certainly don't want Clojure to try and package a "real" typesetting engine for docstrings 👅

[genovese]

I am having a persistent but somewhat different problem with docstring indentation.
Depending on the content of the docstring (in particular, punctuation), the second line
of a single-paragraph docstring is not indented. The code example below illustrates
the problem and show that it only arises in particular circumstances. One of the
four functions below fills (its docstring) incorrectly, the other three -- with only
slightly varied docstrings -- fill correctly. The results of the filling are given at
the end for each case. I haven't had a chance to trace this problem through the code
yet, but I thought I'd submit the examples. The comment in the code gives details
of the context.

;;; Using M-q inside each of the following three docstrings gives different
;;; behavior, with the second incorrect. Note the period at the end of the 
;;; first line makes the difference, though it depends on the line length as 
;;; the third example shows. But if the period is there and it starts filled 
;;; properly, it stays filled properly, as the fourth example shows.
;;;
;;; platform:             Mac OS X 10.7.5
;;; emacs-version:        Gnu Emacs 24.4.1 (installed by homebrew)
;;; clojure-mode-version: 3.1.0-snapshot
;;;
;;; paragraph regexps in Emacs set at initialization time by:
;;; (setq paragraph-start paragraph-separate            ; Use blank lines to separate paragraphs by default
;;;       adaptive-fill-regexp "[ \t]*\\([>*%#]+ +\\)?" ; Fill around comment beginnings and yanked-messages
;;;       sentence-end-double-space nil                 ; Allow frenchspacing
;;;       page-delimiter "^\\(\f\\|\n\n+\\)")           ; FF or 2+ consecutive blank lines
;;;
;;; Resetting paragraph-start to its original value has no effect.
;;;
;;; fill-prefix is nil and fill-column is 72. Changing the latter has no effect.
;;; 
;;; Filling of these examples works fine in analogous defuns in elisp and CL.

(defn fills-right
  "Lorem ipsum dolor sit amet, consectetur adipiscing elit sed do ZZZ
eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim
ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut
  aliquip ex ea commodo consequat. Duis aute irure dolor in
  reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla
  pariatur. Excepteur sint occaecat cupidatat non proident, sunt in
  culpa qui officia deserunt mollit anim id est laborum."
  []
  true)

(defn fills-wrong
  "Lorem ipsum dolor sit amet, consectetur adipiscing elit sed do ZZZ.
eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim
ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut
  aliquip ex ea commodo consequat. Duis aute irure dolor in
  reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla
  pariatur. Excepteur sint occaecat cupidatat non proident, sunt in
  culpa qui officia deserunt mollit anim id est laborum."
  []
  true)

(defn fills-right-too
  "Lorem ipsum dolor sit amet, consectetur adipiscing. 
elit sed do ZZZ
eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim
ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut
  aliquip ex ea commodo consequat. Duis aute irure dolor in
  reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla
  pariatur. Excepteur sint occaecat cupidatat non proident, sunt in
  culpa qui officia deserunt mollit anim id est laborum."
  []
  true)

(defn fills-right-stays-filled
  "Lorem ipsum dolor sit amet, consectetur adipiscing elit sed do ZZZ.
  eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad
  minim veniam, quis nostrud exercitation ullamco laboris nisi ut
  aliquip ex ea commodo consequat. Duis aute irure dolor in
  reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla
  pariatur. Excepteur sint occaecat cupidatat non proident, sunt in
  culpa qui officia deserunt mollit anim id est laborum."
  []
  true)


;;; Results of filling in these three examples, M-q with point in docstring

(defn fills-right-filled
  "Lorem ipsum dolor sit amet, consectetur adipiscing elit sed do ZZZ
  eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad
  minim veniam, quis nostrud exercitation ullamco laboris nisi ut
  aliquip ex ea commodo consequat. Duis aute irure dolor in
  reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla
  pariatur. Excepteur sint occaecat cupidatat non proident, sunt in
  culpa qui officia deserunt mollit anim id est laborum."
  []
  true)

(defn fills-wrong-filled
  "Lorem ipsum dolor sit amet, consectetur adipiscing elit sed do ZZZ.
eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad
  minim veniam, quis nostrud exercitation ullamco laboris nisi ut
  aliquip ex ea commodo consequat. Duis aute irure dolor in
  reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla
  pariatur. Excepteur sint occaecat cupidatat non proident, sunt in
  culpa qui officia deserunt mollit anim id est laborum."
  []
  true)

(defn fills-right-too-filled
  "Lorem ipsum dolor sit amet, consectetur adipiscing. elit sed do ZZZ
  eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad
  minim veniam, quis nostrud exercitation ullamco laboris nisi ut
  aliquip ex ea commodo consequat. Duis aute irure dolor in
  reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla
  pariatur. Excepteur sint occaecat cupidatat non proident, sunt in
  culpa qui officia deserunt mollit anim id est laborum."
  []
  true)

[bzg]

Hi Christopher,

Christopher Genovese notifications@github.com writes:

I am having a persistent but somewhat different problem with
docstring indentation.

Sorry, I won't have time to check this particular issue before long, I hope someone else will.