Make `--format` preserve comments

[byorgey]

I noticed it drops comments, but we can fix that later.

Originally posted by @xsebek in #1459 (comment)

[xsebek]

I wonder how we would store those.

For formatting, it would be nice to have them in the AST, like the SrcLoc.

For parsing though, it seems easiest to put comments in ParserT State monad and pair them afterwards.

@byorgey do you have any experience with round-tripping comments? 😃

[byorgey]

@byorgey do you have any experience with round-tripping comments? 😃

I do not. It will be a fun learning experience. 😄

For formating, it would be nice to have them in the AST, like the SrcLoc.

For parsing though, it seems easiest to put comments in ParserT State monad and pair them afterwards.

Agreed... I wonder about a scheme where we save comments out of band via ParserT State, and then do a pass at the end where we go through the whole AST and stuff comments into appropriate Syntax nodes?

[xsebek]

@byorgey that sounds doable, comments should apply to the next Syntax node (comparing SrcLoc, usually it will be a bind) so one traversal should do it. I assume you mean adding comments to the Syntax' data? That is:

data Syntax' ty = Syntax'
  { _sLoc :: SrcLoc
  , _sComment :: Maybe Text -- NEW!
  , _sTerm :: Term' ty
  , _sType :: ty
  }

The one parser I know of that tries hard to preserve comments is Spoon for Java, which has quite nice docs:
https://spoon.gforge.inria.fr/comments.html
Java is is a bit different from Swarm, but there is one case (orphan comments) noted in Spoon docs, that we should handle:

if (1 + 1 == 2) {
  log "Everything OK in Swarm"
} {
  // will not happen
}

In that scenario, we could attach the comment to the implicit noop. 🤔

[byorgey]

I assume you mean adding comments to the Syntax' data?

Yes, that's exactly what I mean. It may not even be necessary to do the State + second pass thing, maybe we can capture comments directly into Syntax' nodes similarly to how we currently capture the SrcLoc? I'm not sure, it would require some playing around.

Yes, we'll probably want a special case for the orphan comment thing.