[meta issue] preserving syntax trivia and comments

[smoothdeveloper]

I noticed that the pretty printer basically drops all comments, and that upstream, comments are parsed but basically dropped.

In F# compiler, there has been some ground work done to better preserve syntax trivia, such as precise placement of keywords, and also preserving the comments.

One thing which is done there, is that all the comments are tucked in a single list for the implementation (or signature) file:

https://github.com/dotnet/fsharp/blob/70171b7ebd461c10345861558c8afddc84e8ef29/src/Compiler/SyntaxTree/SyntaxTrivia.fs#L31-L35

This allows pretty printer tool (for F#, the main one is called Fantomas) to reintegrate the comments while also not making the AST more complex due to handling of comments.

I think it also keeps the parsing simple, where we could start building this comment list and attach it to a top level element in the AST.

If I can get a bit of guidance about which part of the AST we'd like to aggregate the comments, and how to do that in the parser implementation, I may give it a shot.

related:

• Comment between continuation lines in F77 breaks lexer #45
• What to do with inline comments? #91
• Construct SrcSpans in Blocks #102
• Reprinter deletes commentaries in multiline declaration #115
• Full line comments before the first CASE in a CASE block are dropped #160
• F90 parsing failure for multiple comments in case constructs #267

[dorchard]

The pretty printer doesn't preserve secondary notation (white space + comments), but if this is desired, e.g., to build a refactoring tool, then this can be done in two ways:

1. Using the "rewriter" https://github.com/camfort/fortran-src/blob/main/src/Language/Fortran/Rewriter.hs
2. Using the "reprinter" technique, which is part of CamFort or separately here https://github.com/camfort/reprinter.

Does it seem like these might do what you need them to do? If you could say a bit more about your use case I might be able to direct you in using either of the above approaches.