FCS API internalizations and renamings

[dsyme]

The FSharp.Compiler.Service API has grown organically as various contributors (including myself) made more things more public. Many of these were never designed nor intended to be made public, and are not used in any way by any particular clients that can't be adjusted.

For example, much of FSharp.Compiler.AbstractIL.Internal.* was public - the hint is in the name, this stuff is internal to the F# compiler. Indeed all of FSharp.Compiler.AbstractIL should be regarded as internal apart from the ILModuleReader hook used to allow alternative backing stores.

This trims off about 23,000 lines out of the baseline for the FCS API (that's about half of the API surface area).

* Renamings
*     FSharp.Compiler.AbstractIL.Internal.Library.IFileSystem -> FSharp.Compiler.SourceCodeServices.IFileSystem
*     FSharp.Compiler.AbstractIL.Internal.Library.Shim -> FSharp.Compiler.SourceCodeServices.FileSystemAutoOpens
*     FSharp.Compiler.AbstractIL.Internal.Layout  -> FSharp.Compiler.TextLayout.Layout
*     FSharp.Compiler.AbstractIL.Internal.TaggedText  -> FSharp.Compiler.TextLayout.TaggedText
*     type FSharp.Compiler.Layout.layout -> FSharp.Compiler.TextLayout.Layout
*     type FSharp.Compiler.Layout.Layout -> FSharp.Compiler.TextLayout.Layout
*     module FSharp.Compiler.Layout -> FSharp.Compiler.TextLayout.LayoutRender
*     FSharp.Compiler.LayoutOps -> FSharp.Compiler.TextLayout.Layout
*     FSharp.Compiler.Layout.TaggedText  -> FSharp.Compiler.TextLayout.TaggedText
*     FSharp.Compiler.Layout.TaggedTextOps  -> FSharp.Compiler.TextLayout.TaggedText
*     FSharp.Compiler.Layout.TaggedTextOps.Literals  -> FSharp.Compiler.TextLayout.TaggedText
*
* Renamings in FSharp.Compiler.SourceCodeServices
*   Lexer.* --> FSharp.Compiler.SourceCodeServices.*
*   FSharpSyntaxToken*  --> FSharpToken*
*   FSharpErrorInfo     --> FSharpDiagnostic
*   FSharpErrorSeverity --> FSharpDiagnosticSeverity
*   ExternalSymbol      --> FSharpExternalSymbol
*   UnresolvedSymbol    --> FSharpUnresolvedSymbol
*   CompletionKind      --> FSharpCompletionKind
*   module Keywords     --> FSharpKeywords
*   module Tooltips     --> FSharpTooltip
*
* Extension methods in `ServiceAssemblyContent.fsi` now intrinsic methods on symbol types
* 
* Internalizations:
*   FSharp.Compiler.AbstractIL.* now internal
*   FSharp.Compiler.ErrorLogger.* now internal
*
* New functions in the SourceCodeServices API:
*
*    `FSharpDiagnostic.NewlineifyErrorString`
*    `FSharpDiagnostic.NormalizeErrorString`

The vast majority of the rest of the "inadvertent" part of the API is FSharp.Compiler.SyntaxTree, we can discuss that later.

[cartermp]

Yes, none of these should have ever been accessible IMO.

[baronfel]

Of these, the only ones FSAC cares about is the File System and Shims, and since those are explicitly called out I think we'll be fine.

[dsyme]

Of these, the only ones FSAC cares about is the File System and Shims, and since those are explicitly called out I think we'll be fine.

@baronfel Thanks for confirming this - I'm also working on Internal.Utilities.StructuredFormat and FSharp.Compiler.Layout. Do you use anything from there?

[baronfel]

I'm also working on Internal.Utilities.StructuredFormat and FSharp.Compiler.Layout. Do you use anything from there?

@dsyme Nope, we seem to be free of usages for those as well.

[auduchinok]

I'm also working on Internal.Utilities.StructuredFormat and FSharp.Compiler.Layout. Do you use anything from there?

We don't use them yet, but there're plans to start.

[dsyme]

@auduchinok Could you run me through what you need please?

IIRC you need

type ILModuleReader =
    abstract ILModuleDef: ILModuleDef
    abstract ILAssemblyRefs: ILAssemblyRef list
    
    /// ILModuleReader objects only need to be explicitly disposed if memory mapping is used, i.e. reduceMemoryUsage = false
    inherit System.IDisposable

[<AutoOpen>]
module Shim =

    type IAssemblyReader =
        abstract GetILModuleReader: filename: string * readerOptions: ILReaderOptions -> ILModuleReader

    [<Sealed>]
    type DefaultAssemblyReader =
        interface IAssemblyReader

    val mutable AssemblyReader: IAssemblyReader

plus the ability to create ILModuleDef and its consitutent parts? But you don't need to create IL code.

Yes I think we can make enough public for you

[dsyme]

@auduchinok Could you run me through what you need please?

I've made the things necessary for the ILModuleReader API hook public, I think it's enough for your needs but you will need to verify

[dsyme]

@cartermp @KevinRansom @TIHan This work was motivated by discussion today with @migueldeicaza concerning getting sufficient analyzer support into VS Code. It doesn't get us anywhere near that point but is at least a step towards a binary compatible FCS

[dsyme]

As part of this I noticed a subtle difference int the layout code where we have two different implementations of mkNode

    let mkNode l r joint =
        if isEmptyL l then r else
        if isEmptyL r then l else
        Node(l, r, joint)

and

    let mkNode l r joint =
        Node(l, r, joint)

I'm switching to use the first (the second gives incorrect results). But the second is the one that's currenly in use in FSharp.Core. Tests should reveal if there are any small discrepencies between these (I'd only expect improvements TBH)

[dsyme]

This is ready

@KevinRansom @cartermp @TIHan @vzarytovskii Please approve when you can

@auduchinok I believe what's public should be sufficient for the assembly reader hook, if not we can make exactly the right things public in a separate PR. I might also move the two hooks into a separate explicit API in SourceCodeServices.

[goswinr]

In the past I used FSharp.Compiler.Layout.showL to get the signature as string of a Layout .
in FCS 39.0.0 the new FSharp.Compiler.TextLayout.Layout doesn't have a showL function anymore.
Is this internal now? What is the best way to get a signature string from an FSharpToolTipElement ?
(I also asked here: #10467 (comment))

[nightroman]

@goswinr This is what we use in FSharpFar, hope this helps (you need to chaise formatComment and maybe other code paths):

https://github.com/nightroman/FarNet/blob/673f89165c614c9f8811d0fc445f52d1bbf8162d/FSharpFar/src/FSharpFar/Tips.fs#L164-L177

[dsyme]

In the past I used FSharp.Compiler.Layout.showL to get the signature as string of a Layout .

You can use TextLayout.renderL and collect up the text fragments and concatenate them together

https://github.com/dotnet/fsharp/pull/10756/files#diff-9484a8ec9d431e4165fe52c0f1aaebd0082797250ea7f333f643bb9ddc9e86a4R31

In FCS 40.0 the Layout will actually disappear and just be replaced by TaggedText[]

If you are also composing layouts please let me know.