Fable as a library

[nojaf]

Description

It is related to #3549.

Could we compile F# code using Fable in a library scenario?
Similar to how FCS works today. The things I would be most interested in are getting Fable AST and compiling JS programmatically.

I envision this would be a carve-out of what Fable.Cli does today.
Things like project cracking, the file watcher and writing files to disk should remain in Fable.Cli.

[MangelMaxime]

I have a difficult time seeing what would be the separations in term of features/API between the packages.

I suppose for Fable.Cli, the pipeline would be something like:

1. You provide an fsproj to it
2. It handles project cracking
3. Call Fable.Compiler.transformFile
4. Call Fable.Compiler.Transforms.transformFile
5. Call Fable.Compiler.JavaScript.transformFile
6. Write output to the disk

Still need to find where the FableCompiler instances needs to live in.

Things like project cracking, the file watcher and writing files to disk should remain in Fable.Cli.

I don't get that part, if you keep the project cracking inside of Fable.Cli and not in the Fable.Compiler library. How will you restore the dependencies inside of your Vite plugin ?

Things would be different when thinking of the MSBuild integration because I believe inside of MSBuild, we can have access to the restored dependencies etc.

[nojaf]

Project cracking would still need to be solved for the Vite plugin. However, there are situations where you want to go with a hardcoded or custom project. I'm thinking about some serverless function that would convert F# into Fable AST or JS. The compilation should not need to care where the input information came from.

There are multiple ways of solving the project cracking. The MSBuild route would probably get all the information first-hand, so you don't want to take any dependency on what Fable.Cli uses.

As for the Vite Plugin, I was actually thinking about trying the new MSBuild capabilities announced in dotnet 8 rc 2. It could be convenient to run a design-time build from the dotnet cli and capture all the arguments from there.

Similar to how FCS works, you also need to provide the FSharpProject and they also don't provide a good way to construct those.

[ncave]

@nojaf Just FYI, the JS-only fable-compiler-js uses a simple ProjectCracker using Regex instead of MSBuild, in order to be usable on systems that don't even have .NET installed.

I'm not saying it's a prime solution, since the JS-version of Fable is 3x slower than the .NET one, it's just somewhat relevant to the conversation of project cracking and using Fable as a library.

[nojaf]

Oh, that is interesting.

On the topic of project cracking, in dotnet 8 RC 2 you can do the following:

dotnet msbuild /t:ResolveAssemblyReferencesDesignTime,ResolveProjectReferencesDesignTime,ResolvePackageDependenciesDesignTime,FindReferenceAssembliesForReferences,_GenerateCompileDependencyCache,_ComputeNonExistentFileProperty,BeforeBuild,BeforeCompile,CoreCompile /p:DesignTimeBuild=True /p:SkipCompilerExecution=True /p:ProvideCommandLineArgs=True --getItem:FscCommandLineArgs

This dumps everything as JSON:

{
  "Items": {
    "FscCommandLineArgs": [
      {
        "Identity": "-o:obj\\Debug\\net7.0\\telplin.dll",
        "FullPath": "C:\\Users\\nojaf\\Projects\\telplin\\src\\Telplin\\-o:obj\\Debug\\net7.0\\telplin.dll",
        "RootDir": "C:\\",
        "Filename": "telplin",
        "Extension": ".dll",
        "RelativeDir": "-o:obj\\Debug\\net7.0\\",
        "Directory": "Users\\nojaf\\Projects\\telplin\\src\\Telplin\\-o:obj\\Debug\\net7.0\\",
        "RecursiveDir": "",
        "ModifiedTime": "",
        "CreatedTime": "",
        "AccessedTime": "",
        "DefiningProjectFullPath": "C:\\Program Files\\dotnet\\sdk\\8.0.100-rc.2.23502.2\\FSharp\\Microsoft.FSharp.Targets",
        "DefiningProjectDirectory": "C:\\Program Files\\dotnet\\sdk\\8.0.100-rc.2.23502.2\\FSharp\\",
        "DefiningProjectName": "Microsoft.FSharp",
        "DefiningProjectExtension": ".Targets"
      },
      {
        "Identity": "-g",
        "FullPath": "C:\\Users\\nojaf\\Projects\\telplin\\src\\Telplin\\-g",
        "RootDir": "C:\\",
        "Filename": "-g",
        "Extension": "",
        "RelativeDir": "",
        "Directory": "Users\\nojaf\\Projects\\telplin\\src\\Telplin\\",
        "RecursiveDir": "",
        "ModifiedTime": "",
        "CreatedTime": "",
        "AccessedTime": "",
        "DefiningProjectFullPath": "C:\\Program Files\\dotnet\\sdk\\8.0.100-rc.2.23502.2\\FSharp\\Microsoft.FSharp.Targets",
        "DefiningProjectDirectory": "C:\\Program Files\\dotnet\\sdk\\8.0.100-rc.2.23502.2\\FSharp\\",
        "DefiningProjectName": "Microsoft.FSharp",
        "DefiningProjectExtension": ".Targets"
      },
      {
        "Identity": "--debug:embedded",
        "FullPath": "C:\\Users\\nojaf\\Projects\\telplin\\src\\Telplin\\--debug:embedded",
        "RootDir": "C:\\",
        "Filename": "--debug:embedded",
        "Extension": "",
        "RelativeDir": "",
        "Directory": "Users\\nojaf\\Projects\\telplin\\src\\Telplin\\",
        "RecursiveDir": "",
        "ModifiedTime": "",
        "CreatedTime": "",
        "AccessedTime": "",
        "DefiningProjectFullPath": "C:\\Program Files\\dotnet\\sdk\\8.0.100-rc.2.23502.2\\FSharp\\Microsoft.FSharp.Targets",
        "DefiningProjectDirectory": "C:\\Program Files\\dotnet\\sdk\\8.0.100-rc.2.23502.2\\FSharp\\",
        "DefiningProjectName": "Microsoft.FSharp",
        "DefiningProjectExtension": ".Targets"
      },
...
]

That is another reason I want to de-couple project cracking from Fable compilation.

[ncave]

@nojaf Yes, parsing projects with Regex works, but obviously only for simple F# projects (non-conditional, no multi-target, no pre/post build steps, etc.), unless you want to reimplement most of MSBuild machinery.

The Fable CLI already has a robust project cracking using Buildalyzer, so why not use the Fable CLI as a library, just expose what additional API you need.

Although if now MSBuild can export all we need as JSON, perhaps that's a good option too.

[MangelMaxime]

Although if now MSBuild can export all we need as JSON, perhaps that's a good option too.

This could remove the trick we do with creating a .csproj for having BuildAnalyzer to work correctly.

Does this means that the user needs to use .NET 8?

What if the user has a global.json which set .NET version to 6 for the repository. Is it possible to override this rule for invoking dotnet msbuild with the feature we want or am I imagining a problem?

[nojaf]

This could remove the trick we do with creating a .csproj for having BuildAnalyzer to work correctly.

I agree this was probably more robust than proj-info, but in my opinion, it is a hack.
Temporarily seeing that *.csproj file in the file explorer was so confusing the first time I saw it.

Anyway, my main ask is to extract core bits of Fable.Cli. I would like this to be as dependency-free as possible. (In a perfect world only having the FCS fork dep). This would open the door to experimenting more outside of the Fable repository.Short term, I would not change how Fable.Cli works or revisit project cracking.

[MangelMaxime]

Here is a summary of our discussion

@nojaf @jwosty Please feel free to complete or correct me

We want to continue the exploration of introducing "Fable as a library".

Goals:

• Expose an API which allows to get the Project Cracked information.

  This is to offer an easy solution for people who don't want to manually crack a project.

• Expose an API which allows you to pass the Project Cracker information to create a Fable instance.

  Useful if you want to experiment with others way to crack a project. Example by using msbuild 8 or when invoking Fable from inside MSBuild.

• Figure out the minimal API to expose to create a Fable compiler instance.

  Minimal API is probably not CLIArgs records. CLIArgs record is just way for Fable CLI to interact with the user input. For example, in the case of vite-fable the language options doesn't make sense to expose. It will probably always use JavaScript.

• When working on vite-fable or MSBuild Fable, one of the edge cases to check is that dependant file are handle correctly. And also, that inline code works too across files.

  Fo example, if compiling File1.fs also re-compiles File2.fs.

[nojaf]

@jwosty what are the restrictions in terms of target framework to use this in MSBuild?
Would net6.0 work out?

@ALL what would be a good name for this library? Fable.Compiler.Service might be too similar to FCS.

[nojaf]

To also elaborate a bit more on my end-game for this:

I basically see Fable as an alternative to TypeScript or Elm. When you create a front-end application these days, you would strongly consider Vite given its popularity. When using Vite TS works out of the box and Elm uses a plugin. In both cases, you start Vite.

Currently, with Fable, you are forced to start dotnet fable and take it from there. I would like to have a smoother integration with Vite. For example, Vite would watch the files for changes and each individual transpiled file doesn't need to exist on disk while Vite is running. I believe a plugin could really streamline the whole experience.

To make this Vite plugin, I would like to spawn some dotnet process and talk to it from the JavaScript side. To pull this off, I want to carve out the basics that I'm after. I need a way to tell Fable to compile a single file and give the result in memory. And of course, I need to be able to do this over and over again as that dotnet process will live as long as Vite is running.

Disclaimer: This sounds a bit ambitious and so far I have no evidence this will work out or even be efficient. All I have is a hunch right now and I'm scratching my own itch. You should really see this as an experiment and nothing more. On the flip side, if you are interested in this, do please reach out, I would love to collaborate with others on this!

[nojaf]

When working on vite-fable or MSBuild Fable, one of the edge cases to check is that dependant file are handle correctly.

I believe ncave/fsharp#14 could be beneficial to tackle this case.

[jwosty]

@nojaf net6.0 would be fine for the SDK.

In terms of naming, may I suggest simply Fable.Compiler or Fable.API