Make GeneratorDriverOptions.BaseDirectory public

[tmat]

Background and Motivation

The compiler uses this option to construct the path of a source generated file. This path is then used to generate metadata name of file-scoped types.

This path is currently only set by the compiler task to the value of CompilerGeneratedFilesOutputPath msbuild property. It does not flow from the project system in the IDE. This causes compiler crashes during EnC since the metadata names generated by the detal emit differ from those generated by the compiler during build.

Proposed API

namespace Microsoft.CodeAnalysis;

public readonly struct GeneratorDriverOptions
{
     // existing public API:
     public readonly IncrementalGeneratorOutputKind DisabledOutputs;
     public readonly bool TrackIncrementalGeneratorSteps;

     public GeneratorDriverOptions(IncrementalGeneratorOutputKind disabledOutputs);
     public GeneratorDriverOptions(IncrementalGeneratorOutputKind disabledOutputs, bool trackIncrementalGeneratorSteps);

-    internal string? BaseDirectory { get; init; }

+    /// <summary>
+    /// Absolute path to the base directory used for file paths of generated files. Usually the project's output directory unless <see cref="CommandLineArguments.GeneratedFilesOutputDirectory"/> is specified.
+    /// </summary>
+    public string? BaseDirectory { get; }

+    /// <summary>
+    /// Creates <see cref="GeneratorDriverOptions"/>.
+    /// </summary>
+    /// <param name="baseDirectory">Absolute path to the base directory used for file paths of generated files, or null.</param>
+    /// <param name="disabledOutputs"></param>
+    /// <param name="trackIncrementalGeneratorSteps"></param>
+    /// <exception cref="ArgumentException"><paramref name="baseDirectory"/> is specified but not an absolute path.</exception>
+    public GeneratorDriverOptions(IncrementalGeneratorOutputKind disabledOutputs = IncrementalGeneratorOutputKind.None, bool trackIncrementalGeneratorSteps = false, string? baseDirectory = null);
}

Update
Make baseDirectory optional.

Usage Examples

https://github.com/dotnet/roslyn/pull/75311/files#diff-e5be48a46768235a257d73711ef134c71a03f8f896de4a977012e4b85e7bc599R57

Alternative Designs

Should we make BaseDirectory non-nullable? It would only be null if the obsoleted constructors were used.

Alternative:

public readonly struct GeneratorDriverOptions
{
     // existing public API:
     public readonly IncrementalGeneratorOutputKind DisabledOutputs;
     public readonly bool TrackIncrementalGeneratorSteps;

+    [Obsolete]
     public GeneratorDriverOptions(IncrementalGeneratorOutputKind disabledOutputs);

+    [Obsolete]
     public GeneratorDriverOptions(IncrementalGeneratorOutputKind disabledOutputs, bool trackIncrementalGeneratorSteps);

+    /// <summary>
+    /// Builder that takes a Type of a source generator and a relative file path of a generated file and returns an absolute path of the file.  
+    /// The generated absolute path is used for SyntaxTree.FilePath of syntax trees parsed from the generated source file.
+    /// </summary>
+    public Func<Type, string, string> FilePathBuilder { get; }

+    /// <summary>
+    /// Creates <see cref="GeneratorDriverOptions"/>.
+    /// </summary>
+    /// <param name="filePathBuilder">Builds absolute file paths for generated files.</param>
+    /// <param name="disabledOutputs"></param>
+    /// <param name="trackIncrementalGeneratorSteps"></param>
+    /// <exception cref="ArgumentNullException"><paramref name="filePathBuilder"/> is null.</exception>
+    public GeneratorDriverOptions(Func<Type, string, string> filePathBuilder, IncrementalGeneratorOutputKind disabledOutputs = IncrementalGeneratorOutputKind.None, bool trackIncrementalGeneratorSteps = false);
}

Risks

[jaredpar]

Where else do we expose output paths in the compiler APIs? The case I think of is EmitOptions but I'm not sure how that is consumed at the ENC / project system level?

[tmat]

The project system uses intermediate output path, output path and output ref path as well.

IDE/EnC does not use paths for EmitOptions - EnC only emits in-memory deltas. The intermediate output path is used to read metadata and PDB of the EnC baseline. The output (ref) path is used to convert between project references and metadata references in general (not EnC specific).

[jaredpar]

But where do those come from in the compiler API? I'm not sure that we actually expose paths, pretty much anywhere, in the current set of options. Only EmitOptions does this to my knowledge.

I'm interested in this cause I'm trying to frame whether or not this is the right place for the property. I'm concerned with adding paths about output in layers we haven't done previously.

[tmat]

My understanding is that the generator driver is in a layer above compilation. It is created in the command line compiler and in IDE layers, where we already deal with paths.

SyntaxTree also has a path. Passing a base directory into generator driver for creating syntax tree paths seems reasonable.

[tmat]

Perhaps we can abstract the actual path format a bit and pass in a "source-generated file path builder" instead of BaseDirectory. The path builder would accept the type of the source generator and return a full path. Essentially what GetFilePathPrefixForGenerator is doing. I'll add it to alternatives in the API proposal.

[jaredpar]

SyntaxTree also has a path.

There are definitely a few places where we have input paths in the API like this. The output paths though are conspicuously missing. I've hit it several times trying to write tooling around the compiler. It's so conspicuous that it feels deliberate which is why I hesitate a bit about adding it without thinking through it a bit.

Example: from a CSharpCompilation it's basically impossible to determine an assembly name. You have to look at CommandLineArguments to infer what it should be.

Perhaps we can abstract the actual path format a bit and pass in a "source-generated file path builder" instead of BaseDirectory.

That could work.

To be clear: I'm not 100% against putting better support for output paths in our options. But it is conspicuously missing today to the point I'd want to be thoughtful, and maybe do some archeology, about how it gets into the APIs.

[jaredpar]

Dug into the idea of GeneratorDriverOptions existing on the same layer as EmitOptions hence it's okay for it to have paths. After looking at the code I agree with this assertion. Both types are created during CommonCompiler.CompileAndEmit, they derive their paths from CommandLineArguments directly and it's purely for the emit phase of compilation.