Incremental SG - Unit-testing the pipeline / caching-behaviour using GeneratorDriver

[ErikWe]

🤞 Hope this is a suitable place for general guidance to Roslyn, if not feel free to close :)

Hi all! I've been wrestling a lot with incremental SGs lately. The main thing I'm uncertain about at the moment is how caching behaves in my pipeline. I've been trying to unit-test this using the GeneratorDriver class, similar to what is described in the SG cookbook.

However, I have two main issues:

1. Is there a simple way of accessing data about what stages ran etc? There is some data hidden deep within the GeneratorDriver, but it's not very accessible.

2. I've been simulating changes to source-code using Compilation.ReplaceSyntaxTree. However, when running the GeneratorDriver after such a replace, it seems that all pipeline nodes 'derived' from the original SyntaxTree is re-evaluted (breakpoint is hit), even if the input to the node has been cached. Simply adding a new SyntaxTree using Compilation.AddSyntaxTree does not cause a re-evaluation of the original SyntaxTree. Is there a better way of approaching this, or is this simply the expected behaviour?

Thanks!

[ErikWe]

In case anyone is interested in point 2, here is my eventual solution (using AdhocWorkspace):

A trivial generator:

[Generator]
public class TrivialGenerator : IIncrementalGenerator
{
    public void Initialize(IncrementalGeneratorInitializationContext context)
    {
        var provider = context.SyntaxProvider.CreateSyntaxProvider(static (node, _) => node is TypeDeclarationSyntax, static (context, _) => ((TypeDeclarationSyntax)context.Node).Identifier.Text);

        context.RegisterSourceOutput(provider, Execute);
    }

    private static void Execute(SourceProductionContext context, string identifier)
    {
        context.AddSource($"{identifier}.g.cs", SourceText.From($$"""public class {{identifier}}2 { }""", Encoding.UTF8));
    }
}

And programatically testing the incremental-ness of the generator:

public async void Test(string initialText, string updatedText, IEnumerable<MetadataReference> metadataReferences, CSharpParseOptions parseOptions, CSharpCompilationOptions compilationOptions)
{
    using AdhocWorkspace workspace = new();

    var driver = CSharpGeneratorDriver.Create(new TrivialGenerator()).WithUpdatedParseOptions(parseOptions);

    var solutionInfo = SolutionInfo.Create(SolutionId.CreateNewId(), VersionStamp.Default);
    var projectInfo = ProjectInfo.Create(ProjectId.CreateNewId(), VersionStamp.Default, "Test", "testassembly", "C#",
        parseOptions: parseOptions,
        compilationOptions: compilationOptions,
        metadataReferences: metadataReferences
    );
    var documentID = DocumentId.CreateNewId(projectInfo.Id);

    var solution = workspace.AddSolution(solutionInfo);
    solution = solution.AddProject(projectInfo);
    solution = solution.AddDocument(documentID, "File.cs", SourceText.From(initialText));

    var compilation = await solution.Projects.First().GetCompilationAsync().ConfigureAwait(false);

    driver = driver.RunGeneratorsAndUpdateCompilation(compilation!, out compilation, out var _);

    solution = solution.WithDocumentText(documentID, SourceText.From(updatedText));

    workspace.TryApplyChanges(solution);

    compilation = await solution.Projects.First().GetCompilationAsync().ConfigureAwait(false);

    driver = driver.RunGeneratorsAndUpdateCompilation(compilation!, out compilation, out var _);
}

And some example inputs:

string initialText = """
    public class Foo { }
    """;

string updatedText = """
    public class Foo { }
    public class Bar { }
    """;

Running Test will cause TrivialGenerator.Execute to execute twice, once for "Foo" in the first pass, and once for "Bar" in the second pass - i.e, the cached "Foo" will be used during the second pass.

[tonyhallett]

This may be of help. https://www.meziantou.net/testing-roslyn-incremental-source-generators.htm

// Update the compilation and rerun the generator
// Assert the driver doesn't recompute the output

[ErikWe]

In case anyone is interested in point 2, here is my eventual solution (using AdhocWorkspace):

A trivial generator:

[Generator]
public class TrivialGenerator : IIncrementalGenerator
{
    public void Initialize(IncrementalGeneratorInitializationContext context)
    {
        var provider = context.SyntaxProvider.CreateSyntaxProvider(static (node, _) => node is TypeDeclarationSyntax, static (context, _) => ((TypeDeclarationSyntax)context.Node).Identifier.Text);

        context.RegisterSourceOutput(provider, Execute);
    }

    private static void Execute(SourceProductionContext context, string identifier)
    {
        context.AddSource($"{identifier}.g.cs", SourceText.From($$"""public class {{identifier}}2 { }""", Encoding.UTF8));
    }
}

And programatically testing the incremental-ness of the generator:

public async void Test(string initialText, string updatedText, IEnumerable<MetadataReference> metadataReferences, CSharpParseOptions parseOptions, CSharpCompilationOptions compilationOptions)
{
    using AdhocWorkspace workspace = new();

    var driver = CSharpGeneratorDriver.Create(new TrivialGenerator()).WithUpdatedParseOptions(parseOptions);

    var solutionInfo = SolutionInfo.Create(SolutionId.CreateNewId(), VersionStamp.Default);
    var projectInfo = ProjectInfo.Create(ProjectId.CreateNewId(), VersionStamp.Default, "Test", "testassembly", "C#",
        parseOptions: parseOptions,
        compilationOptions: compilationOptions,
        metadataReferences: metadataReferences
    );
    var documentID = DocumentId.CreateNewId(projectInfo.Id);

    var solution = workspace.AddSolution(solutionInfo);
    solution = solution.AddProject(projectInfo);
    solution = solution.AddDocument(documentID, "File.cs", SourceText.From(initialText));

    var compilation = await solution.Projects.First().GetCompilationAsync().ConfigureAwait(false);

    driver = driver.RunGeneratorsAndUpdateCompilation(compilation!, out compilation, out var _);

    solution = solution.WithDocumentText(documentID, SourceText.From(updatedText));

    workspace.TryApplyChanges(solution);

    compilation = await solution.Projects.First().GetCompilationAsync().ConfigureAwait(false);

    driver = driver.RunGeneratorsAndUpdateCompilation(compilation!, out compilation, out var _);
}

And some example inputs:

string initialText = """
    public class Foo { }
    """;

string updatedText = """
    public class Foo { }
    public class Bar { }
    """;

Running Test will cause TrivialGenerator.Execute to execute twice, once for "Foo" in the first pass, and once for "Bar" in the second pass - i.e, the cached "Foo" will be used during the second pass.

[tonyhallett]

This may be of help. https://www.meziantou.net/testing-roslyn-incremental-source-generators.htm

// Update the compilation and rerun the generator
// Assert the driver doesn't recompute the output