[Bug] Solutions with code generator projects fail to build

[AptiviCEO]

Describe the bug
The projects in the solution with the code generators cause DocFX to fail to generate documentation. After updating Microsoft.CodeAnalysis.CSharp from 4.8.0 to 4.9.2, DocFX began to fail to generate documentation if there were projects that referenced a namespace generated by the code generator.

To Reproduce
Steps to reproduce the behavior:

1. Clone the Terminaux project
2. Install DocFX
3. Execute the docfx command, pointing it to the DocGen/docfx.json file: docfx DocGen/docfx.json

Expected behavior
DocFX successfully recognizes all generated code and builds the documentation.

Context (please complete the following information):

• OS: Ubuntu in GitHub Actions
• Docfx version: 2.75.2

Additional context
The failed build log is here and the docfx.json is here. It worked fine before upgrading Microsoft.CodeAnalysis.CSharp from 4.8.0 to 4.9.2 with all the generated code, such as this.

It looks like that the code generators – Terminaux.ColorDataGen, Terminaux.SequenceTypesGen, and Terminaux.TermInfoGen – didn't generate anything during DocFX's build stage, hence the error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?) errors.

Previously, all code generators worked perfectly fine as in this log.

[paulushub]

I built it by adding the CompilerGeneratedFilesOutputPath property, and the instructions #9619

  <PropertyGroup>
    <AssemblyTitle>Terminaux</AssemblyTitle>
    <AssemblyDescription>Console Manipulation Library</AssemblyDescription>
    <Description>A C# library that allows you to manipulate with your console, like colors, VT sequences, and more.</Description>
    <Version>3.1.5</Version>
    <Authors>Aptivi</Authors>
    <TargetFramework>netstandard2.0</TargetFramework>
    <DebugType>portable</DebugType>
    <AssemblyName>Terminaux</AssemblyName>
    <PackageId>Terminaux</PackageId>
    <PackageTags>console;shell;readline;gnu;cui;tui;cli;manipulation;console;vt;sequences;color;vt100;truecolor;library;csharp;reader;input;writer;styles;formatter</PackageTags>
    <PackageProjectUrl>https://github.com/Aptivi/Terminaux</PackageProjectUrl>
    <RepositoryType>git</RepositoryType>
    <RepositoryUrl>https://github.com/Aptivi/Terminaux</RepositoryUrl>
    <GenerateDocumentationFile>True</GenerateDocumentationFile>
    <GeneratePackageOnBuild>True</GeneratePackageOnBuild>
    <IncludeSymbols>True</IncludeSymbols>
    <SymbolPackageFormat>snupkg</SymbolPackageFormat>
    <Company>Aptivi</Company>
    <Copyright>Copyright (c) 2022-2023 Aptivi.</Copyright>
    <PackageReadmeFile>README.md</PackageReadmeFile>
    <PackageLicenseExpression>GPL-3.0-or-later</PackageLicenseExpression>
    <PackageRequireLicenseAcceptance>True</PackageRequireLicenseAcceptance>
    <PackageIcon>OfficialAppIcon-Terminaux-512.png</PackageIcon>
    <LangVersion>latest</LangVersion>
    <Nullable>enable</Nullable>
    <EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
    <SignAssembly>True</SignAssembly>
    <AssemblyOriginatorKeyFile>..\aptivi_snk.snk</AssemblyOriginatorKeyFile>
    <CompilerGeneratedFilesOutputPath>Generated</CompilerGeneratedFilesOutputPath>
  </PropertyGroup>

NOTE!
Also, consider moving your documentation to the new format and use the current Github workflow.

[AptiviCEO]

Thanks! Your suggestions worked! I also had to start installing DocFX manually in the workflow instead of using Docker to do the same thing. Changes here.

[paulushub]

I think you will also need

    <ItemGroup>
        <Compile Remove="$(CompilerGeneratedFilesOutputPath)/**/*.cs" />
    </ItemGroup>

and add the Generated folder to gitignore if used as above to prevent generated codes being added to the repository.

Reference (work that prompted the #9619):
Project File: https://github.com/paulushub/SVG/blob/master/Source/Svg.csproj
Workflow: https://github.com/paulushub/SVG/blob/master/.github/workflows/builddocs.yml
Output: https://paulushub.github.io/SVG/

[AptiviCEO]

I've added the above ItemGroup to the project file and added the Generated folder to .gitignore. Thanks!

However, the documentation now fails to build again with this:

2024-03-09T09:43:27.2279941Z ##[group]Run docfx DocGen/docfx.json
docfx DocGen/docfx.json
shell: /usr/bin/bash -e {0}
env:
  DOTNET_ROOT: /usr/share/dotnet
Using .NET Core SDK 8.0.201
Loading project /home/runner/work/Terminaux/Terminaux/Terminaux/Terminaux.csproj
  Determining projects to restore...
  All projects are up-to-date for restore.
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Base/Buffered/ScreenPart.cs(24,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Base/Extensions/ConsoleClearing.cs(23,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Base/Extensions/ConsolePositioning.cs(32,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Interactive/InteractiveTuiTools.cs(34,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Presentation/PresentationTools.cs(28,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/ColorSelector.cs(32,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Infobox/InfoBoxButtonsColor.cs(33,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Infobox/InfoBoxColor.cs(34,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Infobox/InfoBoxInputColor.cs(33,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Infobox/InfoBoxInputPasswordColor.cs(33,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Infobox/InfoBoxProgressColor.cs(32,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Infobox/InfoBoxSelectionColor.cs(33,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Infobox/InfoBoxSelectionMultipleColor.cs(33,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Selection/SelectionMultipleStyle.cs(35,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Selection/SelectionStyle.cs(35,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Writer/ConsoleWriters/TextWriterWhereColor.cs(28,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Writer/FancyWriters/BoxColor.cs(25,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Writer/FancyWriters/BoxFrameColor.cs(28,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Writer/FancyWriters/ProgressBarColor.cs(29,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Writer/FancyWriters/ProgressBarVerticalColor.cs(29,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
Creating output...
warning: No .NET API detected for .
Searching custom plugins in directory /home/runner/.dotnet/tools/.store/docfx/2.75.2/docfx/2.75.2/tools/net8.0/any/...
Post processor ExtractSearchIndex loaded.
Post processor SitemapGenerator loaded.
No files are found with glob pattern images/**, excluding <none>, under directory "/home/runner/work/Terminaux/Terminaux/DocGen"
7 plug-in(s) loaded.
Building 1 file(s) in TocDocumentProcessor(BuildTocDocument)...
Building 1 file(s) in ConceptualDocumentProcessor(BuildConceptualDocument=>ValidateConceptualDocumentMetadata)...
/home/runner/work/Terminaux/Terminaux/DocGen/toc.yml: warning: Unable to find either toc.yml or toc.md inside api/. Make sure the file is included in config file docfx.json!
Applying templates to 2 model(s)...
XRef map exported.
Extracting index data from 1 html files
Sitemap file is successfully exported to /home/runner/work/Terminaux/Terminaux/docs/sitemap.xml


Build failed.

    2 warning(s)
    20 error(s)

Process completed with exit code 255.

[paulushub]

Sorry, please reopen the issue.
The ItemGroup removes the previous generated sources and makes Visual Studio happy, but it causes the Docfx to fail.
Still investigating the reason. It seems to work well in my case.

[filzrev]

It seems successfully generate API metadata with latest main branch code with Release build.
(When running Debug build. It failed with #9777 problem)

Because #9753 update Roslyn version to 4.9.2 and MSBuild code analysis is executed with separated process.

[AptiviCEO]

OK, I've re-opened the issue. That's most likely the Roslyn version issue because Terminaux uses 4.9.2 and the latest public release of docfx uses 4.8.0.

[filzrev]

Latest version of docfx (2.76.0) use roslyn version 4.9.2.
And code analysis is executed on separated BuildHost.

So roslyn version relating problems will not expected occurred after docfx updates.

[paulushub]

It might help to include a test sample so that future changes could be monitored.
Code generation is a widely used .NET feature.

[StefanPoelloth]

I think ive run into the same problem, i use an analyzer to generate a class with strings containing filenames. The analyzer is in the same solution.
When running docfx (2.76.0) the build fails because the class wasnt found.
Are there any workarounds for that?