Configuration Schema generator #1383

eerhardt · 2023-12-13T20:20:17Z

Adds a tool that automatically generates ConfigurationSchema.json files for components.

To use the tool, a component adds an assembly attribute. For example:

[assembly: ConfigurationSchema("Aspire:Azure:Storage:Blobs", typeof(AzureStorageBlobsSettings))]
[assembly: ConfigurationSchema("Aspire:Azure:Storage:Blobs:ClientOptions", typeof(BlobClientOptions), exclusionPaths: ["Default"])]

[assembly: LoggingCategories("Azure", "Azure.Core", "Azure.Identity")]

ConfigurationSchemaAttribute

The first parameter is the config section path where the type is loaded from.
The second parameter is the Type that is bound at that path.
ExclusionPaths - (optional) The config sections to exclude from the ConfigurationSchema. This is useful if there are properties you don't want to publicize in the config schema.

LoggingCategoriesAttribute

The list of log categories produced by the component. These categories will show up under the Logging section in appsettings.json

This tool works after compilation by scanning the assembly for ConfigurationSchema and LoggingCategories attributes, and uses Roslyn APIs to inspect all properties of the types. It reuses all the parsing logic from https://github.com/dotnet/runtime/tree/main/src/libraries/Microsoft.Extensions.Configuration.Binder/gen, and using the underlying model objects, generates a JSON schema using System.Text.Json.Nodes APIs.

I've copied the necessary code from dotnet/runtime into the RuntimeSource folder for now. This code doesn't need to be reviewed since it is a straight copy from dotnet/runtime. After this PR goes through, we will set up an automatic "sync" action that will update this code anytime changes in the dotnet/runtime code is updated.

The remaining work to address after the initial PR:

Supporting collection/enumerable properties (skipped for now)
Perf: the tool takes ~400-500 ms to run. An option we should try is to load the Roslyn assemblies from the SDK, which are already R2R'd. This may make the tool faster because it won't need to JIT Roslyn every time it is run. This would be the same logic used in Improve loading of Roslyn assemblies on validate package task sdk#22277.

cc @eiriktsarpalis @tarekgh

Fix #1146

Microsoft Reviewers: Open in CodeFlow

Use XDcoument to parse the doc comments. Strip xml elements from summary, strip new lines, and escape invalid chars.

Skip properties that are obsolete, editorbrowsable(never), or not settable. Support Uris.

Fix doc comments on the root objects - use the Type's summary.

…hey don't match. Add an option to update the checked in file.

Remove unnecessary attributes. Ensure Specs folder matches runtime code. Ensure Roslyn and SourceGenerators common folders match runtime code.

Use the Parser from dotnet/runtime and inject more methods into it using the partial class. Mock out the DiagnosticDescriptors because we don't need localization.

Directory.Packages.props

src/Components/Directory.Build.targets

joperezr · 2023-12-13T22:48:21Z

src/Components/Directory.Build.targets

+    <PropertyGroup>
+
+      <GeneratorCommandLine>"$(DotNetTool)" exec $(ConfigurationSchemaGeneratorPath)</GeneratorCommandLine>
+      <GeneratorCommandLine>$(GeneratorCommandLine) --input "@(IntermediateAssembly)"</GeneratorCommandLine>


Haven't looked at it in a while, but TargetsTriggeredByCompilation is guaranteed to be called after CoreCompile, right?

Correct. See https://github.com/dotnet/roslyn/blob/5249865bc562b64333ae80de6ccb47e7881b7840/src/Compilers/Core/MSBuildTask/Microsoft.CSharp.Core.targets#L84-L173

Whenever CoreCompile runs Csc, it will <CallTarget Targets="$(TargetsTriggeredByCompilation)"

src/Components/Directory.Build.targets

src/Components/Aspire.Azure.Storage.Blobs/AspireBlobStorageExtensions.cs

joperezr · 2023-12-13T22:53:35Z

src/Components/Aspire.Azure.Storage.Blobs/ConfigurationSchema.json

                        },
                        "GeoRedundantSecondaryUri": {
                          "type": "string",
                          "format": "uri",
-                          "description": "Gets or sets the secondary storage Uri that can be read from for the storage account if the account is enabled for RA-GRS."
+                          "description": "Gets or sets the secondary storage T:System.Uri that can be read from for the storage account if the\r\n             account is enabled for RA-GRS.\r\n            \r\n             If this property is set, the secondary Uri will be used for GET or HEAD requests during retries.\r\n             If the status of the response from the secondary Uri is a 404, then subsequent retries for\r\n             the request will not use the secondary Uri again, as this indicates that the resource\r\n             may not have propagated there yet. Otherwise, subsequent retries will alternate back and forth\r\n             between primary and secondary Uri."


Interesting, I'm curious, how does this render in the IDE, I suppose it actually shows the new lines as opposed to having the new line characters printed out?

It renders the newlines correctly. This is an area that will probably need to be tweaked over time. I don't know why the newlines and extra spaces are in there. I need to look at some more Roslyn code to figure out how they strip out all the whitespace in these XML docs.

src/Tools/ConfigurationSchemaGenerator/DiagnosticDescriptors.cs

sebastienros · 2023-12-15T01:13:12Z

src/Components/Directory.Build.targets

+
+  </Target>
+
+  <Target Name="CompareConfigurationSchema"


Is it supposed to be where the logic to verify the json file that is committed in the repos is up to date with the curent code?

Refactor attributes to be more understandable and easier to use.

The TypeParseInfo.BinderInvocation is a nullable property. If it happens to be null, these two places will throw a null ref. .NET Aspire is reusing this code to generate JSON schemas for appsettings.json files. See dotnet/aspire#1383. The plan is to keep this in sync using automated PRs to dotnet/aspire when the dotnet/runtime code is updated. (dotnet/aspire#1424). Contributing this fix back so these two code bases can stay in sync.

eerhardt added 15 commits December 11, 2023 18:14

WIP - initial implementation

f6dc518

Finished initial Source Generator version.

9f93f52

Convert from a source generator to a post-compilation tool.

a9b4752

Fix descriptions.

fc89713

Use XDcoument to parse the doc comments. Strip xml elements from summary, strip new lines, and escape invalid chars.

Support TimeSpan and enums correctly.

b05c864

If there are no properties on an object, skip it.

e6d61c1

Convert to using System.Text.Json.Nodes instead of writing strings.

cc5f8ec

Recurse on complex properties.

807180e

Skip properties that are obsolete, editorbrowsable(never), or not settable. Support Uris.

Support default boolean values.

65c185d

Fix doc comments on properties from base types.

5844f0b

Fix doc comments on the root objects - use the Type's summary.

Allow for excluded paths.

89412fb

Compare the output of the generator to the checked in file, fail if t…

642f9d6

…hey don't match. Add an option to update the checked in file.

Clean up round 1

f736d12

Remove unnecessary attributes. Ensure Specs folder matches runtime code. Ensure Roslyn and SourceGenerators common folders match runtime code.

Final round of clean up

ee374a6

Use the Parser from dotnet/runtime and inject more methods into it using the partial class. Mock out the DiagnosticDescriptors because we don't need localization.

Initial PR feedback

cb63174

eerhardt requested review from sebastienros and joperezr December 13, 2023 20:20

dotnet-issue-labeler bot added the area-integrations Issues pertaining to Aspire Integrations packages label Dec 13, 2023

eerhardt added 2 commits December 13, 2023 15:58

Fix build by using DotNetTool exec

3ed02d4

Quote the DotNetTool path.

4f27f89