diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ed09a7e..34ff194 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -16,14 +16,14 @@ jobs: steps: - name: Checkout - uses: actions/checkout@v3 + uses: actions/checkout@v4 with: fetch-depth: 0 - - name: Install .NET 7.0 - uses: actions/setup-dotnet@v3 + - name: Install .NET 9.0 + uses: actions/setup-dotnet@v4 with: - dotnet-version: '7.0.x' + dotnet-version: '9.0.x' - name: Build and Test (Debug) run: dotnet test src -c Debug @@ -32,4 +32,4 @@ jobs: if: github.event_name == 'push' shell: bash run: | - ./src/dotnet-releaser/bin/Debug/net7.0/dotnet-releaser run --nuget-token "${{secrets.NUGET_TOKEN}}" --github-token "${{secrets.GITHUB_TOKEN}}" src/dotnet-releaser.toml + ./src/dotnet-releaser/bin/Debug/net9.0/dotnet-releaser run --nuget-token "${{secrets.NUGET_TOKEN}}" --github-token "${{secrets.GITHUB_TOKEN}}" src/dotnet-releaser.toml diff --git a/doc/GIITHUB_TOKEN_menu_Actions_General.png b/doc/GIITHUB_TOKEN_menu_Actions_General.png new file mode 100644 index 0000000..bcf8083 Binary files /dev/null and b/doc/GIITHUB_TOKEN_menu_Actions_General.png differ diff --git a/doc/GIITHUB_TOKEN_workflows_permission.png b/doc/GIITHUB_TOKEN_workflows_permission.png new file mode 100644 index 0000000..8fa80e9 Binary files /dev/null and b/doc/GIITHUB_TOKEN_workflows_permission.png differ diff --git a/doc/GITHUB_TOKEN_custom_repository_access.png b/doc/GITHUB_TOKEN_custom_repository_access.png new file mode 100644 index 0000000..fa2b4f1 Binary files /dev/null and b/doc/GITHUB_TOKEN_custom_repository_access.png differ diff --git a/doc/readme.md b/doc/readme.md index e56cfe5..b3e7584 100644 --- a/doc/readme.md +++ b/doc/readme.md @@ -18,6 +18,7 @@ - [2.2. MSBuild](#22-msbuild) - [2.3. Tests](#23-tests) - [2.4. Coverage](#24-coverage) + - [Badge coverage to Gist](#badge-coverage-to-gist) - [2.5. Coveralls](#25-coveralls) - [2.6. GitHub](#26-github) - [2.7. Packaging](#27-packaging) @@ -28,6 +29,8 @@ - [2.12. Service](#212-service) - [2.12.1. Systemd](#2121-systemd) - [2.13. Package Dependencies](#213-package-dependencies) + - [2.14. About GITHUB\_TOKEN](#214-about-github_token) + - [Custom GITHUB\_TOKEN](#custom-github_token) - [3. CLI Usage](#3-cli-usage) - [3.1. `dotnet-releaser new`](#31-dotnet-releaser-new) - [3.2. `dotnet-releaser build`](#32-dotnet-releaser-build) @@ -130,19 +133,11 @@ The `publish` command allows to build and publish all packages to GitHub and NuG dotnet-releaser publish --force --github-token "${{secrets.GITHUB_TOKEN}}" --nuget-token "${{secrets.YOUR_NUGET_SECRET_TOKEN}}" dotnet-releaser.toml ``` -> NOTE: When running from a GitHub Action, it is recommended to use the predefined `GITHUB_TOKEN` accessible from your secrets: `${{secrets.GITHUB_TOKEN}}`. +> NOTE: When running from a GitHub Action, it is recommended to use the predefined `GITHUB_TOKEN` accessible from your secrets: `${{secrets.GITHUB_TOKEN}}`, but you might want to create your own `PAT_GITHUB_TOKEN` to leverage more features of dotnet-releaser. See section later in this document [About GITHUB_TOKEN](#214-about-github_token). > > It is recommended to use `dotnet-releaser run` instead when running from a GitHub Action, as explained in the following section. -If you want to run this from a command line, you need to a [personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) - -You should tick the `public_repo` in the list: - -- [x] public_repo - -And put an appropriate expiration date. - -![GitHub Setup of Personal access token](https://raw.githubusercontent.com/xoofx/dotnet-releaser/main/img/github_new_personal_access_token.png) +If you want to run this from a command line, you need to a [personal access token](#custom-github_token) ### 1.2. Adding dotnet-releaser to your CI on GitHub @@ -172,6 +167,20 @@ Depending on the kind of GitHub event, the run command will automatically: - If it is a `push` with a tag version (e.g `v1.9.6` as configured with `[github]` section in the configuration) it will **perform a full build with publish** - For an application, it will publish multiple cross-compiled packages to your release +> **NOTE about the publish process:** +> +> The publish process relies on a tag commit being pushed, and the GitHub action being run on that tag commit. If you have set up your GitHub action to only run on specific branches and not on tag pushes, the `dotnet-releaser` publish portion of the process will never run. +To fix this and still specify specific branches to run your action on, use something like this in your GitHub action: +> +> ```yml +> on: +> push: +> branches: +> - main # Used for stable releases +> - develop # Used for preview releases +> tags: +> - '*' # run on all tags being pushed +> ``` #### 1.2.2. Example of a GitHub CI Integration An example of a setup with GitHub Actions: @@ -379,6 +388,8 @@ Coverage will produce artifacts in the output folder. | `exclude_by_file` | `string[]` | Ignore specific source files from code coverage. | `include` | `string[]` | Explicitly set what to include in code coverage analysis using filter expressions. | `include_directory` | `string[]` | Explicitly set which directories to include in code coverage analysis. +| `badge_upload_to_gist` | `bool` | Enable to publish a [badge for the coverage to a GitHub gist](#badge-coverage-to-gist). +| `badge_gist_id` | `string` | Gist identifier used to publish a [badge for the coverage to a GitHub gist](#badge-coverage-to-gist). Example to disable coverage: @@ -394,6 +405,28 @@ Example to disable the use of source link: source_link = false ``` +#### Badge coverage to Gist + +`dotnet-releaser` allow to publish a coverage badge directly to a GitHub gist that can then be linked back from your readme. + +Using this feature requires a few setup steps: + +- Create a gist with a readme.md that explain that it is a gist used by dotnet-releaser for all your repositories. For example see this gist [here](https://gist.github.com/xoofx/4b1dc8d0fa14dd6a3846e78e5f0eafae) +- Get the id of the gist: In the URL for example `https://gist.github.com/xoofx/4b1dc8d0fa14dd6a3846e78e5f0eafae` the ID of the gist is `4b1dc8d0fa14dd6a3846e78e5f0eafae` +- You can then configure the coverage by enabling pushing the badge to the specified gist: + ```toml + [coverage] + badge_upload_to_gist = true + badge_gist_id = "4b1dc8d0fa14dd6a3846e78e5f0eafae" + ``` +- Then you need to create a [custom GITHUB_TOKEN](#custom-github_token) with the appropriate permissions +- You need to use this token when passing it to `dotnet-releaser run` +- You can then reference the badge with a URL directly to the SVG published. The file created in the gist by dotnet-releaser will be of the form `dotnet-releaser-coverage-badge-{owner}-{repo}.svg`. For example for the project [xoofx/TurboXml](https://github.com/xoofx/TurboXml/) the file created will be `dotnet-releaser-coverage-badge-xoofx-TurboXml.svg`. The link to the SVG would be simply: + ```md + ![coverage](https://gist.githubusercontent.com/xoofx/4b1dc8d0fa14dd6a3846e78e5f0eafae/raw/dotnet-releaser-coverage-badge-xoofx-TurboXml.svg) + ``` + Notice the gist ID in the URL that you should replace with your own gist ID. + ___ ### 2.5. Coveralls @@ -473,6 +506,7 @@ A group of packages in the TOML configuration is defined by: | `publish` | `bool` | You can disable a particular pack to be build/published. | `rid` | `string` | The target OS + CPU by defining its runtime identifier. See [https://docs.microsoft.com/en-us/dotnet/core/rid-catalog](https://docs.microsoft.com/en-us/dotnet/core/rid-catalog) for all the possible values. | `kinds` | `string` | Defines the kinds of package to create: `zip`, `tar`, `deb` or `rpm`. +| `renamer` | `renamer[]` | A regex renamer for the final filename. A renamer is defined by the object `{ pattern = "regex", replace = "replacement" }`. This is useful to replace a string in the package name. For each `rid` define in a pack, it will create the packages defined by `kinds`. @@ -519,6 +553,15 @@ kinds = ["zip"] By default, all packs declared are `publish = true`. +You can also replace the generated filename by using a replacer: + +```toml +[[pack]] +rid = ["win-x64"] +kinds = ["zip"] +renamer = [{ pattern = "win-x64", replace = "windows-amd64" }] +``` + ___ > `profile = "custom"` @@ -692,6 +735,54 @@ name = ["your-runtime1.0", "your-runtime2.0", "your-runtime3.0"] In order to specify dependencies for `rpm`, you can use a similar syntax with `[[rpm.depends]]`. +### 2.14. About GITHUB_TOKEN + +By default, GitHub Actions are providing a `GITHUB_TOKEN` for the repository. In order for dotnet-releaser to be able to use this token correctly you need to configure in the settings of your repository: + +- Go to the menu `Code and Automation/Actions/General`: + + ![Menu Code and Automation/Actions/General](./GIITHUB_TOKEN_menu_Actions_General.png) +- At the bottom of the page, under `Workflow permissions`, select `Read and write permissions`: + + ![Workflow permissions](GIITHUB_TOKEN_workflows_permission.png) + + +#### Custom GITHUB_TOKEN + +The default `GITHUB_TOKEN` should cover the basic scenarios, but you might want to create your own token to leverage more features of dotnet-releaser: + +- Automatic creation of [homebrew](#29-homebrew) repository +- Publishing [coverage badge on GitHub gist](#badge-coverage-to-gist) + +You can create and reuse the same token for all your repositories or you can create a token for each repository more selectively: + +First, go to the settings for the tokens https://github.com/settings/tokens + +**Standard tokens** + +You should tick the `public_repo` in the list: + +- [x] public_repo + +And put an appropriate expiration date. + +![GitHub Setup of Personal access token](https://raw.githubusercontent.com/xoofx/dotnet-releaser/main/img/github_new_personal_access_token.png) + +**Fine-grained tokens** + +1. Select `Fine-grained tokens (Beta)` and press the button `Generate new token` +2. Select an expiration date +3. Select the repository access + ![](./GITHUB_TOKEN_custom_repository_access.png) +4. In the `Repository permissions`, select: + - `Actions` ⇒ Access: Read and write + - `Administration` ⇒ Access: Read and write + - `Contents` ⇒ Access: Read and write +5. In the `Account permissions`, select: + - `Gists` ⇒ Access: Read and write + +You can then create the token and copy it and add it as a secrets environment for your GitHub Action workflows. You can use the name `PAT_GITHUB_TOKEN` with `PAT` for Personal Access Token. + ## 3. CLI Usage Get some help by typing `dotnet-releaser --help` diff --git a/readme.md b/readme.md index b1b91f6..ba82a85 100644 --- a/readme.md +++ b/readme.md @@ -1,4 +1,4 @@ -# dotnet-releaser [![Build Status](https://github.com/xoofx/dotnet-releaser/workflows/ci/badge.svg?branch=main)](https://github.com/xoofx/dotnet-releaser/actions) [![NuGet](https://img.shields.io/nuget/v/dotnet-releaser.svg)](https://www.nuget.org/packages/dotnet-releaser/) +# dotnet-releaser [![ci](https://github.com/xoofx/dotnet-releaser/actions/workflows/ci.yml/badge.svg)](https://github.com/xoofx/dotnet-releaser/actions/workflows/ci.yml) [![NuGet](https://img.shields.io/nuget/v/dotnet-releaser.svg)](https://www.nuget.org/packages/dotnet-releaser/) @@ -15,7 +15,7 @@ In practice, `dotnet-releaser` will automate the build and publish process of yo - `dotnet nuget push` to publish your package to a NuGet registry - [Pretty changelog](https://github.com/xoofx/dotnet-releaser/blob/main/doc/changelog_user_guide.md#11-overview) creation from pull-requests and commits. - Create and upload the changelog and all the packages packed to your GitHub repository associated with the release tag. -- It will publish automatically the coverage results to https://coveralls.io if your repository is created there. +- It can publish automatically the coverage results to a badge in a GitHub gist or to https://coveralls.io if your repository is created there. ![overview](https://raw.githubusercontent.com/xoofx/dotnet-releaser/main/doc/overview.drawio.svg) @@ -134,4 +134,4 @@ Regular .NET Libraries: - [DotNet.Glob](https://github.com/dazinator/DotNet.Glob) used by changelog filtering on files. ## Author -Alexandre Mutel aka [xoofx](http://xoofx.com). +Alexandre Mutel aka [xoofx](https://xoofx.github.io). diff --git a/src/DotNetReleaser.Tests/BasicTests.cs b/src/DotNetReleaser.Tests/BasicTests.cs index 3a9c491..8deb78a 100644 --- a/src/DotNetReleaser.Tests/BasicTests.cs +++ b/src/DotNetReleaser.Tests/BasicTests.cs @@ -1,6 +1,8 @@ using System; using System.Collections.Generic; +using System.Formats.Tar; using System.IO; +using System.IO.Compression; using System.Linq; using System.Runtime.InteropServices; using System.Text; @@ -74,6 +76,7 @@ public async Task TestBuild() config += @"[[pack]] rid = ""win-x64"" kinds = [""zip""] +renamer = [ { pattern = ""win-x64"", replace = ""windows-amd64"" } ] [[pack]] rid = ""linux-x64"" kinds = [""tar"", ""deb""] @@ -96,7 +99,7 @@ public async Task TestBuild() "HelloWorld.0.1.0.linux-x64.deb", "HelloWorld.0.1.0.linux-x64.tar.gz", "HelloWorld.0.1.0.nupkg", - "HelloWorld.0.1.0.win-x64.zip", + "HelloWorld.0.1.0.windows-amd64.zip", }.OrderBy(x => x).ToList(); foreach (var file in files) @@ -110,6 +113,97 @@ public async Task TestBuild() File.Delete(_configurationFile); } + [Test] + public async Task TestMacOSTarZipAreExecutable() + { + EnsureTestsFolder(); + + File.Delete(_configurationFile); + + await CreateConfiguration(); + + var config = await File.ReadAllTextAsync(_configurationFile); + + if (Directory.Exists(_artifactsFolder)) + { + Directory.Delete(_artifactsFolder, true); + } + + config = "profile = \"custom\"" + Environment.NewLine + config; + config += @" + [msbuild.properties] +SelfContained = false +PublishSingleFile = false +PublishTrimmed = false + [[pack]] +rid = ""osx-x64"" +kinds = [""tar"", ""zip""] +[nuget] +publish = false +"; + config = config.Replace("\r\n", "\n").Replace("\n", Environment.NewLine); + await File.WriteAllTextAsync(_configurationFile, config); + + var resultBuild = await CliWrap.Cli.Wrap(_releaserExe) + .WithArguments("build --force dotnet-releaser.toml") + .WithStandardOutputPipe(PipeTarget.ToDelegate(x => Console.Out.WriteLine(x))) + .WithStandardErrorPipe(PipeTarget.ToDelegate(x => Console.Error.WriteLine(x))) + .WithWorkingDirectory(_helloWorldFolder).ExecuteAsync(); + + Assert.True(Directory.Exists(_artifactsFolder)); + + var files = Directory.GetFiles(_artifactsFolder).Select(Path.GetFileName).OrderBy(x => x).ToList(); + + var expectedFiles = new List() + { + "HelloWorld.0.1.0.osx-x64.tar.gz", + "HelloWorld.0.1.0.osx-x64.zip", + }.OrderBy(x => x).ToList(); + + foreach (var file in files) + { + Console.WriteLine($"-> {file}"); + } + + Assert.AreEqual(expectedFiles, files); + + if (!OperatingSystem.IsWindows()) + { + // ensure files are executable + var tar = Path.Combine(_artifactsFolder, "HelloWorld.0.1.0.osx-x64.tar.gz"); + using FileStream fs = new(tar, FileMode.Open, FileAccess.Read); + using var gzip = new GZipStream(fs, CompressionMode.Decompress); + using var unzippedStream = new MemoryStream(); + { + await gzip.CopyToAsync(unzippedStream); + unzippedStream.Seek(0, SeekOrigin.Begin); + + using var reader = new TarReader(unzippedStream); + + while (reader.GetNextEntry() is TarEntry entry) + { + if (entry.Name == "./HelloWorld") + { + Assert.IsTrue(entry.Mode.HasFlag(UnixFileMode.GroupExecute)); + Assert.IsTrue(entry.Mode.HasFlag(UnixFileMode.OtherExecute)); + Assert.IsTrue(entry.Mode.HasFlag(UnixFileMode.UserExecute)); + break; + } + } + } + // extract zip files and check executable + var zippath = Path.Combine(_artifactsFolder, "HelloWorld.0.1.0.osx-x64.zip"); + ZipFile.ExtractToDirectory(zippath, _artifactsFolder); + var fileMode = File.GetUnixFileMode(Path.Combine(_artifactsFolder, "HelloWorld")); + Assert.IsTrue(fileMode.HasFlag(UnixFileMode.GroupExecute)); + Assert.IsTrue(fileMode.HasFlag(UnixFileMode.OtherExecute)); + Assert.IsTrue(fileMode.HasFlag(UnixFileMode.UserExecute)); + } + + Directory.Delete(_artifactsFolder, true); + File.Delete(_configurationFile); + } + [Test] public async Task TestBuildService() { @@ -157,7 +251,7 @@ public async Task TestBuildService() if (OperatingSystem.IsWindows()) { var wrap = await CliWrap.Cli.Wrap("wsl") - .WithArguments(new string[] { "-d", "Ubuntu-20.04", "--", "dpkg", "-x", Path.GetFileName(debArchive), "./tmp" }, true) + .WithArguments(new string[] { "--", "dpkg", "-x", Path.GetFileName(debArchive), "./tmp" }, true) .WithWorkingDirectory(_artifactsFolder) .ExecuteAsync(); } @@ -204,7 +298,7 @@ public async Task TestBuildService() if (OperatingSystem.IsWindows()) { var wrap = await CliWrap.Cli.Wrap("wsl") - .WithArguments(new string[] { "-d", "Ubuntu-20.04", "--", "dpkg", "--info", Path.GetFileName(debArchive)}, true) + .WithArguments(new string[] { "--", "dpkg", "--info", Path.GetFileName(debArchive)}, true) .WithWorkingDirectory(_artifactsFolder) .WithStandardOutputPipe(PipeTarget.ToStringBuilder(packageInfoOutput)) .ExecuteAsync(); diff --git a/src/DotNetReleaser.Tests/DotNetReleaser.Tests.csproj b/src/DotNetReleaser.Tests/DotNetReleaser.Tests.csproj index a6c602b..e70cab5 100644 --- a/src/DotNetReleaser.Tests/DotNetReleaser.Tests.csproj +++ b/src/DotNetReleaser.Tests/DotNetReleaser.Tests.csproj @@ -1,7 +1,7 @@ - + - net7.0 + net9.0 enable false @@ -21,15 +21,23 @@ - - - - + + + + all runtime; build; native; contentfiles; analyzers; buildtransitive + + + + + + + + diff --git a/src/MsBuildPipeLogger.Logger/AnonymousPipeWriter.cs b/src/MsBuildPipeLogger.Logger/AnonymousPipeWriter.cs new file mode 100644 index 0000000..6340bc8 --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/AnonymousPipeWriter.cs @@ -0,0 +1,15 @@ +using System.IO.Pipes; + +namespace MsBuildPipeLogger +{ + public class AnonymousPipeWriter : PipeWriter + { + public string Handle { get; } + + public AnonymousPipeWriter(string pipeHandleAsString) + : base(new AnonymousPipeClientStream(PipeDirection.Out, pipeHandleAsString)) + { + Handle = pipeHandleAsString; + } + } +} diff --git a/src/MsBuildPipeLogger.Logger/BinaryLogger/BuildEventArgsFieldFlags.cs b/src/MsBuildPipeLogger.Logger/BinaryLogger/BuildEventArgsFieldFlags.cs new file mode 100644 index 0000000..186eda6 --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/BinaryLogger/BuildEventArgsFieldFlags.cs @@ -0,0 +1,27 @@ +using System; + +namespace Microsoft.Build.Logging +{ + /// + /// A bitmask to specify which fields on a BuildEventArgs object are present; used in serialization. + /// + [Flags] + internal enum BuildEventArgsFieldFlags + { + None = 0, + BuildEventContext = 1 << 0, + HelpHeyword = 1 << 1, + Message = 1 << 2, + SenderName = 1 << 3, + ThreadId = 1 << 4, + Timestamp = 1 << 5, + Subcategory = 1 << 6, + Code = 1 << 7, + File = 1 << 8, + ProjectFile = 1 << 9, + LineNumber = 1 << 10, + ColumnNumber = 1 << 11, + EndLineNumber = 1 << 12, + EndColumnNumber = 1 << 13 + } +} diff --git a/src/MsBuildPipeLogger.Logger/BinaryLogger/BuildEventArgsWriter.cs b/src/MsBuildPipeLogger.Logger/BinaryLogger/BuildEventArgsWriter.cs new file mode 100644 index 0000000..009e5f4 --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/BinaryLogger/BuildEventArgsWriter.cs @@ -0,0 +1,769 @@ +using System; +using System.Collections; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using Microsoft.Build.Framework; +using Microsoft.Build.Framework.Profiler; + +namespace Microsoft.Build.Logging +{ + /// + /// Serializes BuildEventArgs-derived objects into a provided BinaryWriter. + /// + internal class BuildEventArgsWriter + { + private readonly BinaryWriter _binaryWriter; + + /// + /// Initializes a new instance of BuildEventArgsWriter with a BinaryWriter. + /// + /// A BinaryWriter to write the BuildEventArgs instances to. + public BuildEventArgsWriter(BinaryWriter binaryWriter) + { + _binaryWriter = binaryWriter; + } + + /// + /// Write a provided instance of BuildEventArgs to the BinaryWriter. + /// + public void Write(BuildEventArgs e) + { + string type = e.GetType().Name; + + // the cases are ordered by most used first for performance + if (e is BuildMessageEventArgs && type != "ProjectImportedEventArgs" && type != "TargetSkippedEventArgs") + { + Write((BuildMessageEventArgs)e); + } + else if (e is TaskStartedEventArgs taskStartedEventArgs) + { + Write(taskStartedEventArgs); + } + else if (e is TaskFinishedEventArgs taskFinishedEventArgs) + { + Write(taskFinishedEventArgs); + } + else if (e is TargetStartedEventArgs targetStartedEventArgs) + { + Write(targetStartedEventArgs); + } + else if (e is TargetFinishedEventArgs targetFinishedEventArgs) + { + Write(targetFinishedEventArgs); + } + else if (e is BuildErrorEventArgs buildErrorEventArgs) + { + Write(buildErrorEventArgs); + } + else if (e is BuildWarningEventArgs buildWarningEventArgs) + { + Write(buildWarningEventArgs); + } + else if (e is ProjectStartedEventArgs projectStartedEventArgs) + { + Write(projectStartedEventArgs); + } + else if (e is ProjectFinishedEventArgs projectFinishedEventArgs) + { + Write(projectFinishedEventArgs); + } + else if (e is BuildStartedEventArgs buildStartedEventArgs) + { + Write(buildStartedEventArgs); + } + else if (e is BuildFinishedEventArgs buildFinishedEventArgs) + { + Write(buildFinishedEventArgs); + } + else if (e is ProjectEvaluationStartedEventArgs projectEvaluationStartedEventArgs) + { + Write(projectEvaluationStartedEventArgs); + } + else if (e is ProjectEvaluationFinishedEventArgs projectEvaluationFinishedEventArgs) + { + Write(projectEvaluationFinishedEventArgs); + } + + // The following cases are due to the fact that StructuredLogger.dll + // only references MSBuild 14.0 .dlls. The following BuildEventArgs types + // were only introduced in MSBuild 15.3 so we can't refer to them statically. + // To still provide a good experience to those who are using the BinaryLogger + // from StructuredLogger.dll against MSBuild 15.3 or later we need to preserve + // these new events, so use reflection to create our "equivalents" of those + // and populate them to be binary identical to the originals. Then serialize + // our copies so that it's impossible to tell what wrote these. + else if (type == "ProjectEvaluationStartedEventArgs") + { + ProjectEvaluationStartedEventArgs evaluationStarted = new ProjectEvaluationStartedEventArgs(e.Message) + { + BuildEventContext = e.BuildEventContext, + ProjectFile = Reflector.GetProjectFileFromEvaluationStarted(e) + }; + Write(evaluationStarted); + } + else if (type == "ProjectEvaluationFinishedEventArgs") + { + ProjectEvaluationFinishedEventArgs evaluationFinished = new ProjectEvaluationFinishedEventArgs(e.Message) + { + BuildEventContext = e.BuildEventContext, + ProjectFile = Reflector.GetProjectFileFromEvaluationFinished(e) + }; + Write(evaluationFinished); + } + else if (type == "ProjectImportedEventArgs") + { + BuildMessageEventArgs message = e as BuildMessageEventArgs; + ProjectImportedEventArgs projectImported = new ProjectImportedEventArgs(message.LineNumber, message.ColumnNumber, e.Message) + { + BuildEventContext = e.BuildEventContext, + ProjectFile = message.ProjectFile, + ImportedProjectFile = Reflector.GetImportedProjectFile(e), + UnexpandedProject = Reflector.GetUnexpandedProject(e) + }; + Write(projectImported); + } + else if (type == "TargetSkippedEventArgs") + { + BuildMessageEventArgs message = e as BuildMessageEventArgs; + TargetSkippedEventArgs targetSkipped = new TargetSkippedEventArgs(e.Message) + { + BuildEventContext = e.BuildEventContext, + ProjectFile = message.ProjectFile, + TargetName = Reflector.GetTargetNameFromTargetSkipped(e), + TargetFile = Reflector.GetTargetFileFromTargetSkipped(e), + ParentTarget = Reflector.GetParentTargetFromTargetSkipped(e), + BuildReason = Reflector.GetBuildReasonFromTargetSkipped(e) + }; + Write(targetSkipped); + } + else + { + // convert all unrecognized objects to message + // and just preserve the message + BuildMessageEventArgs buildMessageEventArgs = new BuildMessageEventArgs( + e.Message, + e.HelpKeyword, + e.SenderName, + MessageImportance.Normal, + e.Timestamp) + { + BuildEventContext = e.BuildEventContext ?? BuildEventContext.Invalid + }; + Write(buildMessageEventArgs); + } + } + + public void WriteBlob(BinaryLogRecordKind kind, byte[] bytes) + { + Write(kind); + Write(bytes.Length); + Write(bytes); + } + + private void Write(BuildStartedEventArgs e) + { + Write(BinaryLogRecordKind.BuildStarted); + WriteBuildEventArgsFields(e); + Write(e.BuildEnvironment); + } + + private void Write(BuildFinishedEventArgs e) + { + Write(BinaryLogRecordKind.BuildFinished); + WriteBuildEventArgsFields(e); + Write(e.Succeeded); + } + + private void Write(ProjectEvaluationStartedEventArgs e) + { + Write(BinaryLogRecordKind.ProjectEvaluationStarted); + WriteBuildEventArgsFields(e); + Write(e.ProjectFile); + } + + private void Write(ProjectEvaluationFinishedEventArgs e) + { + Write(BinaryLogRecordKind.ProjectEvaluationFinished); + + WriteBuildEventArgsFields(e); + Write(e.ProjectFile); + + Write(e.ProfilerResult.HasValue); + if (e.ProfilerResult.HasValue) + { + Write(e.ProfilerResult.Value.ProfiledLocations.Count); + + foreach (KeyValuePair item in e.ProfilerResult.Value.ProfiledLocations) + { + Write(item.Key); + Write(item.Value); + } + } + } + + private void Write(ProjectStartedEventArgs e) + { + Write(BinaryLogRecordKind.ProjectStarted); + WriteBuildEventArgsFields(e); + + if (e.ParentProjectBuildEventContext == null) + { + Write(false); + } + else + { + Write(true); + Write(e.ParentProjectBuildEventContext); + } + + WriteOptionalString(e.ProjectFile); + + Write(e.ProjectId); + Write(e.TargetNames); + WriteOptionalString(e.ToolsVersion); + + if (e.GlobalProperties == null) + { + Write(false); + } + else + { + Write(true); + Write(e.GlobalProperties); + } + + WriteProperties(e.Properties); + + WriteItems(e.Items); + } + + private void Write(ProjectFinishedEventArgs e) + { + Write(BinaryLogRecordKind.ProjectFinished); + WriteBuildEventArgsFields(e); + WriteOptionalString(e.ProjectFile); + Write(e.Succeeded); + } + + private void Write(TargetStartedEventArgs e) + { + Write(BinaryLogRecordKind.TargetStarted); + WriteBuildEventArgsFields(e); + WriteOptionalString(e.TargetName); + WriteOptionalString(e.ProjectFile); + WriteOptionalString(e.TargetFile); + WriteOptionalString(e.ParentTarget); + Write((int)Reflector.GetBuildReasonFromTargetStarted(e)); + } + + private void Write(TargetFinishedEventArgs e) + { + Write(BinaryLogRecordKind.TargetFinished); + WriteBuildEventArgsFields(e); + Write(e.Succeeded); + WriteOptionalString(e.ProjectFile); + WriteOptionalString(e.TargetFile); + WriteOptionalString(e.TargetName); + WriteItemList(e.TargetOutputs); + } + + private void Write(TaskStartedEventArgs e) + { + Write(BinaryLogRecordKind.TaskStarted); + WriteBuildEventArgsFields(e); + WriteOptionalString(e.TaskName); + WriteOptionalString(e.ProjectFile); + WriteOptionalString(e.TaskFile); + } + + private void Write(TaskFinishedEventArgs e) + { + Write(BinaryLogRecordKind.TaskFinished); + WriteBuildEventArgsFields(e); + Write(e.Succeeded); + WriteOptionalString(e.TaskName); + WriteOptionalString(e.ProjectFile); + WriteOptionalString(e.TaskFile); + } + + private void Write(BuildErrorEventArgs e) + { + Write(BinaryLogRecordKind.Error); + WriteBuildEventArgsFields(e); + WriteOptionalString(e.Subcategory); + WriteOptionalString(e.Code); + WriteOptionalString(e.File); + WriteOptionalString(e.ProjectFile); + Write(e.LineNumber); + Write(e.ColumnNumber); + Write(e.EndLineNumber); + Write(e.EndColumnNumber); + } + + private void Write(BuildWarningEventArgs e) + { + Write(BinaryLogRecordKind.Warning); + WriteBuildEventArgsFields(e); + WriteOptionalString(e.Subcategory); + WriteOptionalString(e.Code); + WriteOptionalString(e.File); + WriteOptionalString(e.ProjectFile); + Write(e.LineNumber); + Write(e.ColumnNumber); + Write(e.EndLineNumber); + Write(e.EndColumnNumber); + } + + private void Write(BuildMessageEventArgs e) + { + if (e is CriticalBuildMessageEventArgs criticalBuildMessageEventArgs) + { + Write(criticalBuildMessageEventArgs); + return; + } + + if (e is TaskCommandLineEventArgs taskCommandLineEventArgs) + { + Write(taskCommandLineEventArgs); + return; + } + + if (e is ProjectImportedEventArgs projectImportedEventArgs) + { + Write(projectImportedEventArgs); + return; + } + + if (e is TargetSkippedEventArgs targetSkippedEventArgs) + { + Write(targetSkippedEventArgs); + return; + } + + Write(BinaryLogRecordKind.Message); + WriteMessageFields(e); + } + + private void Write(ProjectImportedEventArgs e) + { + Write(BinaryLogRecordKind.ProjectImported); + WriteMessageFields(e); + Write(e.ImportIgnored); + WriteOptionalString(e.ImportedProjectFile); + WriteOptionalString(e.UnexpandedProject); + } + + private void Write(TargetSkippedEventArgs e) + { + Write(BinaryLogRecordKind.TargetSkipped); + WriteMessageFields(e); + WriteOptionalString(e.TargetFile); + WriteOptionalString(e.TargetName); + WriteOptionalString(e.ParentTarget); + Write((int)e.BuildReason); + } + + private void Write(CriticalBuildMessageEventArgs e) + { + Write(BinaryLogRecordKind.CriticalBuildMessage); + WriteMessageFields(e); + } + + private void Write(TaskCommandLineEventArgs e) + { + Write(BinaryLogRecordKind.TaskCommandLine); + WriteMessageFields(e); + WriteOptionalString(e.CommandLine); + WriteOptionalString(e.TaskName); + } + + private void WriteBuildEventArgsFields(BuildEventArgs e) + { + BuildEventArgsFieldFlags flags = GetBuildEventArgsFieldFlags(e); + Write((int)flags); + WriteBaseFields(e, flags); + } + + private void WriteBaseFields(BuildEventArgs e, BuildEventArgsFieldFlags flags) + { + if ((flags & BuildEventArgsFieldFlags.Message) != 0) + { + Write(e.Message); + } + + if ((flags & BuildEventArgsFieldFlags.BuildEventContext) != 0) + { + Write(e.BuildEventContext); + } + + if ((flags & BuildEventArgsFieldFlags.ThreadId) != 0) + { + Write(e.ThreadId); + } + + if ((flags & BuildEventArgsFieldFlags.HelpHeyword) != 0) + { + Write(e.HelpKeyword); + } + + if ((flags & BuildEventArgsFieldFlags.SenderName) != 0) + { + Write(e.SenderName); + } + + if ((flags & BuildEventArgsFieldFlags.Timestamp) != 0) + { + Write(e.Timestamp); + } + } + + private void WriteMessageFields(BuildMessageEventArgs e) + { + BuildEventArgsFieldFlags flags = GetBuildEventArgsFieldFlags(e); + flags = GetMessageFlags(e, flags); + + Write((int)flags); + + WriteBaseFields(e, flags); + + if ((flags & BuildEventArgsFieldFlags.Subcategory) != 0) + { + Write(e.Subcategory); + } + + if ((flags & BuildEventArgsFieldFlags.Code) != 0) + { + Write(e.Code); + } + + if ((flags & BuildEventArgsFieldFlags.File) != 0) + { + Write(e.File); + } + + if ((flags & BuildEventArgsFieldFlags.ProjectFile) != 0) + { + Write(e.ProjectFile); + } + + if ((flags & BuildEventArgsFieldFlags.LineNumber) != 0) + { + Write(e.LineNumber); + } + + if ((flags & BuildEventArgsFieldFlags.ColumnNumber) != 0) + { + Write(e.ColumnNumber); + } + + if ((flags & BuildEventArgsFieldFlags.EndLineNumber) != 0) + { + Write(e.EndLineNumber); + } + + if ((flags & BuildEventArgsFieldFlags.EndColumnNumber) != 0) + { + Write(e.EndColumnNumber); + } + + Write((int)e.Importance); + } + + private static BuildEventArgsFieldFlags GetMessageFlags(BuildMessageEventArgs e, BuildEventArgsFieldFlags flags) + { + if (e.Subcategory != null) + { + flags |= BuildEventArgsFieldFlags.Subcategory; + } + + if (e.Code != null) + { + flags |= BuildEventArgsFieldFlags.Code; + } + + if (e.File != null) + { + flags |= BuildEventArgsFieldFlags.File; + } + + if (e.ProjectFile != null) + { + flags |= BuildEventArgsFieldFlags.ProjectFile; + } + + if (e.LineNumber != 0) + { + flags |= BuildEventArgsFieldFlags.LineNumber; + } + + if (e.ColumnNumber != 0) + { + flags |= BuildEventArgsFieldFlags.ColumnNumber; + } + + if (e.EndLineNumber != 0) + { + flags |= BuildEventArgsFieldFlags.EndLineNumber; + } + + if (e.EndColumnNumber != 0) + { + flags |= BuildEventArgsFieldFlags.EndColumnNumber; + } + + return flags; + } + + private static BuildEventArgsFieldFlags GetBuildEventArgsFieldFlags(BuildEventArgs e) + { + BuildEventArgsFieldFlags flags = BuildEventArgsFieldFlags.None; + if (e.BuildEventContext != null) + { + flags |= BuildEventArgsFieldFlags.BuildEventContext; + } + + if (e.HelpKeyword != null) + { + flags |= BuildEventArgsFieldFlags.HelpHeyword; + } + + if (!string.IsNullOrEmpty(e.Message)) + { + flags |= BuildEventArgsFieldFlags.Message; + } + + // no need to waste space for the default sender name + if (e.SenderName != null && e.SenderName != "MSBuild") + { + flags |= BuildEventArgsFieldFlags.SenderName; + } + + if (e.ThreadId > 0) + { + flags |= BuildEventArgsFieldFlags.ThreadId; + } + + if (e.Timestamp != default(DateTime)) + { + flags |= BuildEventArgsFieldFlags.Timestamp; + } + + return flags; + } + + private void WriteItemList(IEnumerable items) + { + if (items is IEnumerable taskItems) + { + Write(taskItems.Count()); + + foreach (ITaskItem item in taskItems) + { + Write(item); + } + + return; + } + + Write(0); + } + + private void WriteItems(IEnumerable items) + { + if (items == null) + { + Write(0); + return; + } + + DictionaryEntry[] entries = items.OfType() + .Where(e => e.Key is string && e.Value is ITaskItem) + .ToArray(); + Write(entries.Length); + + foreach (DictionaryEntry entry in entries) + { + string key = entry.Key as string; + ITaskItem item = entry.Value as ITaskItem; + Write(key); + Write(item); + } + } + + private void Write(ITaskItem item) + { + Write(item.ItemSpec); + IDictionary customMetadata = item.CloneCustomMetadata(); + Write(customMetadata.Count); + + foreach (string metadataName in customMetadata.Keys) + { + Write(metadataName); + Write(item.GetMetadata(metadataName)); + } + } + + private void WriteProperties(IEnumerable properties) + { + if (properties == null) + { + Write(0); + return; + } + + // there are no guarantees that the properties iterator won't change, so + // take a snapshot and work with the readonly copy + DictionaryEntry[] propertiesArray = properties.OfType().ToArray(); + + Write(propertiesArray.Length); + + foreach (DictionaryEntry entry in propertiesArray) + { + if (entry.Key is string && entry.Value is string) + { + Write((string)entry.Key); + Write((string)entry.Value); + } + else + { + // to keep the count accurate + Write(string.Empty); + Write(string.Empty); + } + } + } + + private void Write(BuildEventContext buildEventContext) + { + Write(buildEventContext.NodeId); + Write(buildEventContext.ProjectContextId); + Write(buildEventContext.TargetId); + Write(buildEventContext.TaskId); + Write(buildEventContext.SubmissionId); + Write(buildEventContext.ProjectInstanceId); + Write(Reflector.GetEvaluationId(buildEventContext)); + } + + private void Write(IEnumerable> keyValuePairs) + { + if (keyValuePairs?.Any() == true) + { + Write(keyValuePairs.Count()); + foreach (KeyValuePair kvp in keyValuePairs) + { + Write(kvp.Key.ToString()); + Write(kvp.Value.ToString()); + } + } + else + { + Write(false); + } + } + + private void Write(BinaryLogRecordKind kind) + { + Write((int)kind); + } + + private void Write(int value) + { + Write7BitEncodedInt(_binaryWriter, value); + } + + private void Write(long value) + { + _binaryWriter.Write(value); + } + + private void Write7BitEncodedInt(BinaryWriter writer, int value) + { + // Write out an int 7 bits at a time. The high bit of the byte, + // when on, tells reader to continue reading more bytes. + uint v = (uint)value; // support negative numbers + while (v >= 0x80) + { + writer.Write((byte)(v | 0x80)); + v >>= 7; + } + writer.Write((byte)v); + } + + private void Write(byte[] bytes) + { + _binaryWriter.Write(bytes); + } + + private void Write(bool boolean) + { + _binaryWriter.Write(boolean); + } + + private void Write(string text) + { + if (text != null) + { + _binaryWriter.Write(text); + } + else + { + _binaryWriter.Write(false); + } + } + + private void WriteOptionalString(string text) + { + if (text == null) + { + Write(false); + } + else + { + Write(true); + Write(text); + } + } + + private void Write(DateTime timestamp) + { + _binaryWriter.Write(timestamp.Ticks); + Write((int)timestamp.Kind); + } + + private void Write(TimeSpan timeSpan) + { + _binaryWriter.Write(timeSpan.Ticks); + } + + private void Write(EvaluationLocation item) + { + WriteOptionalString(item.ElementName); + WriteOptionalString(item.ElementDescription); + WriteOptionalString(item.EvaluationPassDescription); + WriteOptionalString(item.File); + Write((int)item.Kind); + Write((int)item.EvaluationPass); + + Write(item.Line.HasValue); + if (item.Line.HasValue) + { + Write(item.Line.Value); + } + + Write(item.Id); + Write(item.ParentId.HasValue); + if (item.ParentId.HasValue) + { + Write(item.ParentId.Value); + } + } + + private void Write(ProfiledLocation e) + { + Write(e.NumberOfHits); + Write(e.ExclusiveTime); + Write(e.InclusiveTime); + } + } +} diff --git a/src/MsBuildPipeLogger.Logger/BinaryLogger/EvaluationIdProvider.cs b/src/MsBuildPipeLogger.Logger/BinaryLogger/EvaluationIdProvider.cs new file mode 100644 index 0000000..10ef02b --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/BinaryLogger/EvaluationIdProvider.cs @@ -0,0 +1,40 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. +//----------------------------------------------------------------------- +// +//----------------------------------------------------------------------- + +using System; +using System.Diagnostics; +using System.Threading; + +namespace Microsoft.Build.Framework.Profiler +{ + /// + /// Assigns unique evaluation ids. Thread safe. + /// + internal static class EvaluationIdProvider + { + private static readonly long ProcessId = Process.GetCurrentProcess().Id; + private static long _sAssignedId = -1; + + /// + /// Returns a unique evaluation id. + /// + /// + /// The id is guaranteed to be unique across all running processes. + /// Additionally, it is monotonically increasing for callers on the same process id. + /// + public static long GetNextId() + { + checked + { + long nextId = Interlocked.Increment(ref _sAssignedId); + + // Returns a unique number based on nextId (a unique number for this process) and the current process Id + // Uses the Cantor pairing function (https://en.wikipedia.org/wiki/Pairing_function) to guarantee uniqueness + return (((nextId + ProcessId) * (nextId + ProcessId + 1)) / 2) + ProcessId; + } + } + } +} diff --git a/src/MsBuildPipeLogger.Logger/BinaryLogger/EvaluationLocation.cs b/src/MsBuildPipeLogger.Logger/BinaryLogger/EvaluationLocation.cs new file mode 100644 index 0000000..f795104 --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/BinaryLogger/EvaluationLocation.cs @@ -0,0 +1,302 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. +//----------------------------------------------------------------------- +// +// Location for different elements tracked by the evaluation profiler. +//----------------------------------------------------------------------- + +using System; +using System.Collections.Generic; + +namespace Microsoft.Build.Framework.Profiler +{ + /// + /// Evaluation main phases used by the profiler. + /// + /// + /// Order matters since the profiler pretty printer orders profiled items from top to bottom using + /// the pass they belong to. + /// + public enum EvaluationPass : byte + { + TotalEvaluation = 0, + TotalGlobbing = 1, + InitialProperties = 2, + Properties = 3, + ItemDefinitionGroups = 4, + Items = 5, + LazyItems = 6, + UsingTasks = 7, + Targets = 8 + } + + /// + /// The kind of the evaluated location being tracked. + /// + public enum EvaluationLocationKind : byte + { + Element = 0, + Condition = 1, + Glob = 2 + } + + /// + /// Represents a location for different evaluation elements tracked by the EvaluationProfiler. + /// +#if FEATURE_BINARY_SERIALIZATION + [Serializable] +#endif + public struct EvaluationLocation + { + /// + /// Default descriptions for locations that are used in case a description is not provided. + /// + private static readonly Dictionary PassDefaultDescription = + new Dictionary + { + { EvaluationPass.TotalEvaluation, "Total evaluation" }, + { EvaluationPass.TotalGlobbing, "Total evaluation for globbing" }, + { EvaluationPass.InitialProperties, "Initial properties (pass 0)" }, + { EvaluationPass.Properties, "Properties (pass 1)" }, + { EvaluationPass.ItemDefinitionGroups, "Item definition groups (pass 2)" }, + { EvaluationPass.Items, "Items (pass 3)" }, + { EvaluationPass.LazyItems, "Lazy items (pass 3.1)" }, + { EvaluationPass.UsingTasks, "Using tasks (pass 4)" }, + { EvaluationPass.Targets, "Targets (pass 5)" }, + }; + + /// + public long Id { get; } + + /// + public long? ParentId { get; } + + /// + public EvaluationPass EvaluationPass { get; } + + /// + public string EvaluationPassDescription { get; } + + /// + public string File { get; } + + /// + public int? Line { get; } + + /// + public string ElementName { get; } + + /// + public string ElementDescription { get; } + + /// + public EvaluationLocationKind Kind { get; } + + /// + public bool IsEvaluationPass => File == null; + + /// + public static EvaluationLocation CreateLocationForCondition( + long? parentId, + EvaluationPass evaluationPass, + string evaluationDescription, + string file, + int? line, + string condition) + { + return new EvaluationLocation(parentId, evaluationPass, evaluationDescription, file, line, "Condition", condition, kind: EvaluationLocationKind.Condition); + } + + /// + public static EvaluationLocation CreateLocationForGlob( + long? parentId, + EvaluationPass evaluationPass, + string evaluationDescription, + string file, + int? line, + string globDescription) + { + return new EvaluationLocation(parentId, evaluationPass, evaluationDescription, file, line, "Glob", globDescription, kind: EvaluationLocationKind.Glob); + } + + /// + public static EvaluationLocation CreateLocationForAggregatedGlob() + { + return new EvaluationLocation( + EvaluationPass.TotalGlobbing, + PassDefaultDescription[EvaluationPass.TotalGlobbing], + file: null, + line: null, + elementName: null, + elementDescription: null, + kind: EvaluationLocationKind.Glob); + } + + /// + /// Constructs a generic evaluation location. + /// + /// + /// Used by serialization/deserialization purposes. + /// + public EvaluationLocation( + long id, + long? parentId, + EvaluationPass evaluationPass, + string evaluationPassDescription, + string file, + int? line, + string elementName, + string elementDescription, + EvaluationLocationKind kind) + { + Id = id; + ParentId = parentId == EmptyLocation.Id ? null : parentId; // The empty location doesn't count as a parent id, since it's just a dummy starting point + EvaluationPass = evaluationPass; + EvaluationPassDescription = evaluationPassDescription; + File = file; + Line = line; + ElementName = elementName; + ElementDescription = elementDescription; + Kind = kind; + } + + /// + /// Constructs a generic evaluation location based on a (possibly null) parent Id. + /// + /// + /// A unique Id gets assigned automatically + /// Used by serialization/deserialization purposes. + /// + public EvaluationLocation(long? parentId, EvaluationPass evaluationPass, string evaluationPassDescription, string file, int? line, string elementName, string elementDescription, EvaluationLocationKind kind) + : this(EvaluationIdProvider.GetNextId(), parentId, evaluationPass, evaluationPassDescription, file, line, elementName, elementDescription, kind) + { + } + + /// + /// Constructs a generic evaluation location with no parent. + /// + /// + /// A unique Id gets assigned automatically + /// Used by serialization/deserialization purposes. + /// + public EvaluationLocation(EvaluationPass evaluationPass, string evaluationPassDescription, string file, int? line, string elementName, string elementDescription, EvaluationLocationKind kind) + : this(null, evaluationPass, evaluationPassDescription, file, line, elementName, elementDescription, kind) + { + } + + /// + /// An empty location, used as the starting instance. + /// + public static EvaluationLocation EmptyLocation { get; } = CreateEmptyLocation(); + + /// + public EvaluationLocation WithEvaluationPass(EvaluationPass evaluationPass, string passDescription = null) + { + return new EvaluationLocation( + Id, + evaluationPass, + passDescription ?? PassDefaultDescription[evaluationPass], + File, + Line, + ElementName, + ElementDescription, + Kind); + } + + /// + public EvaluationLocation WithParentId(long? parentId) + { + // Simple optimization. If the new parent id is the same as the current one, then we just return this + if (parentId == ParentId) + { + return this; + } + + return new EvaluationLocation( + Id, + parentId, + EvaluationPass, + EvaluationPassDescription, + File, + Line, + ElementName, + ElementDescription, + Kind); + } + + /// + public EvaluationLocation WithFile(string file) + { + return new EvaluationLocation(Id, EvaluationPass, EvaluationPassDescription, file, null, null, null, Kind); + } + + /// + public EvaluationLocation WithFileLineAndCondition(string file, int? line, string condition) + { + return CreateLocationForCondition(Id, EvaluationPass, EvaluationPassDescription, file, line, condition); + } + + /// + public EvaluationLocation WithGlob(string globDescription) + { + return CreateLocationForGlob(Id, EvaluationPass, EvaluationPassDescription, File, Line, globDescription); + } + + /// + public override bool Equals(object obj) + { + if (obj is EvaluationLocation evaluationLocation) + { + EvaluationLocation other = evaluationLocation; + return + Id == other.Id + && ParentId == other.ParentId + && EvaluationPass == other.EvaluationPass + && EvaluationPassDescription == other.EvaluationPassDescription + && string.Equals(File, other.File, StringComparison.OrdinalIgnoreCase) + && Line == other.Line + && ElementName == other.ElementName + && ElementDescription == other.ElementDescription + && Kind == other.Kind; + } + return false; + } + + /// + public override string ToString() + { + return + $"{Id}\t{ParentId?.ToString() ?? string.Empty}\t{EvaluationPassDescription ?? string.Empty}\t{File ?? string.Empty}\t{Line?.ToString() ?? string.Empty}\t{ElementName ?? string.Empty}\tDescription:{ElementDescription}\t{EvaluationPassDescription}"; + } + + /// + public override int GetHashCode() + { + int hashCode = 1198539463; + hashCode = (hashCode * -1521134295) + base.GetHashCode(); + hashCode = (hashCode * -1521134295) + Id.GetHashCode(); + hashCode = (hashCode * -1521134295) + EqualityComparer.Default.GetHashCode(ParentId); + hashCode = (hashCode * -1521134295) + EvaluationPass.GetHashCode(); + hashCode = (hashCode * -1521134295) + EqualityComparer.Default.GetHashCode(EvaluationPassDescription); + hashCode = (hashCode * -1521134295) + EqualityComparer.Default.GetHashCode(File); + hashCode = (hashCode * -1521134295) + EqualityComparer.Default.GetHashCode(Line); + hashCode = (hashCode * -1521134295) + EqualityComparer.Default.GetHashCode(ElementName); + hashCode = (hashCode * -1521134295) + EqualityComparer.Default.GetHashCode(ElementDescription); + return (hashCode * -1521134295) + Kind.GetHashCode(); + } + + private static EvaluationLocation CreateEmptyLocation() + { + return new EvaluationLocation( + EvaluationIdProvider.GetNextId(), + null, + default(EvaluationPass), + null, + null, + null, + null, + null, + default(EvaluationLocationKind)); + } + } +} diff --git a/src/MsBuildPipeLogger.Logger/BinaryLogger/ProfilerResult.cs b/src/MsBuildPipeLogger.Logger/BinaryLogger/ProfilerResult.cs new file mode 100644 index 0000000..26557a8 --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/BinaryLogger/ProfilerResult.cs @@ -0,0 +1,109 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. +//----------------------------------------------------------------------- +// +// The result profiling an evaluation. +//----------------------------------------------------------------------- + +using System; +using System.Collections.Generic; +using System.Collections.ObjectModel; +using System.Linq; + +namespace Microsoft.Build.Framework.Profiler +{ + /// + /// Result of profiling an evaluation. + /// +#if FEATURE_BINARY_SERIALIZATION + [Serializable] +#endif + public struct ProfilerResult + { + /// + public IReadOnlyDictionary ProfiledLocations { get; } + + /// + public ProfilerResult(IDictionary profiledLocations) + { + ProfiledLocations = new ReadOnlyDictionary(profiledLocations); + } + + /// + public override bool Equals(object obj) + { + if (!(obj is ProfilerResult)) + { + return false; + } + + ProfilerResult result = (ProfilerResult)obj; + + return (ProfiledLocations == result.ProfiledLocations) + || (ProfiledLocations.Count == result.ProfiledLocations.Count + && !ProfiledLocations.Except(result.ProfiledLocations).Any()); + } + + /// + public override int GetHashCode() + { + return ProfiledLocations.Keys.Aggregate(0, (acum, location) => acum + location.GetHashCode()); + } + } + + /// + /// Result of timing the evaluation of a given element at a given location. + /// +#if FEATURE_BINARY_SERIALIZATION + [Serializable] +#endif + public struct ProfiledLocation + { + /// + public TimeSpan InclusiveTime { get; } + + /// + public TimeSpan ExclusiveTime { get; } + + /// + public int NumberOfHits { get; } + + /// + public ProfiledLocation(TimeSpan inclusiveTime, TimeSpan exclusiveTime, int numberOfHits) + { + InclusiveTime = inclusiveTime; + ExclusiveTime = exclusiveTime; + NumberOfHits = numberOfHits; + } + + /// + public override bool Equals(object obj) + { + if (!(obj is ProfiledLocation)) + { + return false; + } + + ProfiledLocation location = (ProfiledLocation)obj; + return InclusiveTime.Equals(location.InclusiveTime) + && ExclusiveTime.Equals(location.ExclusiveTime) + && NumberOfHits == location.NumberOfHits; + } + + /// + public override int GetHashCode() + { + int hashCode = -2131368567; + hashCode = (hashCode * -1521134295) + base.GetHashCode(); + hashCode = (hashCode * -1521134295) + EqualityComparer.Default.GetHashCode(InclusiveTime); + hashCode = (hashCode * -1521134295) + EqualityComparer.Default.GetHashCode(ExclusiveTime); + return (hashCode * -1521134295) + NumberOfHits.GetHashCode(); + } + + /// + public override string ToString() + { + return $"[{InclusiveTime} - {ExclusiveTime}]: {NumberOfHits} hits"; + } + } +} diff --git a/src/MsBuildPipeLogger.Logger/BinaryLogger/ProjectEvaluationFinishedEventArgs.cs b/src/MsBuildPipeLogger.Logger/BinaryLogger/ProjectEvaluationFinishedEventArgs.cs new file mode 100644 index 0000000..7b25405 --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/BinaryLogger/ProjectEvaluationFinishedEventArgs.cs @@ -0,0 +1,45 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; +using Microsoft.Build.Framework.Profiler; + +namespace Microsoft.Build.Framework +{ + /// + /// Arguments for the project evaluation finished event. + /// +#if FEATURE_BINARY_SERIALIZATION + [Serializable] +#endif + public sealed class ProjectEvaluationFinishedEventArgs : BuildStatusEventArgs + { + /// + /// Initializes a new instance of the ProjectEvaluationFinishedEventArgs class. + /// + public ProjectEvaluationFinishedEventArgs() + { + } + + /// + /// Initializes a new instance of the ProjectEvaluationFinishedEventArgs class. + /// + public ProjectEvaluationFinishedEventArgs(string message, params object[] messageArgs) + : base(message, null, null, DateTime.UtcNow, messageArgs) + { + } + + /// + /// Gets or sets the full path of the project that started evaluation. + /// + public string ProjectFile { get; set; } + + /// + /// The result of profiling a project. + /// + /// + /// Null if profiling is not turned on. + /// + public ProfilerResult? ProfilerResult { get; set; } + } +} diff --git a/src/MsBuildPipeLogger.Logger/BinaryLogger/ProjectEvaluationStartedEventArgs.cs b/src/MsBuildPipeLogger.Logger/BinaryLogger/ProjectEvaluationStartedEventArgs.cs new file mode 100644 index 0000000..3ff45c8 --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/BinaryLogger/ProjectEvaluationStartedEventArgs.cs @@ -0,0 +1,36 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; + +namespace Microsoft.Build.Framework +{ + /// + /// Arguments for the project evaluation started event. + /// +#if FEATURE_BINARY_SERIALIZATION + [Serializable] +#endif + public class ProjectEvaluationStartedEventArgs : BuildStatusEventArgs + { + /// + /// Initializes a new instance of the ProjectEvaluationStartedEventArgs class. + /// + public ProjectEvaluationStartedEventArgs() + { + } + + /// + /// Initializes a new instance of the ProjectEvaluationStartedEventArgs class. + /// + public ProjectEvaluationStartedEventArgs(string message, params object[] messageArgs) + : base(message, null, null, DateTime.UtcNow, messageArgs) + { + } + + /// + /// Gets or sets the full path of the project that started evaluation. + /// + public string ProjectFile { get; set; } + } +} \ No newline at end of file diff --git a/src/MsBuildPipeLogger.Logger/BinaryLogger/ProjectImportedEventArgs.cs b/src/MsBuildPipeLogger.Logger/BinaryLogger/ProjectImportedEventArgs.cs new file mode 100644 index 0000000..e5605e2 --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/BinaryLogger/ProjectImportedEventArgs.cs @@ -0,0 +1,55 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; + +namespace Microsoft.Build.Framework +{ + /// + /// Arguments for the project imported event. + /// +#if FEATURE_BINARY_SERIALIZATION + [Serializable] +#endif + public class ProjectImportedEventArgs : BuildMessageEventArgs + { + /// + /// Initializes a new instance of the ProjectImportedEventArgs class. + /// + public ProjectImportedEventArgs() + { + } + + /// + /// Initializes a new instance of the ProjectImportedEventArgs class. + /// + public ProjectImportedEventArgs( + int lineNumber, + int columnNumber, + string message, + params object[] messageArgs) + : base(null, null, null, lineNumber, columnNumber, 0, 0, message, null, null, MessageImportance.Low, DateTime.UtcNow, messageArgs) + { + } + + /// + /// Gets or sets the original value of the Project attribute. + /// + public string UnexpandedProject { get; set; } + + /// + /// Gets or sets the full path to the project file that was imported. Will be. null + /// if the import statement was a glob and no files matched, or the condition (if any) evaluated + /// to false. + /// + public string ImportedProjectFile { get; set; } + + /// + /// Gets or sets if this import was ignored. Ignoring imports is controlled by. + /// ProjectLoadSettings. This is only set when an import would have been included + /// but was ignored to due being invalid. This does not include when a globbed import returned + /// no matches, or a conditioned import that evaluated to false. + /// + public bool ImportIgnored { get; set; } + } +} \ No newline at end of file diff --git a/src/MsBuildPipeLogger.Logger/BinaryLogger/Reflector.cs b/src/MsBuildPipeLogger.Logger/BinaryLogger/Reflector.cs new file mode 100644 index 0000000..c605b17 --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/BinaryLogger/Reflector.cs @@ -0,0 +1,166 @@ +using System; +using System.Reflection; +using Microsoft.Build.Framework; +using MsBuildPipeLogger; + +namespace Microsoft.Build.Logging +{ + /// + /// This class accesses the properties on event args that were added in MSBuild 15.3. + /// As the StructuredLogger.dll references MSBuild 14.0 and gracefully degrades + /// when used with MSBuild 14.0, we have to use Reflection to dynamically + /// retrieve the values if present and gracefully degrade if we're running with + /// an earlier MSBuild. + /// + internal class Reflector + { + private static Func projectFileFromEvaluationStarted; + private static Func projectFileFromEvaluationFinished; + private static Func unexpandedProjectGetter; + private static Func importedProjectFileGetter; + private static Func evaluationIdGetter; + private static Func targetNameFromTargetSkipped; + private static Func targetFileFromTargetSkipped; + private static Func parentTargetFromTargetSkipped; + private static Func buildReasonFromTargetSkipped; + + internal static string GetProjectFileFromEvaluationStarted(BuildEventArgs e) + { + if (projectFileFromEvaluationStarted == null) + { + Type type = e.GetType(); + MethodInfo method = type.GetProperty("ProjectFile").GetGetMethod(); + projectFileFromEvaluationStarted = b => method.Invoke(b, null) as string; + } + + return projectFileFromEvaluationStarted(e); + } + + internal static string GetProjectFileFromEvaluationFinished(BuildEventArgs e) + { + if (projectFileFromEvaluationFinished == null) + { + Type type = e.GetType(); + MethodInfo method = type.GetProperty("ProjectFile").GetGetMethod(); + projectFileFromEvaluationFinished = b => method.Invoke(b, null) as string; + } + + return projectFileFromEvaluationFinished(e); + } + + internal static string GetTargetNameFromTargetSkipped(BuildEventArgs e) + { + if (targetNameFromTargetSkipped == null) + { + Type type = e.GetType(); + MethodInfo method = type.GetProperty("TargetName").GetGetMethod(); + targetNameFromTargetSkipped = b => method.Invoke(b, null) as string; + } + + return targetNameFromTargetSkipped(e); + } + + internal static string GetTargetFileFromTargetSkipped(BuildEventArgs e) + { + if (targetFileFromTargetSkipped == null) + { + Type type = e.GetType(); + MethodInfo method = type.GetProperty("TargetFile").GetGetMethod(); + targetFileFromTargetSkipped = b => method.Invoke(b, null) as string; + } + + return targetFileFromTargetSkipped(e); + } + + internal static string GetParentTargetFromTargetSkipped(BuildEventArgs e) + { + if (parentTargetFromTargetSkipped == null) + { + Type type = e.GetType(); + MethodInfo method = type.GetProperty("ParentTarget").GetGetMethod(); + parentTargetFromTargetSkipped = b => method.Invoke(b, null) as string; + } + + return parentTargetFromTargetSkipped(e); + } + + internal static TargetBuiltReason GetBuildReasonFromTargetStarted(BuildEventArgs e) + { + Type type = e.GetType(); + PropertyInfo property = type.GetProperty("BuildReason"); + if (property == null) + { + return TargetBuiltReason.None; + } + + MethodInfo method = property.GetGetMethod(); + return (TargetBuiltReason)method.Invoke(e, null); + } + + internal static TargetBuiltReason GetBuildReasonFromTargetSkipped(BuildEventArgs e) + { + if (buildReasonFromTargetSkipped == null) + { + Type type = e.GetType(); + PropertyInfo property = type.GetProperty("BuildReason"); + if (property == null) + { + return TargetBuiltReason.None; + } + + MethodInfo method = property.GetGetMethod(); + buildReasonFromTargetSkipped = b => (TargetBuiltReason)method.Invoke(b, null); + } + + return buildReasonFromTargetSkipped(e); + } + + internal static string GetUnexpandedProject(BuildEventArgs e) + { + if (unexpandedProjectGetter == null) + { + Type type = e.GetType(); + MethodInfo method = type.GetProperty("UnexpandedProject").GetGetMethod(); + unexpandedProjectGetter = b => method.Invoke(b, null) as string; + } + + return unexpandedProjectGetter(e); + } + + internal static string GetImportedProjectFile(BuildEventArgs e) + { + if (importedProjectFileGetter == null) + { + Type type = e.GetType(); + MethodInfo method = type.GetProperty("ImportedProjectFile").GetGetMethod(); + importedProjectFileGetter = b => method.Invoke(b, null) as string; + } + + return importedProjectFileGetter(e); + } + + internal static int GetEvaluationId(BuildEventContext buildEventContext) + { + if (buildEventContext == null) + { + return -1; + } + + if (evaluationIdGetter == null) + { + Type type = buildEventContext.GetType(); + FieldInfo field = type.GetField("_evaluationId"/*, BindingFlags.Instance | BindingFlags.NonPublic*/); + if (field != null) + { + evaluationIdGetter = b => (int)field.GetValue(b); + } + else + { + evaluationIdGetter = b => b.ProjectContextId <= 0 ? -b.ProjectContextId : -1; + } + } + + return evaluationIdGetter(buildEventContext); + } + } +} diff --git a/src/MsBuildPipeLogger.Logger/BinaryLogger/TargetBuiltReason.cs b/src/MsBuildPipeLogger.Logger/BinaryLogger/TargetBuiltReason.cs new file mode 100644 index 0000000..d318a51 --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/BinaryLogger/TargetBuiltReason.cs @@ -0,0 +1,31 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +namespace Microsoft.Build.Framework +{ + /// + /// The reason that a target was built by its parent target. + /// + public enum TargetBuiltReason + { + /// + /// This wasn't built on because of a parent. + /// + None, + + /// + /// The target was part of the parent's BeforeTargets list. + /// + BeforeTargets, + + /// + /// The target was part of the parent's DependsOn list. + /// + DependsOn, + + /// + /// The target was part of the parent's AfterTargets list. + /// + AfterTargets + } +} diff --git a/src/MsBuildPipeLogger.Logger/BinaryLogger/TargetSkippedEventArgs.cs b/src/MsBuildPipeLogger.Logger/BinaryLogger/TargetSkippedEventArgs.cs new file mode 100644 index 0000000..d53f21f --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/BinaryLogger/TargetSkippedEventArgs.cs @@ -0,0 +1,53 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; + +namespace Microsoft.Build.Framework +{ + /// + /// Arguments for the target skipped event. + /// +#if FEATURE_BINARY_SERIALIZATION + [Serializable] +#endif + public class TargetSkippedEventArgs : BuildMessageEventArgs + { + /// + /// Initializes a new instance of the TargetSkippedEventArgs class. + /// + public TargetSkippedEventArgs() + { + } + + /// + /// Initializes a new instance of the TargetSkippedEventArgs class. + /// + public TargetSkippedEventArgs( + string message, + params object[] messageArgs) + : base(null, null, null, 0, 0, 0, 0, message, null, null, MessageImportance.Low, DateTime.UtcNow, messageArgs) + { + } + + /// + /// Gets or sets the name of the target being skipped. + /// + public string TargetName { get; set; } + + /// + /// Gets or sets the parent target of the target being skipped. + /// + public string ParentTarget { get; set; } + + /// + /// File where this target was declared. + /// + public string TargetFile { get; set; } + + /// + /// Why the parent target built this target. + /// + public TargetBuiltReason BuildReason { get; set; } + } +} diff --git a/src/MsBuildPipeLogger.Logger/IPipeWriter.cs b/src/MsBuildPipeLogger.Logger/IPipeWriter.cs new file mode 100644 index 0000000..6438266 --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/IPipeWriter.cs @@ -0,0 +1,10 @@ +using System; +using Microsoft.Build.Framework; + +namespace MsBuildPipeLogger +{ + public interface IPipeWriter : IDisposable + { + void Write(BuildEventArgs e); + } +} diff --git a/src/MsBuildPipeLogger.Logger/MsBuildPipeLogger.Logger.csproj b/src/MsBuildPipeLogger.Logger/MsBuildPipeLogger.Logger.csproj new file mode 100644 index 0000000..5187fb1 --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/MsBuildPipeLogger.Logger.csproj @@ -0,0 +1,29 @@ + + + netstandard2.0 + MsBuildPipeLogger + A logger for MSBuild that sends event data over anonymous or named pipes. + false + + + + compile; build; native; contentfiles; analyzers; buildtransitive + + + compile; build; native; contentfiles; analyzers; buildtransitive + + + + + + + <_Parameter1>MsBuildPipeLogger.Logger.Tests + + + <_Parameter1>MsBuildPipeLogger.Tests + + + <_Parameter1>MsBuildPipeLogger.Tests.Client + + + \ No newline at end of file diff --git a/src/MsBuildPipeLogger.Logger/NamedPipeWriter.cs b/src/MsBuildPipeLogger.Logger/NamedPipeWriter.cs new file mode 100644 index 0000000..fa14acd --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/NamedPipeWriter.cs @@ -0,0 +1,30 @@ +using System.IO.Pipes; + +namespace MsBuildPipeLogger +{ + public class NamedPipeWriter : PipeWriter + { + public string ServerName { get; } + + public string PipeName { get; } + + public NamedPipeWriter(string pipeName) + : this(".", pipeName) + { + } + + public NamedPipeWriter(string serverName, string pipeName) + : base(InitializePipe(serverName, pipeName)) + { + ServerName = serverName; + PipeName = pipeName; + } + + private static PipeStream InitializePipe(string serverName, string pipeName) + { + NamedPipeClientStream pipeStream = new NamedPipeClientStream(serverName, pipeName, PipeDirection.Out); + pipeStream.Connect(); + return pipeStream; + } + } +} diff --git a/src/MsBuildPipeLogger.Logger/ParameterParser.cs b/src/MsBuildPipeLogger.Logger/ParameterParser.cs new file mode 100644 index 0000000..06db0a6 --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/ParameterParser.cs @@ -0,0 +1,84 @@ +using System; +using System.Collections.Generic; +using System.Linq; +using Microsoft.Build.Framework; + +namespace MsBuildPipeLogger +{ + internal static class ParameterParser + { + internal enum ParameterType + { + Handle, + Name, + Server + } + + public static IPipeWriter GetPipeFromParameters(string parameters) + { + KeyValuePair[] segments = ParseParameters(parameters); + + if (segments.Any(x => string.IsNullOrWhiteSpace(x.Value))) + { + throw new LoggerException($"Invalid or empty parameter value"); + } + + // Anonymous pipe + if (segments[0].Key == ParameterType.Handle) + { + if (segments.Length > 1) + { + throw new LoggerException("Handle can only be specified as a single parameter"); + } + return new AnonymousPipeWriter(segments[0].Value); + } + + // Named pipe + if (segments[0].Key == ParameterType.Name) + { + if (segments.Length == 1) + { + return new NamedPipeWriter(segments[0].Value); + } + if (segments[1].Key != ParameterType.Server) + { + throw new LoggerException("Only server and name can be specified for a named pipe"); + } + return new NamedPipeWriter(segments[1].Value, segments[0].Value); + } + if (segments.Length == 1 || segments[1].Key != ParameterType.Name) + { + throw new LoggerException("Pipe name must be specified for a named pipe"); + } + return new NamedPipeWriter(segments[0].Value, segments[1].Value); + } + + internal static KeyValuePair[] ParseParameters(string parameters) + { + string[] segments = parameters.Split(';'); + if (segments.Length < 1 || segments.Length > 2) + { + throw new LoggerException("Unexpected number of parameters"); + } + return segments.Select(x => ParseParameter(x)).ToArray(); + } + + private static KeyValuePair ParseParameter(string parameter) + { + string[] parts = parameter.Trim().Trim('"').Split('='); + + // No parameter name specified + if (parts.Length == 1) + { + return new KeyValuePair(ParameterType.Handle, parts[0].Trim()); + } + + // Parse the parameter name + if (!Enum.TryParse(parts[0].Trim(), true, out ParameterType parameterType)) + { + throw new LoggerException($"Invalid parameter name {parts[0]}"); + } + return new KeyValuePair(parameterType, string.Join("=", parts.Skip(1)).Trim()); + } + } +} diff --git a/src/MsBuildPipeLogger.Logger/PipeLogger.cs b/src/MsBuildPipeLogger.Logger/PipeLogger.cs new file mode 100644 index 0000000..3e6fbb1 --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/PipeLogger.cs @@ -0,0 +1,47 @@ +using System; +using Microsoft.Build.Framework; +using Microsoft.Build.Utilities; + +namespace MsBuildPipeLogger +{ + /// + /// Logger to send messages from the MSBuild logging system over an anonymous pipe. + /// + /// + /// Heavily based on the work of Kirill Osenkov and the MSBuildStructuredLog project. + /// + public class PipeLogger : Logger + { + protected IPipeWriter Pipe { get; private set; } + + public override void Initialize(IEventSource eventSource) + { + InitializeEnvironmentVariables(); + Pipe = InitializePipeWriter(); + InitializeEvents(eventSource); + } + + protected virtual void InitializeEnvironmentVariables() + { + Environment.SetEnvironmentVariable("MSBUILDTARGETOUTPUTLOGGING", "true"); + Environment.SetEnvironmentVariable("MSBUILDLOGIMPORTS", "1"); + } + + protected virtual IPipeWriter InitializePipeWriter() => ParameterParser.GetPipeFromParameters(Parameters); + + protected virtual void InitializeEvents(IEventSource eventSource) + { + eventSource.AnyEventRaised += (_, e) => Pipe.Write(e); + } + + public override void Shutdown() + { + base.Shutdown(); + if (Pipe != null) + { + Pipe.Dispose(); + Pipe = null; + } + } + } +} diff --git a/src/MsBuildPipeLogger.Logger/PipeWriter.cs b/src/MsBuildPipeLogger.Logger/PipeWriter.cs new file mode 100644 index 0000000..9794c9c --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/PipeWriter.cs @@ -0,0 +1,99 @@ +using System; +using System.Collections.Concurrent; +using System.IO; +using System.IO.Pipes; +using System.Threading; +using Microsoft.Build.Framework; +using Microsoft.Build.Logging; + +namespace MsBuildPipeLogger +{ + public abstract class PipeWriter : IPipeWriter + { + private readonly BlockingCollection _queue = + new BlockingCollection(new ConcurrentQueue()); + + private readonly AutoResetEvent _doneProcessing = new AutoResetEvent(false); + + private readonly PipeStream _pipeStream; + private readonly BinaryWriter _binaryWriter; + private readonly BuildEventArgsWriter _argsWriter; + + // Buffer writes through a memory stream since the args writer does a bunch of small writes + private readonly MemoryStream _memoryStream = new MemoryStream(); + + protected PipeWriter(PipeStream pipeStream) + { + _pipeStream = pipeStream ?? throw new ArgumentNullException(nameof(pipeStream)); + _binaryWriter = new BinaryWriter(_memoryStream); + _argsWriter = new BuildEventArgsWriter(_binaryWriter); + + Thread writerThread = new Thread(() => + { + BuildEventArgs eventArgs; + while ((eventArgs = TakeEventArgs()) != null) + { + // Reset the memory stream (but reuse the memory) + _memoryStream.Seek(0, SeekOrigin.Begin); + _memoryStream.SetLength(0); + + // Buffer to the memory stream + _argsWriter.Write(eventArgs); + _binaryWriter.Flush(); + + // ...then write that to the pipe + _memoryStream.WriteTo(_pipeStream); + _pipeStream.Flush(); + } + _doneProcessing.Set(); + }) + { + IsBackground = true, + }; + writerThread.Start(); + } + + private BuildEventArgs TakeEventArgs() + { + if (!_queue.IsCompleted) + { + try + { + return _queue.Take(); + } + catch (InvalidOperationException) + { + } + } + return null; + } + + public void Dispose() + { + if (!_queue.IsAddingCompleted) + { + try + { + _queue.CompleteAdding(); + _doneProcessing.WaitOne(); + if (IsWindows) + { + _pipeStream.WaitForPipeDrain(); + } + _pipeStream.Dispose(); + } + catch + { + } + } + } + + public void Write(BuildEventArgs e) => _queue.Add(e); + + private static readonly bool IsWindows = Environment.OSVersion.Platform == PlatformID.Win32NT || + Environment.OSVersion.Platform == PlatformID.Win32S || + Environment.OSVersion.Platform == PlatformID.Win32Windows || + Environment.OSVersion.Platform == PlatformID.WinCE || + Environment.OSVersion.Platform == PlatformID.Xbox; + } +} diff --git a/src/MsBuildPipeLogger.Logger/TypeExtensions.cs b/src/MsBuildPipeLogger.Logger/TypeExtensions.cs new file mode 100644 index 0000000..27544da --- /dev/null +++ b/src/MsBuildPipeLogger.Logger/TypeExtensions.cs @@ -0,0 +1,16 @@ +using System; +using System.Collections.Generic; +using System.Reflection; +using System.Text; + +namespace MsBuildPipeLogger +{ + internal static class TypeExtensions + { + public static PropertyInfo GetProperty(this Type type, string name) => type.GetTypeInfo().GetDeclaredProperty(name); + + public static MethodInfo GetGetMethod(this PropertyInfo property) => property.GetMethod; + + public static FieldInfo GetField(this Type type, string name) => type.GetTypeInfo().GetDeclaredField(name); + } +} diff --git a/src/MsBuildPipeLogger.Server/AnonymousPipeLoggerServer.cs b/src/MsBuildPipeLogger.Server/AnonymousPipeLoggerServer.cs new file mode 100644 index 0000000..a0ae7f7 --- /dev/null +++ b/src/MsBuildPipeLogger.Server/AnonymousPipeLoggerServer.cs @@ -0,0 +1,55 @@ +using System; +using System.IO; +using System.IO.Pipes; +using System.Threading; +using System.Threading.Tasks; + +namespace MsBuildPipeLogger +{ + /// + /// A server for receiving MSBuild logging events over an anonymous pipe. + /// + public class AnonymousPipeLoggerServer : PipeLoggerServer + { + private string _clientHandle; + + /// + /// Creates an anonymous pipe server for receiving MSBuild logging events. + /// + public AnonymousPipeLoggerServer() + : this(CancellationToken.None) + { + } + + /// + /// Creates an anonymous pipe server for receiving MSBuild logging events. + /// + /// A that will cancel read operations if triggered. + public AnonymousPipeLoggerServer(CancellationToken cancellationToken) + : base(new AnonymousPipeServerStream(PipeDirection.In, HandleInheritability.Inheritable), cancellationToken) + { + } + + /// + /// Gets the client handle as a string. The local copy of the handle will be automatically disposed on the first call to Read. + /// + /// The client handle as a string. + public string GetClientHandle() => _clientHandle ?? (_clientHandle = PipeStream.GetClientHandleAsString()); + + protected override void Connect() + { + // Wait for the first write, there's a chicken-and-egg problem with the pipe handle + // I can only dispose the local handle after the first pipe read, which blocks + // But I can only catch the pipe disposal from cancellation after the handle has been disposed + Buffer.FillFromStream(PipeStream, CancellationToken); + + // Dispose the client handle if we asked for one + // If we don't do this we won't get notified when the stream closes, see https://stackoverflow.com/q/39682602/807064 + if (_clientHandle != null) + { + PipeStream.DisposeLocalCopyOfClientHandle(); + _clientHandle = null; + } + } + } +} diff --git a/src/MsBuildPipeLogger.Server/BuildEventArgsReaderProxy.cs b/src/MsBuildPipeLogger.Server/BuildEventArgsReaderProxy.cs new file mode 100644 index 0000000..b91c5dc --- /dev/null +++ b/src/MsBuildPipeLogger.Server/BuildEventArgsReaderProxy.cs @@ -0,0 +1,34 @@ +using System; +using System.IO; +using System.Reflection; +using Microsoft.Build.Framework; +using Microsoft.Build.Logging; + +namespace MsBuildPipeLogger +{ + internal class BuildEventArgsReaderProxy + { + private readonly Func _read; + + public BuildEventArgsReaderProxy(BinaryReader reader) + { + // Use reflection to get the Microsoft.Build.Logging.BuildEventArgsReader.Read() method + object argsReader; + Type buildEventArgsReader = typeof(BinaryLogger).GetTypeInfo().Assembly.GetType("Microsoft.Build.Logging.BuildEventArgsReader"); + ConstructorInfo readerCtor = buildEventArgsReader.GetConstructor(new[] { typeof(BinaryReader) }); + if (readerCtor != null) + { + argsReader = readerCtor.Invoke(new[] { reader }); + } + else + { + readerCtor = buildEventArgsReader.GetConstructor(new[] { typeof(BinaryReader), typeof(int) }); + argsReader = readerCtor.Invoke(new object[] { reader, 7 }); + } + MethodInfo readMethod = buildEventArgsReader.GetMethod("Read"); + _read = (Func)readMethod.CreateDelegate(typeof(Func), argsReader); + } + + public BuildEventArgs Read() => _read(); + } +} diff --git a/src/MsBuildPipeLogger.Server/CancellationTokenExtensions.cs b/src/MsBuildPipeLogger.Server/CancellationTokenExtensions.cs new file mode 100644 index 0000000..71b4317 --- /dev/null +++ b/src/MsBuildPipeLogger.Server/CancellationTokenExtensions.cs @@ -0,0 +1,41 @@ +using System; +using System.Threading; +using System.Threading.Tasks; + +namespace MsBuildPipeLogger +{ + internal static class CancellationTokenExtensions + { + public static TResult Try(this CancellationToken cancellationToken, Func action, Func cancelled) + { + if (cancellationToken.IsCancellationRequested) + { + return cancelled == null ? default(TResult) : cancelled(); + } + try + { + return action(); + } + catch (TaskCanceledException) + { + // Thrown if the task itself was canceled from inside the read method + return cancelled == null ? default(TResult) : cancelled(); + } + catch (OperationCanceledException) + { + // Thrown if the operation was canceled (I.e., the task didn't deal with cancellation) + return cancelled == null ? default(TResult) : cancelled(); + } + catch (AggregateException ex) + { + // Sometimes the cancellation exceptions are thrown in aggregate + if (!(ex.InnerException is TaskCanceledException) + && !(ex.InnerException is OperationCanceledException)) + { + throw; + } + return cancelled == null ? default(TResult) : cancelled(); + } + } + } +} diff --git a/src/MsBuildPipeLogger.Server/IPipeLoggerServer.cs b/src/MsBuildPipeLogger.Server/IPipeLoggerServer.cs new file mode 100644 index 0000000..ee24dff --- /dev/null +++ b/src/MsBuildPipeLogger.Server/IPipeLoggerServer.cs @@ -0,0 +1,20 @@ +using System; +using Microsoft.Build.Framework; + +namespace MsBuildPipeLogger +{ + public interface IPipeLoggerServer : IDisposable + { + /// + /// Reads a single event from the pipe. This method blocks until an event is received, + /// there are no more events, or the pipe is closed. + /// + /// The read event or null if there are no more events or the pipe is closed. + BuildEventArgs Read(); + + /// + /// Reads all events from the pipe and blocks until there are no more events or the pipe is closed. + /// + void ReadAll(); + } +} diff --git a/src/MsBuildPipeLogger.Server/InterlockedBool.cs b/src/MsBuildPipeLogger.Server/InterlockedBool.cs new file mode 100644 index 0000000..d12dd5f --- /dev/null +++ b/src/MsBuildPipeLogger.Server/InterlockedBool.cs @@ -0,0 +1,44 @@ +using System; +using System.IO.Pipes; +using System.Threading; +using System.Threading.Tasks; + +namespace MsBuildPipeLogger +{ + internal class InterlockedBool + { + private volatile int _set; + + public InterlockedBool() + { + _set = 0; + } + + public InterlockedBool(bool initialState) + { + _set = initialState ? 1 : 0; + } + + // Returns the previous switch state of the switch + public bool Set() + { +#pragma warning disable 420 + return Interlocked.Exchange(ref _set, 1) != 0; +#pragma warning restore 420 + } + + // Returns the previous switch state of the switch + public bool Unset() + { +#pragma warning disable 420 + return Interlocked.Exchange(ref _set, 0) != 0; +#pragma warning restore 420 + } + + // Returns the current state + public static implicit operator bool(InterlockedBool interlockedBool) + { + return interlockedBool._set != 0; + } + } +} diff --git a/src/MsBuildPipeLogger.Server/MsBuildPipeLogger.Server.csproj b/src/MsBuildPipeLogger.Server/MsBuildPipeLogger.Server.csproj new file mode 100644 index 0000000..8fad8ea --- /dev/null +++ b/src/MsBuildPipeLogger.Server/MsBuildPipeLogger.Server.csproj @@ -0,0 +1,16 @@ + + + netstandard2.0 + MsBuildPipeLogger + A logger for MSBuild that sends event data over anonymous or named pipes. + false + + + + + + + <_Parameter1>MsBuildPipeLogger.Tests + + + \ No newline at end of file diff --git a/src/MsBuildPipeLogger.Server/NamedPipeLoggerServer.cs b/src/MsBuildPipeLogger.Server/NamedPipeLoggerServer.cs new file mode 100644 index 0000000..3a1160a --- /dev/null +++ b/src/MsBuildPipeLogger.Server/NamedPipeLoggerServer.cs @@ -0,0 +1,58 @@ +using System; +using System.IO.Pipes; +using System.Threading; +using System.Threading.Tasks; + +namespace MsBuildPipeLogger +{ + /// + /// A server for receiving MSBuild logging events over a named pipe. + /// + public class NamedPipeLoggerServer : PipeLoggerServer + { + private readonly InterlockedBool _connected = new InterlockedBool(false); + + public string PipeName { get; } + + /// + /// Creates a named pipe server for receiving MSBuild logging events. + /// + /// The name of the pipe to create. + public NamedPipeLoggerServer(string pipeName) + : this(pipeName, CancellationToken.None) + { + } + + /// + /// Creates a named pipe server for receiving MSBuild logging events. + /// + /// The name of the pipe to create. + /// A that will cancel read operations if triggered. + public NamedPipeLoggerServer(string pipeName, CancellationToken cancellationToken) + : base(new NamedPipeServerStream(pipeName, PipeDirection.In), cancellationToken) + { + PipeName = pipeName; + CancellationToken.Register(CancelConnectionWait); + } + + protected override void Connect() + { + PipeStream.WaitForConnection(); + _connected.Set(); + } + + private void CancelConnectionWait() + { + if (!_connected.Set()) + { + // This is a crazy hack that stops the WaitForConnection by connecting a dummy client + // We have to do it this way instead of checking for .IsConnected because if we connect + // and then disconnect very quickly, .IsConnected will never observe as true and we'll lock + using (NamedPipeClientStream pipeStream = new NamedPipeClientStream(".", PipeName, PipeDirection.Out)) + { + pipeStream.Connect(); + } + } + } + } +} diff --git a/src/MsBuildPipeLogger.Server/PipeBuffer.cs b/src/MsBuildPipeLogger.Server/PipeBuffer.cs new file mode 100644 index 0000000..e1dfa1f --- /dev/null +++ b/src/MsBuildPipeLogger.Server/PipeBuffer.cs @@ -0,0 +1,181 @@ +using System; +using System.Collections.Concurrent; +using System.IO; +using System.IO.Pipes; +using System.Threading; +using System.Threading.Tasks; + +namespace MsBuildPipeLogger +{ + internal class PipeBuffer : Stream + { + private const int BufferSize = 8192; + + private readonly ConcurrentBag _pool = new ConcurrentBag(); + + private readonly BlockingCollection _queue = + new BlockingCollection(new ConcurrentQueue()); + + private Buffer _current; + + public void CompleteAdding() => _queue.CompleteAdding(); + + public bool IsCompleted => _queue.IsCompleted; + + public bool FillFromStream(Stream stream, CancellationToken cancellationToken) + { + if (!_pool.TryTake(out Buffer buffer)) + { + buffer = new Buffer(); + } + if (buffer.FillFromStream(stream, cancellationToken) == 0) + { + // Didn't write anything, return it to the pool + _pool.Add(buffer); + return false; + } + _queue.Add(buffer); + return true; + } + + public override void Write(byte[] buffer, int offset, int count) => + _queue.Add(new Buffer(buffer, offset, count)); + + public override int Read(byte[] buffer, int offset, int count) + { + int read = 0; + while (read < count) + { + // Ensure a buffer is available + if (TakeBuffer()) + { + // Get as much as we can from the current buffer + read += _current.Read(buffer, offset + read, count - read); + if (_current.Count == 0) + { + // Used up this buffer, return to the pool if it's a pool buffer + if (_current.FromPool) + { + _pool.Add(_current); + } + _current = null; + } + } + else + { + break; + } + } + return read; + } + + private bool TakeBuffer() + { + if (_current == null) + { + // Take() can throw when marked as complete from another thread + // https://docs.microsoft.com/en-us/dotnet/api/system.collections.concurrent.blockingcollection-1.take?view=netcore-3.1 + try + { + _current = _queue.Take(); + } + catch (ObjectDisposedException) + { + return false; + } + catch (OperationCanceledException) + { + return false; + } + catch (InvalidOperationException) + { + return false; + } + } + return true; + } + + private class Buffer + { + private readonly byte[] _buffer; + + private int _offset; + + public int Count { get; private set; } + + public bool FromPool { get; } + + public Buffer() + { + _buffer = new byte[BufferSize]; + FromPool = true; + } + + public Buffer(byte[] buffer, int offset, int count) + { + _buffer = buffer; + _offset = offset; + Count = count; + } + + public int FillFromStream(Stream stream, CancellationToken cancellationToken) + { + if (stream is AnonymousPipeServerStream || stream is AnonymousPipeClientStream) + { + // We can't use ReadAsync with Anonymous PipeStream + // https://github.com/dotnet/runtime/issues/23638 + // https://docs.microsoft.com/en-us/windows/win32/ipc/anonymous-pipe-operations + // Asynchronous (overlapped) read and write operations are not supported by anonymous pipes + _offset = 0; + Count = cancellationToken.IsCancellationRequested ? 0 : stream.Read(_buffer, _offset, BufferSize); + } + else + { + Count = cancellationToken.Try( + () => + { + _offset = 0; + Task readTask = stream.ReadAsync(_buffer, _offset, BufferSize, cancellationToken); +#pragma warning disable VSTHRD002 // Synchronously waiting on tasks or awaiters may cause deadlocks. Use await or JoinableTaskFactory.Run instead. + readTask.Wait(cancellationToken); + return readTask.Status == TaskStatus.Canceled ? 0 : readTask.Result; +#pragma warning restore VSTHRD002 + }, + () => 0); + } + return Count; + } + + public int Read(byte[] buffer, int offset, int count) + { + int available = count > Count ? Count : count; + Array.Copy(_buffer, _offset, buffer, offset, available); + _offset += available; + Count -= available; + return available; + } + } + + // Not implemented + + public override bool CanRead => true; + + public override bool CanSeek => false; + + public override bool CanWrite => false; + + public override long Length => throw new NotImplementedException(); + + public override long Position + { + get => throw new NotImplementedException(); + set => throw new NotImplementedException(); + } + + public override void Flush() => throw new NotImplementedException(); + + public override long Seek(long offset, SeekOrigin origin) => throw new NotImplementedException(); + + public override void SetLength(long value) => throw new NotImplementedException(); + } +} \ No newline at end of file diff --git a/src/MsBuildPipeLogger.Server/PipeLoggerServer.cs b/src/MsBuildPipeLogger.Server/PipeLoggerServer.cs new file mode 100644 index 0000000..730196a --- /dev/null +++ b/src/MsBuildPipeLogger.Server/PipeLoggerServer.cs @@ -0,0 +1,128 @@ +using System; +using System.Collections.Generic; +using System.IO; +using System.IO.Pipes; +using System.Text; +using System.Threading; +using Microsoft.Build.Framework; +using Microsoft.Build.Logging; + +namespace MsBuildPipeLogger +{ + /// + /// Receives MSBuild logging events over a pipe. This is the base class for + /// and . + /// + public abstract class PipeLoggerServer : EventArgsDispatcher, IPipeLoggerServer + where TPipeStream : PipeStream + { + private readonly BinaryReader _binaryReader; + private readonly BuildEventArgsReaderProxy _buildEventArgsReader; + + internal PipeBuffer Buffer { get; } = new PipeBuffer(); + + protected TPipeStream PipeStream { get; } + + protected CancellationToken CancellationToken { get; } + + /// + /// Creates a server that receives MSBuild events over a specified pipe. + /// + /// The pipe to receive events from. + protected PipeLoggerServer(TPipeStream pipeStream) + : this(pipeStream, CancellationToken.None) + { + } + + /// + /// Creates a server that receives MSBuild events over a specified pipe. + /// + /// The pipe to receive events from. + /// A that will cancel read operations if triggered. + protected PipeLoggerServer(TPipeStream pipeStream, CancellationToken cancellationToken) + { + PipeStream = pipeStream; + _binaryReader = new BinaryReader(Buffer); + _buildEventArgsReader = new BuildEventArgsReaderProxy(_binaryReader); + CancellationToken = cancellationToken; + + Thread readerThread = new Thread(() => + { + try + { + Connect(); + while (Buffer.FillFromStream(PipeStream, CancellationToken)) + { + } + } + catch (IOException) + { + // The client broke the stream so we're done + } + catch (ObjectDisposedException) + { + // The pipe was disposed + } + + // Add a final 0 (BinaryLogRecordKind.EndOfFile) into the stream in case the BuildEventArgsReader is waiting for a read + Buffer.Write(new byte[1] { 0 }, 0, 1); + + Buffer.CompleteAdding(); + }) + { + IsBackground = true + }; + + readerThread.Start(); + } + + protected abstract void Connect(); + + /// + public BuildEventArgs Read() + { + if (Buffer.IsCompleted) + { + return null; + } + + try + { + BuildEventArgs args = _buildEventArgsReader.Read(); + if (args != null) + { + Dispatch(args); + return args; + } + } + catch (EndOfStreamException) + { + // The stream may have been closed or otherwise stopped + } + + return null; + } + + /// + public void ReadAll() + { + BuildEventArgs args = Read(); + while (args != null) + { + if (args is BuildFinishedEventArgs) + { + return; + } + args = Read(); + } + } + + /// + public void Dispose() + { + _binaryReader.Dispose(); + Buffer.Dispose(); + PipeStream.Dispose(); + } + } +} diff --git a/src/dotnet-releaser.sln b/src/dotnet-releaser.sln index 791e3f6..b05671d 100644 --- a/src/dotnet-releaser.sln +++ b/src/dotnet-releaser.sln @@ -18,6 +18,16 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution EndProject Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "DotNetReleaser.Tests", "DotNetReleaser.Tests\DotNetReleaser.Tests.csproj", "{B34D6239-0F5A-429D-B710-D571CE6BE58B}" EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Documentation", "Documentation", "{258DCAB7-E550-499B-8682-21FF31324B7E}" + ProjectSection(SolutionItems) = preProject + ..\doc\changelog_user_guide.md = ..\doc\changelog_user_guide.md + ..\doc\readme.md = ..\doc\readme.md + EndProjectSection +EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "MsBuildPipeLogger.Logger", "MsBuildPipeLogger.Logger\MsBuildPipeLogger.Logger.csproj", "{6066CF00-1AF0-E8BD-6D0C-A3BCF6E934DE}" +EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "MsBuildPipeLogger.Server", "MsBuildPipeLogger.Server\MsBuildPipeLogger.Server.csproj", "{918CA12A-5C20-6ED8-FC47-21D580CD1E46}" +EndProject Global GlobalSection(SolutionConfigurationPlatforms) = preSolution Debug|Any CPU = Debug|Any CPU @@ -32,6 +42,14 @@ Global {B34D6239-0F5A-429D-B710-D571CE6BE58B}.Debug|Any CPU.Build.0 = Debug|Any CPU {B34D6239-0F5A-429D-B710-D571CE6BE58B}.Release|Any CPU.ActiveCfg = Release|Any CPU {B34D6239-0F5A-429D-B710-D571CE6BE58B}.Release|Any CPU.Build.0 = Release|Any CPU + {6066CF00-1AF0-E8BD-6D0C-A3BCF6E934DE}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {6066CF00-1AF0-E8BD-6D0C-A3BCF6E934DE}.Debug|Any CPU.Build.0 = Debug|Any CPU + {6066CF00-1AF0-E8BD-6D0C-A3BCF6E934DE}.Release|Any CPU.ActiveCfg = Release|Any CPU + {6066CF00-1AF0-E8BD-6D0C-A3BCF6E934DE}.Release|Any CPU.Build.0 = Release|Any CPU + {918CA12A-5C20-6ED8-FC47-21D580CD1E46}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {918CA12A-5C20-6ED8-FC47-21D580CD1E46}.Debug|Any CPU.Build.0 = Debug|Any CPU + {918CA12A-5C20-6ED8-FC47-21D580CD1E46}.Release|Any CPU.ActiveCfg = Release|Any CPU + {918CA12A-5C20-6ED8-FC47-21D580CD1E46}.Release|Any CPU.Build.0 = Release|Any CPU EndGlobalSection GlobalSection(SolutionProperties) = preSolution HideSolutionNode = FALSE diff --git a/src/dotnet-releaser/Configuration/ChangelogConfiguration.cs b/src/dotnet-releaser/Configuration/ChangelogConfiguration.cs index 1b35e7c..0906b63 100644 --- a/src/dotnet-releaser/Configuration/ChangelogConfiguration.cs +++ b/src/dotnet-releaser/Configuration/ChangelogConfiguration.cs @@ -19,7 +19,7 @@ public ChangelogConfiguration() //ChangeTitleEscape = @"\<*_&@"; Owners = new List(); Autolabelers = new List(); - Replacers = new List(); + Replacers = new List(); Include = new ChangelogFilter(); Exclude = new ChangelogFilter(); BodyTemplate = @"# Changes @@ -64,7 +64,7 @@ public ChangelogConfiguration() public List Autolabelers { get; } [DataMember(Name = "replacer")] - public List Replacers { get; } // TBD: not implemented yet + public List Replacers { get; } // TBD: not implemented yet [DataMember(Name = "category")] public List Categories { get; } diff --git a/src/dotnet-releaser/Configuration/ChangelogReplacer.cs b/src/dotnet-releaser/Configuration/ChangelogReplacer.cs deleted file mode 100644 index 0527728..0000000 --- a/src/dotnet-releaser/Configuration/ChangelogReplacer.cs +++ /dev/null @@ -1,14 +0,0 @@ -namespace DotNetReleaser.Configuration; - -public class ChangelogReplacer -{ - public ChangelogReplacer() - { - Search = string.Empty; - Replace = string.Empty; - } - - public string Search { get; set; } - - public string Replace { get; set; } -} \ No newline at end of file diff --git a/src/dotnet-releaser/Configuration/PackagingConfiguration.cs b/src/dotnet-releaser/Configuration/PackagingConfiguration.cs index 0029b86..b350c78 100644 --- a/src/dotnet-releaser/Configuration/PackagingConfiguration.cs +++ b/src/dotnet-releaser/Configuration/PackagingConfiguration.cs @@ -1,5 +1,6 @@ using System.Collections.Generic; using System.Runtime.Serialization; +using System.Text.RegularExpressions; namespace DotNetReleaser.Configuration; @@ -9,6 +10,7 @@ public PackagingConfiguration() { RuntimeIdentifiers = new List(); Kinds = new List(); + Renamers = new List(); } [DataMember(Name = "rid")] @@ -16,6 +18,9 @@ public PackagingConfiguration() [DataMember(Name = "kinds")] public List Kinds { get; } + + [DataMember(Name = "renamer")] + public List Renamers { get; } public static string ToStringRidAndKinds(List rids, List kinds) => $"platform{(rids.Count > 1 ? "s" : string.Empty)} [{string.Join(", ", rids)}] with [{string.Join(", ", kinds)}] package{(kinds.Count > 1 ? "s" : string.Empty)}"; diff --git a/src/dotnet-releaser/Configuration/RegexReplacer.cs b/src/dotnet-releaser/Configuration/RegexReplacer.cs new file mode 100644 index 0000000..2cc750a --- /dev/null +++ b/src/dotnet-releaser/Configuration/RegexReplacer.cs @@ -0,0 +1,21 @@ +using System.Runtime.Serialization; +using System.Text.RegularExpressions; + +namespace DotNetReleaser.Configuration; + +public class RegexReplacer +{ + public RegexReplacer() + { + Pattern = string.Empty; + Replace = string.Empty; + } + + [DataMember(Name = "pattern")] + public string Pattern { get; set; } + + [DataMember(Name = "replace")] + public string Replace { get; set; } + + internal string Run(string input) => Regex.Replace(input, Pattern, Replace); +} \ No newline at end of file diff --git a/src/dotnet-releaser/DevHosting/GitHubDevHosting.cs b/src/dotnet-releaser/DevHosting/GitHubDevHosting.cs index 5e08ae3..45c0446 100644 --- a/src/dotnet-releaser/DevHosting/GitHubDevHosting.cs +++ b/src/dotnet-releaser/DevHosting/GitHubDevHosting.cs @@ -10,6 +10,7 @@ using DotNetReleaser.Changelog; using DotNetReleaser.Configuration; using DotNetReleaser.Logging; +using DotNetReleaser.Runners; using NuGet.Versioning; using Octokit; @@ -20,17 +21,21 @@ public class GitHubDevHosting : IDevHosting private readonly ISimpleLogger _log; private readonly string _url; private readonly string _apiToken; + private readonly string _gistApiToken; private readonly GitHubClient _client; - - public GitHubDevHosting(ISimpleLogger log, DevHostingConfiguration hostingConfiguration, string apiToken, string apiTokenUsage) + private readonly GitHubClient _clientGist; + + public GitHubDevHosting(ISimpleLogger log, DevHostingConfiguration hostingConfiguration, string apiToken, string apiTokenUsage, string? gistApiToken = null) { Logger = log; Configuration = hostingConfiguration; _log = log; _url = hostingConfiguration.Base; _apiToken = apiToken; + _gistApiToken = gistApiToken ?? apiToken; ApiTokenUsage = apiTokenUsage; _client = new GitHubClient(new ProductHeaderValue(nameof(ReleaserApp)), new Uri(hostingConfiguration.Api)); + _clientGist = new GitHubClient(new ProductHeaderValue(nameof(ReleaserApp)), new Uri(hostingConfiguration.Api)); } public ISimpleLogger Logger { get; } @@ -43,7 +48,7 @@ public GitHubDevHosting(ISimpleLogger log, DevHostingConfiguration hostingConfig public async Task Connect() { - var tokenAuth = new Credentials(_apiToken); // NOTE: not real token + var tokenAuth = new Credentials(_apiToken); _client.Credentials = tokenAuth; _log.Info($"Connecting to GitHub ({ApiTokenUsage})"); @@ -56,6 +61,13 @@ public async Task Connect() _log.Error($"Unable to connect GitHub ({ApiTokenUsage}). Reason: {ex.Message}"); return false; } + + _clientGist.Credentials = tokenAuth; + if (_gistApiToken != _apiToken) + { + _clientGist.Credentials = new Credentials(_gistApiToken); + } + return true; } @@ -75,7 +87,17 @@ public async Task> GetAllReleaseTags(string user, string re public async Task CreateOrUpdateGist(string gistId, string fileName, string content) { - var gist = await _client.Gist.Get(gistId); + Gist gist; + try + { + gist = await _clientGist.Gist.Get(gistId); + } + catch (Exception ex) + { + _log.Error($"Unable to get the gist {gistId}. Reason: {ex.Message}"); + return; + } + if (gist is null) { _log.Error($"The gist {gistId} for code coverage was not found"); @@ -90,24 +112,23 @@ public async Task CreateOrUpdateGist(string gistId, string fileName, string cont _log.Info($"No need to update the gist {gistId} as the content is the same."); return; } + + // Update the file + GistUpdate gistUpdate = new GistUpdate(); + //gistUpdate.Files.Add(); + gistUpdate.Files.Add(fileName, new GistFileUpdate() { NewFileName = fileName, Content = content }); + await _clientGist.Gist.Edit(gistId, gistUpdate); } else { - _log.Warn($"Cannot update gist {gistId} as it does not contain the required file {fileName}"); - return; + _log.Error($"The gist {gistId} does not contain a file {fileName}"); } - - // Update the file - GistUpdate gistUpdate = new GistUpdate(); - //gistUpdate.Files.Add(); - gistUpdate.Files.Add(fileName, new GistFileUpdate() { NewFileName = fileName, Content = content }); - await _client.Gist.Edit(gistId, gistUpdate); } private async Task> GetAllReleaseTagsImpl(string user, string repo, string tagPrefix) { var tags = await _client.Repository.GetAllTags(user, repo); - var regex = new Regex(@$"{tagPrefix}(\d+(\.\d+)+.*)"); + var regex = new Regex(@$"^{tagPrefix}(\d+(\.\d+)+.*)"); var versions = new List<(RepositoryTag, NuGetVersion)>(); foreach (var tag in tags) { diff --git a/src/dotnet-releaser/GitInformation.cs b/src/dotnet-releaser/GitInformation.cs index 459cf78..e5658f4 100644 --- a/src/dotnet-releaser/GitInformation.cs +++ b/src/dotnet-releaser/GitInformation.cs @@ -43,8 +43,9 @@ private GitInformation(Repository repository, Branch branch, string branchName) { logger.Error($@"Unable to retrieve the current branch from the commit {repository.Head.Tip.Sha}. The current action requires it. Please make sure that: 1) The current commit is a checkout on a valid branch. -2) If running on GitHub Action, you are using `actions/checkout@v2` and not v1, but also that the property `fetch-depth: 0` is correctly setup. +2) If running on GitHub Action, you are using `actions/checkout@v4` and not v1, but also that the property `fetch-depth: 0` is correctly setup. 3) We have found the following branches containing this commit [{string.Join(",", repository.Branches.Select(x => x.FriendlyName))}]. +4) You pushed only tags (`git push --tags`) and forget to push whole branch.h "); return null; } @@ -70,4 +71,4 @@ private static string GetShortBranchName(Branch branch) return branchName; } -} \ No newline at end of file +} diff --git a/src/dotnet-releaser/Helpers/CompressionHelper.cs b/src/dotnet-releaser/Helpers/CompressionHelper.cs new file mode 100644 index 0000000..f1c88b8 --- /dev/null +++ b/src/dotnet-releaser/Helpers/CompressionHelper.cs @@ -0,0 +1,51 @@ +using System.Formats.Tar; +using System.IO; +using System.IO.Compression; + +namespace DotNetReleaser.Helpers; + +internal static class CompressionHelper +{ + public static string? MakeTarGz(ProjectPackageInfo projectPackageInfo, string publishDir, string artifactsFolder, string rid) + { + string? outputPath = null; + var publishPath = Path.Combine(Path.GetDirectoryName(projectPackageInfo.ProjectFullPath) ?? "", publishDir); + if (Directory.Exists(publishPath)) + { + var gzipPath = Path.GetFullPath(Path.Combine(artifactsFolder, + projectPackageInfo.Name + "." + projectPackageInfo.Version + "." + rid + ".tar.gz")); + if (File.Exists(gzipPath)) + { + return null; // file already exists + } + using FileStream fs = new(gzipPath, FileMode.CreateNew, FileAccess.Write); + using GZipStream gz = new(fs, CompressionMode.Compress, leaveOpen: true); + { + TarFile.CreateFromDirectory(publishPath, gz, includeBaseDirectory: false); + } + outputPath = gzipPath; + } + return outputPath; + } + + public static string? MakeZip(ProjectPackageInfo projectPackageInfo, string publishDir, string artifactsFolder, string rid) + { + string? outputPath = null; + var publishPath = Path.Combine(Path.GetDirectoryName(projectPackageInfo.ProjectFullPath) ?? "", publishDir); + if (Directory.Exists(publishPath)) + { + var zipPath = Path.GetFullPath(Path.Combine(artifactsFolder, + projectPackageInfo.Name + "." + projectPackageInfo.Version + "." + rid + ".zip")); + if (File.Exists(zipPath)) + { + return null; // file already exists + } + using FileStream fs = new(zipPath, FileMode.CreateNew, FileAccess.Write); + { + ZipFile.CreateFromDirectory(publishPath, fs, compressionLevel: CompressionLevel.Optimal, includeBaseDirectory: false); + } + outputPath = zipPath; + } + return outputPath; + } +} \ No newline at end of file diff --git a/src/dotnet-releaser/Program.cs b/src/dotnet-releaser/Program.cs index a1c7521..099cd1e 100644 --- a/src/dotnet-releaser/Program.cs +++ b/src/dotnet-releaser/Program.cs @@ -1,4 +1,3 @@ -using System; -using System.Diagnostics; using DotNetReleaser; + return await ReleaserApp.Run(args); \ No newline at end of file diff --git a/src/dotnet-releaser/ReleaserApp.AppPackaging.cs b/src/dotnet-releaser/ReleaserApp.AppPackaging.cs index 788b09e..bbcd210 100644 --- a/src/dotnet-releaser/ReleaserApp.AppPackaging.cs +++ b/src/dotnet-releaser/ReleaserApp.AppPackaging.cs @@ -4,8 +4,10 @@ using System.IO; using System.Linq; using System.Security.Cryptography; +using System.Text.RegularExpressions; using System.Threading.Tasks; using DotNetReleaser.Configuration; +using DotNetReleaser.Helpers; using DotNetReleaser.Logging; using Spectre.Console; @@ -71,7 +73,7 @@ private async Task> BuildAppPackages(BuildInformation build { foreach (var rid in pack.RuntimeIdentifiers) { - var list = await PackPlatform(packageInfo, pack.Publish, rid, pack.Kinds.ToArray()); + var list = await PackPlatform(packageInfo, pack.Publish, rid, pack.Renamers.ToArray(), pack.Kinds.ToArray()); if (HasErrors) goto exitPackOnError; // break on first errors if (list is not null && pack.Publish) @@ -99,7 +101,7 @@ private async Task> BuildAppPackages(BuildInformation build /// /// This is the part that handles the packaging for tar, zip, deb, rpm /// - private async Task?> PackPlatform(ProjectPackageInfo projectPackageInfo, bool publish, string rid, params PackageKind[] kinds) + private async Task?> PackPlatform(ProjectPackageInfo projectPackageInfo, bool publish, string rid, RegexReplacer[] renamers, PackageKind[] kinds) { var properties = new Dictionary(_config.MSBuild.Properties) { @@ -225,7 +227,54 @@ private async Task> BuildAppPackages(BuildInformation build // Copy the file to the output var path = result[0].ItemSpec; - path = CopyToArtifacts(path); + if (target == ReleaserConstants.DotNetReleaserPublishAndCreateTar) + { + path = CompressionHelper.MakeTarGz(projectPackageInfo, path, _config.ArtifactsFolder, rid); + if (path is null) + { + Error("Unable to make tar file with publish directory " + path + "; does the file already exist?"); + break; + } + } + else if (target == ReleaserConstants.DotNetReleaserPublishAndCreateZip) + { + path = CompressionHelper.MakeZip(projectPackageInfo, path, _config.ArtifactsFolder, rid); + if (path is null) + { + Error("Unable to make zip file with publish directory " + path + "; does the file already exist?"); + break; + } + } + else + { + path = CopyToArtifacts(path); + } + + // Give a chance to rename the artifact file + var folder = Path.GetDirectoryName(path)!; + var oldFilename = Path.GetFileName(path); + var filename = oldFilename; + string? newPath = null; + try + { + + foreach (var renamer in renamers) + { + filename = renamer.Run(filename); + } + + if (filename != oldFilename) + { + newPath = Path.Combine(folder, filename); + File.Move(path, newPath); + path = newPath; + } + } + catch (Exception ex) + { + Error($"Error renaming file {path} to {newPath}: {ex.Message}"); + break; + } var sha256 = string.Join("", SHA256.HashData(await File.ReadAllBytesAsync(path)).Select(x => x.ToString("x2"))); diff --git a/src/dotnet-releaser/ReleaserApp.BuildAndTests.cs b/src/dotnet-releaser/ReleaserApp.BuildAndTests.cs index 617378f..2ad5db3 100644 --- a/src/dotnet-releaser/ReleaserApp.BuildAndTests.cs +++ b/src/dotnet-releaser/ReleaserApp.BuildAndTests.cs @@ -81,12 +81,12 @@ private async Task BuildAndTest(IDevHosting? devHosting, BuildInformation { if (_config.Coverage.Enable) { - var coverageResult = LoadAndDisplayCoverageResults(); + var lineCoverageResult = LoadAndDisplayCoverageResults(); // Publish badge if requested if (devHosting is not null) { - await PublishCoverageToGist(devHosting, buildInfo, coverageResult); + await PublishCoverageToGist(devHosting, buildInfo, lineCoverageResult); } } @@ -160,7 +160,7 @@ private HitCoverage LoadAndDisplayCoverageResults() _logger.InfoMarkup("Coverage Results:", table); - return totalMethodRate; + return totalLineRate; } private async Task Build(string projectFile, bool isTestProject) diff --git a/src/dotnet-releaser/ReleaserApp.Changelog.cs b/src/dotnet-releaser/ReleaserApp.Changelog.cs index c954d13..1b6bf4e 100644 --- a/src/dotnet-releaser/ReleaserApp.Changelog.cs +++ b/src/dotnet-releaser/ReleaserApp.Changelog.cs @@ -26,7 +26,7 @@ private async Task ListOrUpdateChangelog(string configurationFilePath, str return false; } - var devHosting = await ConnectToDevHosting(_config.GitHub, githubApiToken, "For Fetching Changelog"); + var devHosting = await ConnectToDevHosting(_config.GitHub, githubApiToken, "For Fetching Changelog", null); if (devHosting is null) return false; return await ListOrUpdateChangelog(devHosting, version, update); diff --git a/src/dotnet-releaser/ReleaserApp.Configuring.cs b/src/dotnet-releaser/ReleaserApp.Configuring.cs index e4dcdcd..e35b4c8 100644 --- a/src/dotnet-releaser/ReleaserApp.Configuring.cs +++ b/src/dotnet-releaser/ReleaserApp.Configuring.cs @@ -15,7 +15,8 @@ namespace DotNetReleaser; public partial class ReleaserApp { - private async Task<(BuildInformation? buildInformation, IDevHosting? devHosting, IDevHosting? devHostingExtra)?> Configuring(string configurationFile, BuildKind buildKind, string githubApiToken, string? githubApiTokenExtra, string? nugetApiToken, bool forceArtifactsFolder, string? publishVersion) + private async Task<(BuildInformation? buildInformation, IDevHosting? devHosting, IDevHosting? devHostingExtra)?> Configuring(string configurationFile, BuildKind buildKind, string githubApiToken, string? githubApiTokenExtra, + string? githubApiTokenGist, string? nugetApiToken, bool forceArtifactsFolder, string? publishVersion) { // ------------------------------------------------------------------ // Load Configuration @@ -53,7 +54,7 @@ public partial class ReleaserApp // Connect to GitHub if we have a token if (!string.IsNullOrEmpty(githubApiToken)) { - devHosting = await ConnectToDevHosting(hostingConfiguration, githubApiToken, "For this CI"); + devHosting = await ConnectToDevHosting(hostingConfiguration, githubApiToken, "For this CI", githubApiTokenGist); if (devHosting is null) { return null; // return false; @@ -102,6 +103,8 @@ public partial class ReleaserApp return null; } + buildInformation.IsPush = gitHubInfo.EventName == "push"; + // Automatically convert a run into a publish if we have a release tag if (gitHubInfo.EventName == "push" && gitHubInfo.RefType == GitHubActionRefType.Tag) { diff --git a/src/dotnet-releaser/ReleaserApp.Coverage.cs b/src/dotnet-releaser/ReleaserApp.Coverage.cs index eccc751..c047216 100644 --- a/src/dotnet-releaser/ReleaserApp.Coverage.cs +++ b/src/dotnet-releaser/ReleaserApp.Coverage.cs @@ -13,9 +13,9 @@ namespace DotNetReleaser; public partial class ReleaserApp { - private async Task PublishCoverageToGist(IDevHosting devHosting, BuildInformation buildInfo, HitCoverage coverage) + private async Task PublishCoverageToGist(IDevHosting devHosting, BuildInformation buildInfo, HitCoverage lineCoverage) { - if (!_config.Coverage.BadgeUploadToGist) return; + if (!_config.Coverage.BadgeUploadToGist || !buildInfo.IsPush) return; var gistId = _config.Coverage.BadgeGistId; if (string.IsNullOrWhiteSpace(gistId)) @@ -24,9 +24,19 @@ private async Task PublishCoverageToGist(IDevHosting devHosting, BuildInformatio return; } - var rate = (int)(Math.Round((double)coverage.Rate) * 100); + var rate = (int)Math.Round((double)lineCoverage.Rate * 100); + + // TODO: We could make many of these things configurable (colors, size of the badge, etc.) + var color = rate switch + { + >= 95 => "#4c1", + >= 90 => "#a3c51c", + >= 75 => "#dfb317", + _ => "#e05d44" + }; + var svg = $""" - coveragecoverage{rate:##}%{rate:##}% + coveragecoverage{rate:##}%{rate:##}% """; var fileName = $"dotnet-releaser-coverage-badge-{_config.GitHub.User}-{_config.GitHub.Repo}.svg"; @@ -36,7 +46,7 @@ private async Task PublishCoverageToGist(IDevHosting devHosting, BuildInformatio private async Task PublishCoveralls(IDevHosting devHosting, BuildInformation buildInfo) { - if (!_config.Coveralls.Publish || _assemblyCoverages.Count == 0) return; + if (!_config.Coveralls.Publish || _assemblyCoverages.Count == 0 || !buildInfo.IsPush) return; var ownerRepo = $"{devHosting.Configuration.User}/{devHosting.Configuration.Repo}"; diff --git a/src/dotnet-releaser/ReleaserApp.DevHosting.cs b/src/dotnet-releaser/ReleaserApp.DevHosting.cs index 9c1c7bf..c3cdf0d 100644 --- a/src/dotnet-releaser/ReleaserApp.DevHosting.cs +++ b/src/dotnet-releaser/ReleaserApp.DevHosting.cs @@ -6,9 +6,9 @@ namespace DotNetReleaser; public partial class ReleaserApp { - private async Task ConnectToDevHosting(DevHostingConfiguration hostingConfiguration, string githubApiToken, string apiTokenUsage) + private async Task ConnectToDevHosting(DevHostingConfiguration hostingConfiguration, string githubApiToken, string apiTokenUsage, string? githubApiTokenGist = null) { - var hosting = new GitHubDevHosting(_logger, hostingConfiguration, githubApiToken, apiTokenUsage); + var hosting = new GitHubDevHosting(_logger, hostingConfiguration, githubApiToken, apiTokenUsage, githubApiTokenGist); if (await hosting.Connect()) { diff --git a/src/dotnet-releaser/ReleaserApp.Projects.cs b/src/dotnet-releaser/ReleaserApp.Projects.cs index 5e1a8f6..4ac0ee5 100644 --- a/src/dotnet-releaser/ReleaserApp.Projects.cs +++ b/src/dotnet-releaser/ReleaserApp.Projects.cs @@ -1,4 +1,4 @@ -using System; +using System; using System.Collections.Concurrent; using System.Collections.Generic; using System.Diagnostics; @@ -10,7 +10,6 @@ using System.Threading.Tasks; using DotNetReleaser.Helpers; using DotNetReleaser.Runners; -using Microsoft.Build.Construction; using Microsoft.Build.Framework; using NuGet.Frameworks; using Spectre.Console; @@ -43,6 +42,8 @@ public BuildInformation(ProjectPackageInfoCollection[] projectPackageInfoCollect public bool PublishNuGet { get; set; } + public bool IsPush { get; set; } + public bool AllowPublishDraft { get; set; } public BuildKind BuildKind { get; set; } @@ -210,31 +211,30 @@ public partial class ReleaserApp foreach (var msBuildProject in _config.MSBuild.Projects) { - if (msBuildProject.EndsWith(".sln", StringComparison.OrdinalIgnoreCase)) + var basePath = Path.GetDirectoryName(msBuildProject)!; + var solutionSerializer = Microsoft.VisualStudio.SolutionPersistence.Serializer.SolutionSerializers.GetSerializerByMoniker(msBuildProject); + if (solutionSerializer is not null) { // solution file try { - var solutionFile = SolutionFile.Parse(msBuildProject); - foreach (var subProject in solutionFile.ProjectsInOrder) + var solutionFile = await solutionSerializer.OpenAsync(msBuildProject, CancellationToken.None); + foreach (var subProject in solutionFile.SolutionProjects) { - if (subProject.ProjectType == SolutionProjectType.KnownToBeMSBuildFormat) - { - var fullProjectPath = Path.GetFullPath(subProject.AbsolutePath); + var fullProjectPath = Path.GetFullPath(Path.Combine(basePath, subProject.FilePath)); - if (allProjectPaths.Add(fullProjectPath)) - { - if (!solutionToProjects.TryGetValue(msBuildProject, out var listOfProjectsPerSolution)) - { - listOfProjectsPerSolution = new List(); - solutionToProjects[msBuildProject] = listOfProjectsPerSolution; - } - listOfProjectsPerSolution.Add(fullProjectPath); - } - else + if (allProjectPaths.Add(fullProjectPath)) + { + if (!solutionToProjects.TryGetValue(msBuildProject, out var listOfProjectsPerSolution)) { - Error($"The project `{fullProjectPath}` is duplicated in the list of input projects."); + listOfProjectsPerSolution = new List(); + solutionToProjects[msBuildProject] = listOfProjectsPerSolution; } + listOfProjectsPerSolution.Add(fullProjectPath); + } + else + { + Error($"The project `{fullProjectPath}` is duplicated in the list of input projects."); } } } diff --git a/src/dotnet-releaser/ReleaserApp.cs b/src/dotnet-releaser/ReleaserApp.cs index e555d6e..2d4d68a 100644 --- a/src/dotnet-releaser/ReleaserApp.cs +++ b/src/dotnet-releaser/ReleaserApp.cs @@ -1,4 +1,4 @@ -using System; +using System; using System.Collections.Generic; using System.IO; using System.Linq; @@ -12,6 +12,7 @@ using DotNetReleaser.Logging; using Lunet.Extensions.Logging.SpectreConsole; using McMaster.Extensions.CommandLineUtils; +using Microsoft.Build.Locator; using Microsoft.Extensions.Logging; using Spectre.Console; using Spectre.Console.Rendering; @@ -58,6 +59,8 @@ private ReleaserApp(ISimpleLogger logger) /// 0 if successful; 1 otherwise. public static async Task Run(string[] args) { + MSBuildLocator.RegisterDefaults(); + Console.OutputEncoding = Encoding.UTF8; // Create our log var runningOnGitHubAction = GitHubActionHelper.IsRunningOnGitHubAction; @@ -174,6 +177,11 @@ CommandOption AddGitHubTokenExtra(CommandLineApplication cmd) return cmd.Option("--github-token-extra ", "GitHub Api Token. Required if publish homebrew to GitHub is true in the config file. In that case dotnet-releaser needs a personal access GitHub token which can create the homebrew repository. This token has usually more access than the --github-token that is only used for the current repository. ", CommandOptionType.SingleValue); } + CommandOption AddGitHubTokenGist(CommandLineApplication cmd) + { + return cmd.Option("--github-token-gist ", "GitHub Api Token. Required if publishing to a gist used for e.g coverage.", CommandOptionType.SingleValue); + } + CommandArgument AddTomlConfigurationArgument(CommandLineApplication cmd, bool forNew) { var arg = cmd.Argument("dotnet-releaser.toml", forNew ? "TOML configuration file path to create. Default is: dotnet-releaser.toml" : "The input TOML configuration file."); @@ -185,6 +193,7 @@ void AddPublishOrBuildArgs(CommandLineApplication cmd) { CommandOption? nugetToken = null; CommandOption? gitHubTokenExtra = null; + CommandOption? gitHubTokenGist = null; CommandOption? skipAppPackagesOption = null; var githubToken = AddGitHubToken(cmd); @@ -195,6 +204,7 @@ void AddPublishOrBuildArgs(CommandLineApplication cmd) nugetToken = cmd.Option("--nuget-token ", "NuGet Api Token. Required if publish to NuGet is true in the config file", CommandOptionType.SingleValue); gitHubTokenExtra = AddGitHubTokenExtra(cmd); + gitHubTokenGist = AddGitHubTokenGist(cmd); } else { @@ -237,7 +247,7 @@ void AddPublishOrBuildArgs(CommandLineApplication cmd) { appReleaser._tableBorder = GetTableBorderFromKind(tableKindOption.ParsedValue); } - var result = await appReleaser.RunImpl(configurationFilePath, buildKind, githubToken.ParsedValue, gitHubTokenExtra?.ParsedValue, nugetToken?.ParsedValue, forceOption.ParsedValue, forceUploadOption?.ParsedValue ?? false, publishVersion?.ParsedValue); + var result = await appReleaser.RunImpl(configurationFilePath, buildKind, githubToken.ParsedValue, gitHubTokenExtra?.ParsedValue, gitHubTokenGist?.ParsedValue, nugetToken?.ParsedValue, forceOption.ParsedValue, forceUploadOption?.ParsedValue ?? false, publishVersion?.ParsedValue); return result ? 0 : 1; }); } @@ -299,7 +309,8 @@ private async Task LoadConfiguration(string configurationFile) /// /// Runs the releaser app /// - private async Task RunImpl(string configurationFile, BuildKind buildKind, string githubApiToken, string? githubApiTokenExtra, string? nugetApiToken, bool forceArtifactsFolder, bool forceUpload, string? publishVersion) + private async Task RunImpl(string configurationFile, BuildKind buildKind, string githubApiToken, string? githubApiTokenExtra, string? gitHubTokenGist, string? nugetApiToken, bool forceArtifactsFolder, bool forceUpload, + string? publishVersion) { BuildInformation? buildInformation = null; GitHubDevHostingConfiguration? hostingConfiguration = null; @@ -310,7 +321,7 @@ private async Task RunImpl(string configurationFile, BuildKind buildKind, { _logger.Info($"dotnet-releaser {Version} - {buildKind.ToString().ToLowerInvariant()}"); _logger.LogStartGroup($"Configuring"); - var result = await Configuring(configurationFile, buildKind, githubApiToken, githubApiTokenExtra, nugetApiToken, forceArtifactsFolder, publishVersion); + var result = await Configuring(configurationFile, buildKind, githubApiToken, githubApiTokenExtra, gitHubTokenGist, nugetApiToken, forceArtifactsFolder, publishVersion); if (result is null) return false; buildInformation = result.Value.buildInformation!; devHosting = result.Value.devHosting; diff --git a/src/dotnet-releaser/Runners/DotNetRunner.cs b/src/dotnet-releaser/Runners/DotNetRunner.cs index 13b91f3..5738ba2 100644 --- a/src/dotnet-releaser/Runners/DotNetRunner.cs +++ b/src/dotnet-releaser/Runners/DotNetRunner.cs @@ -8,7 +8,7 @@ public DotNetRunner(string command) : base(command) { } - public async Task Run() + public async Task Run() { return await RunImpl(); } diff --git a/src/dotnet-releaser/Runners/DotNetRunnerBase.cs b/src/dotnet-releaser/Runners/DotNetRunnerBase.cs index 499a7a4..5100b5c 100644 --- a/src/dotnet-releaser/Runners/DotNetRunnerBase.cs +++ b/src/dotnet-releaser/Runners/DotNetRunnerBase.cs @@ -1,4 +1,4 @@ -using System; +using System; using System.Collections.Generic; using System.Diagnostics; using System.Globalization; @@ -9,7 +9,7 @@ namespace DotNetReleaser.Runners; -public record DotNetResult(CommandResult CommandResult, string CommandLine, string Output) +public record CommandResulExtended(CommandResult CommandResult, string CommandLine, string Output) { public bool HasErrors => CommandResult.ExitCode != 0; } @@ -43,7 +43,7 @@ protected DotNetRunnerBase(string command) protected virtual IReadOnlyDictionary ComputeProperties() => Properties; - protected async Task RunImpl() + protected async Task RunImpl() { return await Run(Command, ComputeArguments(), ComputeProperties(), WorkingDirectory); } @@ -83,7 +83,7 @@ private static string GetPropertyValueAsString(object value) return value.ToString() ?? string.Empty; } - private async Task Run(string command, IEnumerable args, IReadOnlyDictionary? properties = null, string? workingDirectory = null) + private async Task Run(string command, IEnumerable args, IReadOnlyDictionary? properties = null, string? workingDirectory = null) { var stdOutAndErrorBuffer = new StringBuilder(); @@ -102,7 +102,7 @@ private async Task Run(string command, IEnumerable args, I RunAfterStart?.Invoke(); var result = await wrap.ConfigureAwait(false); - return new DotNetResult(result, $"dotnet {arguments}",stdOutAndErrorBuffer.ToString()); + return new CommandResulExtended(result, $"dotnet {arguments}",stdOutAndErrorBuffer.ToString()); } protected virtual void Dispose(bool disposing) diff --git a/src/dotnet-releaser/Runners/GitRunner.cs b/src/dotnet-releaser/Runners/GitRunner.cs new file mode 100644 index 0000000..d3c8616 --- /dev/null +++ b/src/dotnet-releaser/Runners/GitRunner.cs @@ -0,0 +1,39 @@ +// Copyright (c) Alexandre Mutel. All rights reserved. +// Licensed under the BSD-Clause 2 license. +// See license.txt file in the project root for full license information. + +using CliWrap; +using System.Collections.Generic; +using System.Text; +using System.Threading.Tasks; +using System; +using CliWrap.Builders; + +namespace DotNetReleaser.Runners; + +public static class GitRunner +{ + public static async Task Run(string command, IEnumerable args, string? workingDirectory = null) + { + var stdOutAndErrorBuffer = new StringBuilder(); + + var argsBuilder = new ArgumentsBuilder(); + argsBuilder.Add(command); + foreach (var arg in args) + { + argsBuilder.Add(arg); + } + var arguments = argsBuilder.Build(); + + var wrap = Cli.Wrap("git") + .WithArguments(arguments) + .WithWorkingDirectory(workingDirectory ?? Environment.CurrentDirectory) + .WithStandardOutputPipe(PipeTarget.ToStringBuilder(stdOutAndErrorBuffer)) + .WithStandardErrorPipe(PipeTarget.ToStringBuilder(stdOutAndErrorBuffer)) + .WithValidation(CommandResultValidation.None) + .ExecuteAsync(); + + var result = await wrap.ConfigureAwait(false); + return new CommandResulExtended(result, $"git {arguments}", stdOutAndErrorBuffer.ToString()); + } +} diff --git a/src/dotnet-releaser/Runners/MSBuildProgram.cs b/src/dotnet-releaser/Runners/MSBuildProgram.cs index 9198e0c..fb2afaf 100644 --- a/src/dotnet-releaser/Runners/MSBuildProgram.cs +++ b/src/dotnet-releaser/Runners/MSBuildProgram.cs @@ -1,5 +1,6 @@ -using System; +using System; using System.Collections.Generic; +using System.Diagnostics; using System.Linq; using System.Threading; using System.Threading.Tasks; @@ -10,7 +11,7 @@ namespace DotNetReleaser.Runners; -public record MSBuildResult(CommandResult CommandResult, string CommandLine, string Output, Dictionary> TargetOutputs) : DotNetResult(CommandResult, CommandLine, Output); +public record MSBuildResult(CommandResult CommandResult, string CommandLine, string Output, Dictionary> TargetOutputs) : CommandResulExtended(CommandResult, CommandLine, Output); public class MSBuildRunner : DotNetRunnerBase { @@ -81,11 +82,14 @@ public async Task Run(ISimpleLogger logger) if (string.IsNullOrEmpty(Project)) throw new InvalidOperationException("MSBuildRunner.Project cannot be empty"); // Create the server - var reader = new AnonymousPipeLoggerServer(); + var readerSource = new CancellationTokenSource(); + var reader = new AnonymousPipeLoggerServer(readerSource.Token); // Get the pipe handle _pipeHandle = reader.GetClientHandle(); + Thread? readerThread = null; + //logger.Info($"MSBuild handle {_pipeHandle}"); //var taskReader = Task.Factory.StartNew(() => @@ -120,12 +124,24 @@ public async Task Run(ISimpleLogger logger) RunAfterStart = () => { - //Console.WriteLine($"Start ReadAll {Thread.CurrentThread.ManagedThreadId}"); - reader.ReadAll(); - //Console.WriteLine($"End ReadAll {Thread.CurrentThread.ManagedThreadId}"); + readerThread = new Thread(() => + { + //Console.WriteLine($"Start ReadAll {Thread.CurrentThread.ManagedThreadId}"); + reader.ReadAll(); + //Console.WriteLine($"End ReadAll {Thread.CurrentThread.ManagedThreadId}"); + } + ) + { + Name = "AnonymousPipeLoggerServer.ReadAll", + IsBackground = true, + }; + readerThread.Start(); + }; var result = await base.RunImpl(); + readerSource.Cancel(); + if (result.CommandResult.ExitCode != 0) { logger.Error($"Failing to run {result.CommandLine}. Reason: {result.Output}"); diff --git a/src/dotnet-releaser/dotnet-releaser.csproj b/src/dotnet-releaser/dotnet-releaser.csproj index d1e376e..043bf6a 100644 --- a/src/dotnet-releaser/dotnet-releaser.csproj +++ b/src/dotnet-releaser/dotnet-releaser.csproj @@ -1,7 +1,7 @@  Exe - net7.0 + net9.0 DotNetReleaser dotnet-releaser enable @@ -44,36 +44,36 @@ - - - - + + + + NU1608 - + NU1608 - + + NU1608 - + NU1608 - - + + + all runtime; build; native; contentfiles; analyzers; buildtransitive - - - - - - - - - - + + + + + + + + all @@ -81,4 +81,9 @@ + + + + + diff --git a/src/dotnet-releaser/dotnet-releaser.targets b/src/dotnet-releaser/dotnet-releaser.targets index d1be68d..0f434f0 100644 --- a/src/dotnet-releaser/dotnet-releaser.targets +++ b/src/dotnet-releaser/dotnet-releaser.targets @@ -48,7 +48,7 @@ - + all runtime; build; native; contentfiles; analyzers; buildtransitive @@ -56,7 +56,7 @@ - + all runtime; build; native; contentfiles; analyzers; buildtransitive @@ -100,6 +100,7 @@ <_DotNetReleaserGetPackageInfo Include="$(IsPackable)" Kind="IsNuGetPackable"/> <_DotNetReleaserGetPackageInfo Include="$(IsTestProject)" Kind="IsTestProject"/> <_DotNetReleaserGetPackageInfo Include="@(ProjectReference)" Kind="ProjectReference"/> + <_DotNetReleaserGetPackageInfo Include="$([System.IO.Path]::GetFullPath('$(PublishDir)'))" Kind="PublishDir"/> @@ -116,14 +117,14 @@ <_DotNetReleaserPublishAndCreateRpm Include="$(RpmPath)" Kind="RpmPath"/> - + - <_DotNetReleaserPublishAndCreateZip Include="$(ZipPath)" Kind="ZipPath"/> + <_DotNetReleaserPublishAndCreateZip Include="$(PublishDir)" Kind="PublishDir"/> - + - <_DotNetReleaserPublishAndCreateTar Include="$(TarballPath)" Kind="TarballPath"/> + <_DotNetReleaserPublishAndCreateTar Include="$(PublishDir)" Kind="PublishDir"/> \ No newline at end of file diff --git a/src/global.json b/src/global.json index aaf58f5..9c364a0 100644 --- a/src/global.json +++ b/src/global.json @@ -1,6 +1,6 @@ { "sdk": { - "version": "7.0.100", + "version": "9.0.100", "rollForward": "latestMinor", "allowPrerelease": false }