Initial ScottPlotExporter with just Bar Plot and Unit Tests - #2560
Merged
Conversation
timcassell
reviewed
Apr 13, 2024
timcassell
reviewed
Apr 13, 2024
Contributor
Author
|
@dotnet-policy-service agree |
timcassell
reviewed
Apr 14, 2024
timcassell
reviewed
Apr 14, 2024
timcassell
reviewed
Apr 15, 2024
timcassell
reviewed
Apr 19, 2024
….Exporters.Plotting.Tests.csproj
Contributor
Author
|
Hey @timcassell and @AndreyAkinshin just checking in for a status update on this one. I'm happy for any feedback. My plan was to add the rest of the charts that the R plot has once this PR was accepted. |
timcassell
approved these changes
May 13, 2024
timcassell
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, but @AndreyAkinshin controls the Nuget packages, so he'll need to approve this and get it on Nuget.
AndreyAkinshin
approved these changes
Jul 20, 2024
Member
|
@FlatlinerDOA thanks! |
Contributor
|
I have been trying out this new ScottPlotExporter but getting some weird results. Perhaps I am using it wrong? var exporter = new ScottPlotExporter();
var logger = new AccumulationLogger();
var summaries = BenchmarkSwitcher.FromAssembly(Assembly.GetExecutingAssembly()).Run(args);
foreach (var summary in summaries)
{
var filePaths = exporter.ExportToFiles(summary, logger).ToList();
WriteLine(string.Join(Environment.NewLine, filePaths));
} |
Contributor
Author
Contributor
|
@FlatlinerDOA not sure I fully follow given what I show is way off from plot, e. g. PlainStruct has mean +170 ns but nothing in plot is anywhere near that, all values around 10-12ns... that is they are much lower. |
This was referenced Jul 23, 2025
This was referenced Aug 1, 2025
This was referenced May 14, 2026
This was referenced Jun 7, 2026
This was referenced Jun 14, 2026
This was referenced Jun 25, 2026
This was referenced Jul 6, 2026
Closed
Closed
This was referenced Jul 20, 2026
liuhaoyang
pushed a commit
to dotnetcore/mocha
that referenced
this pull request
Jul 27, 2026
Updated [BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet) from 0.13.12 to 0.15.8. <details> <summary>Release notes</summary> _Sourced from [BenchmarkDotNet's releases](https://github.com/dotnet/BenchmarkDotNet/releases)._ ## 0.15.8 This release adds OpenMetrics exporter support for Prometheus-compatible metrics export, improves the Roslyn analyzers with multi-target support and better type checking, and fixes several bugs including process deadlocks and WASM trimming issues. ## Features - Add OpenMetrics exporter for Prometheus-compatible metrics output (#2801) - Add Job info to DisassemblyDiagnoser report headers to distinguish assemblies when using multiple coreruns (#2884, fixes #2573) - Add `NO_COLOR` environment variable support for disabling console colors (#2870) ## Improvements - Multi-target analyzers with improved type assignability checking using semantic model (#2866) - Add new analyzer diagnostic BDN1503 for better argument/params validation (#2865, fixes #2864) - Use PolySharp for `[DynamicallyAccessedMembers]` attribute polyfill (#2883) - Refactor to use `AsyncProcessOutputReader` for cleaner process output handling (#2878) ## Bug Fixes - Fix process deadlock issue when reading process output (#2877) - Fix WASM generated project being trimmed out (#2872) - Allow filters to filter out every benchmark from a type without errors (#2879, fixes #2860) - Fix unhandled exception when running `BenchmarkRunner.Run<T>()` with arguments on invalid benchmark type (#2880, fixes #2724) ## Internal - Update release workflow for analyzers (#2882) - Improve docs building workflow - Cleanup `#if`-`#endif` preprocessor directives using PolySharp polyfills (#2881) **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.7...v0.15.8 ## 0.15.7 This release introduces Roslyn analyzers to catch incorrect BenchmarkDotNet usage at compile time, improves .NET Framework version detection, and updates OS detection support. ## Features - Add Roslyn analyzers to detect incorrect usage of BenchmarkDotNet at compile-time (#2837) - Validates benchmark class structure (public, non-sealed, generic constraints) - Checks `[Arguments]`, `[Params]`, and `[ParamsAllValues]` attribute usage - Verifies `[GenericTypeArguments]` requirements - Ensures only one baseline method per category - Validates `BenchmarkRunner.Run` invocations ## Improvements - Improve .NET Framework version detection by retrieving version from `TargetFrameworkAttribute` (#2682) - Bump Perfolizer 0.6.0 → 0.6.1, bringing updated Windows and macOS version detection in `OsBrandHelper` ## Bug Fixes - Fix null reference handling and exception logging in `TestCaseFilter` for the test adapter - Fix flaky CI tests by increasing build timeout values (#2854) ## Internal - Improve release workflow in `release.yaml` **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.6...v0.15.7 ## 0.15.6 ## v0.15.6 This release adds ref struct parameter support for `[ArgumentsSource]`, fixes Native AOT runtime moniker resolution, and upgrades to Perfolizer 0.6.0 with the new Pragmastat statistical engine. ### Features - Add ref struct parameter support for `[ArgumentsSource]` attribute, enabling `Span<T>` and `ReadOnlySpan<char>` parameters (#2849) ### Bug Fixes - Fix runtime moniker normalization for Native AOT targets (#2852) ### Improvements - Upgrade to Perfolizer 0.6.0 with Pragmastat statistical engine integration ### Documentation - Add documentation for breaking changes related to disassembler native dependencies (#2836) ### Internal - Introduce GitHub Actions release workflow **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.5...v0.15.6 ## 0.15.5 This release fixes job naming consistency when using `--runtimes`, clamps histogram bin bounds to avoid confusing negative values, and reduces output directory clutter by filtering unnecessary runtime and satellite assembly files. ## Features - Add custom MSBuild targets to remove unnecessary files from the bin directory (#2737) - Filters out Capstone native binaries for non-target platforms - Removes satellite assemblies from Microsoft.CodeAnalysis packages ## Bug Fixes - Fix job names consistency between `SimpleJobAttribute` and `--runtimes` CLI option (#2841) - Jobs now use runtime names as IDs consistently across all runtime monikers - Clamp histogram bin lower bounds to non-negative values (#1821) - Prevents confusing negative values in histogram output for non-negative measurements ## Internal - Bump Perfolizer: 0.5.3 → 0.5.4 (#2773) - Update changelog and GitHub Pages generation workflows - Enable `workflow_dispatch` for test workflow (#2835) **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.4...v0.15.5 ## 0.15.4 This release fixes issues with `ParamsSource` attribute resolution in inheritance scenarios and corrects a MSBuild syntax error in the TestAdapter. ## Bug Fixes - Allow `[ParamsSource]` to resolve overridden methods and properties in derived classes (#2832) - Fix MSBuild condition syntax for `TestTfmsInParallel` property that prevented Visual Studio from loading projects (#2831) **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.3...v0.15.4 ## 0.15.3 This release brings .NET 10 NativeAOT instruction set support, improved CPU detection on Windows when WMIC is unavailable, test adapter filtering, and numerous bug fixes. ## Breaking Changes - Deprecated `.WithNuget()` job extension in favor of `.WithMsBuildArguments()` (#2812) ## Features - Add VS Test Adapter filter support for running specific benchmarks (#2788) - Update NativeAOT instruction set support for .NET 10+ (#2828) ## Improvements - Add PowerShell-based CPU detection fallback for Windows when WMIC is unavailable (#2749) - Improve `IsNetCore` and `IsNativeAOT` detection for single-file apps without AOT (#2799) - Use `--nodeReuse:false` for dotnet CLI commands to improve build isolation (#2814) - Enable assembly signing for debug builds (#2774) ## Bug Fixes - Fix `ArgumentsSource` on external types not working if the argument type is not primitive (#2820) - Fix workload warmup mode not working correctly - Fix `EtwProfiler` for file paths slightly under 260 characters (#2808) - Fix console logs being output twice when using TestAdapter (#2790) - Fix `EventProcessor.OnEndValidationStage` not being called when critical validation errors occur (#2816) - Fix `XmlException` thrown when `TextReader.Null` is passed to `AppConfigGenerator` (#2817) - Fix case sensitivity issue in `NativeMemoryLogParser` program name matching (#2795) - Fix typo in `BuildPlots.R` ## Internal - Replace StyleCop.Analyzers with unstable version for improved analysis (#2796) - Add workflow to run selected tests (#2797) - Fix flaky `MemoryDiagnoser` tests on macOS (#2813) - Fix x86 disassembler tests for net462 (#2792) - Split `TimeConsumingBenchmark` class to reduce test time - Update `BenchmarkDotNetDiagnosers` package version (#2805) - Fix comment in package props about `GenerateProgramFile` (#2802) **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.2...v0.15.3 ## 0.15.2 This release improves memory allocation measurement accuracy and adds new features for job ordering and runtime validation. ## Features - Add `JobOrderPolicy` option to sort jobs in numeric order instead of ordinal order (#2770) - Add `RuntimeValidator` to detect benchmarks with null runtime configuration (#2771) ## Improvements - Improve memory diagnoser accuracy with better allocation measurement isolation, warm-up phase, and handling of tiered JIT (#2562) ## Bug Fixes - Auto-generate unique job IDs between benchmark runs to prevent ID collisions - Skip null runtime validation for in-process toolchain (#2780) - Fix flaky memory allocation test (#2782) - Fix benchmark test adapter enumeration issues (#2766) ## Internal - Modify macOS runner image for CI (#2775) - Add setting to skip test reports when original workflow is cancelled (#2772) - Suppress xunit non-serializable data warnings (#2769) - Enable `--force-clone` for docs-fetch in generate-gh-pages workflow - Allow `workflow_dispatch` for publish-nightly workflow - Enhance docs-fetch command with additional options - Remove `docs/_changelog` folder from main branch (migrated to docs-changelog branch) **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.1...v0.15.2 ## 0.15.1 A maintenance release with improved cross-platform compatibility, a new feature for referencing external types in source attributes, and several bug fixes for ARM CPUs and unsupported operating systems. ## Features - Allow `[ArgumentsSource]` and `[ParamsSource]` to reference methods in other types via new constructor overload: `[ArgumentsSource(typeof(MyClass), nameof(MyClass.Values))]` (#2748) ## Bug Fixes - Fix WakeLock P/Invoke compatibility with ARM CPUs by refactoring `REASON_CONTEXT` to use proper union structure (#2745, #2756) - Fix `Console.CancelKeyPress` crash on platforms that don't support it (Android, iOS, tvOS, WASM) (#2739, #2741) - Fix CPU detection crash on unsupported operating systems by returning `CpuInfo.Unknown` (#2740) - Support `.slnx` solution file format when searching for solution files (#2764) ## Improvements - Bump Perfolizer: 0.5.2 → 0.5.3 - Make `ExporterBase.GetArtifactFullName` accessibility modifier more permissive ## Internal - Update .NET SDK version to 8.0.410 (#2762) - Update Microsoft.NET.Test.Sdk and other package dependencies (#2750, #2755) - Rework changelog generation to use `docs-changelog` branch (#93d12c42) - Fix line-endings to LF in several files - Update GitHub Actions workflows **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.0...v0.15.1 ## 0.15.0 BenchmarkDotNet v0.15.0 brings .NET 10 support, a new WakeLock feature to prevent system sleep during benchmarks, improved engine internals for more consistent measurements, and numerous bug fixes and improvements. ## Features - **WakeLock support**: New `[WakeLock]` attribute and `--wakeLock` CLI option to prevent the system from entering sleep mode while benchmarks are running (#2670) - **.NET 10 support**: Added `RuntimeMoniker.Net10`, `NativeAot10`, and `Mono10` with full toolchain support (#2642) - **Box plots in ScottPlotExporter**: New box plot visualization for benchmark results with improved font sizing - **RiscV64 platform support**: Added `Platform.RiscV64` for RISC-V 64-bit architecture (#2644, #2647) - **Required properties support**: Benchmark classes can now use C# 11 `required` properties (#2579) - **HostSignal.AfterProcessStart**: New signal allows diagnosers to obtain the process ID of a benchmark process started in suspended state (#2674) - **Parallel build control**: New `ConfigOptions.DisableParallelBuild` option to force sequential builds (#2725) - **Auto-hide empty metric columns**: `ThreadingDiagnoser` and `ExceptionDiagnoser` now support configuration to hide columns when metrics have no values (#2673) - **Measurements in DiagnoserResults**: Custom diagnosers can now access measurements for calculations (#2731) ## Improvements - **Constant stack size engine refactoring**: Engine stages refactored to use `IEngineStageEvaluator` for more consistent instruction location and simpler code (#2688) - **Use ArtifactsPath instead of IntermediateOutputPath**: Improved build artifact handling for SDK 8+ (#2676) - **InProcessNoEmitRunner NativeAOT support**: Basic support for running `InProcessNoEmitRunner` with NativeAOT (#2702) - **Allow ParamsAttribute values from derived classes**: `ParamsAttribute.Values` setter is now protected instead of private (#2716) - **Updated clrmd to 3.1**: Disassembler now uses `ClrMdV3Disassembler` (#2488) - **Updated ScottPlot to 5.0.54**: Plotting exporter updated to latest ScottPlot version (#2709) - **Perfolizer upgraded to 0.5.2**: CPU/OS detection logic moved to Perfolizer with new `Perfonar` exporters replacing `Phd` exporters - **Log warnings for empty benchmarks**: Clear warning messages when running benchmarks that match no methods (#2718) - **Validation for sealed benchmark classes**: Compiler now warns when benchmark classes are sealed (#2660) - **Improved baseline warning message**: More descriptive warning when baseline benchmarks are misconfigured (#2650) ## Bug Fixes - **Fix async GlobalSetup/GlobalCleanup with InProcessEmit**: Async setup and cleanup methods are now properly awaited (#2109) - **Fix Windows path too long**: Handle Windows MAX_PATH limitations in build paths (#2681) - **Fix builds with --keepFiles**: Include auto-incremented ID in build artifacts directory to avoid conflicts (#2423) - **Fix Ctrl-C handling**: System state (power management, console title) is now properly reverted on process termination (#2483, #2661) - **Fix dotnet command failure detection**: Commands now always fail when dotnet returns non-zero exit code (#2535) - **Fix deadlock in GetDotNetSdkVersion**: Resolved potential deadlock in SDK version detection (#2622) - **Fix lscpu CPU frequency parsing**: Corrected frequency parsing from lscpu output - **Make lscpu call language-invariant**: CPU detection now works correctly regardless of system locale (#2577) - **Clean up unsupported Native AOT flags**: Removed obsolete `IlcGenerateCompleteTypeMetadata` and updated flag names (#2616) - **Native AOT projects copy SettingsWeWantToCopy**: Build settings are now properly propagated (#2665) ## Breaking Changes - **Removed ConfigCompatibilityValidator**: No longer validates config compatibility between runs (#2599) - **Phd exporters renamed to Perfonar**: `[PhdExporter]` → `[PerfonarExporter]`, `PhdJsonExporter` → `PerfonarJsonExporter`, `PhdMdExporter` → `PerfonarMdExporter` - **Removed netstandard1.0 target from Annotations package**: Minimum target is now netstandard2.0 ## Documentation - Added Visual Studio Profiler documentation with samples (#2672) - Added WakeLock documentation and samples (#2670) ... (truncated) ## 0.14.0 Full changelog: https://benchmarkdotnet.org/changelog/v0.14.0.html ## Highlights * Introduce `BenchmarkDotNet.Diagnostics.dotMemory` [#2549](dotnet/BenchmarkDotNet#2549): memory allocation profile of your benchmarks using [dotMemory](https://www.jetbrains.com/dotmemory/), see @BenchmarkDotNet.Samples.IntroDotMemoryDiagnoser * Introduce `BenchmarkDotNet.Exporters.Plotting` [#2560](dotnet/BenchmarkDotNet#2560): plotting via [ScottPlot](https://scottplot.net/) (initial version) * Multiple bugfixes * The default build toolchains have been updated to pass `IntermediateOutputPath`, `OutputPath`, and `OutDir` properties to the `dotnet build` command. This change forces all build outputs to be placed in a new directory generated by BenchmarkDotNet, and fixes many issues that have been reported with builds. You can also access these paths in your own `.csproj` and `.props` from those properties if you need to copy custom files to the output. ## Bug fixes * Fixed multiple build-related bugs including passing MsBuildArguments and .Net 8's `UseArtifactsOutput`. ## Breaking Changes * `DotNetCliBuilder` removed `retryFailedBuildWithNoDeps` constructor option. * `DotNetCliCommand` removed `RetryFailedBuildWithNoDeps` property and `BuildNoRestoreNoDependencies()` and `PublishNoBuildAndNoRestore()` methods (replaced with `PublishNoRestore()`). Commits viewable in [compare view](dotnet/BenchmarkDotNet@v0.13.12...v0.15.8). </details> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

This adds a new project called BenchmarkDotNet.Exporters.Plotting.
Currently just supports Bar Plots with configurable options for Width, Height and whether to RotateLabels (for large bar labels defaults to true).
Example Bar Plot:

NOTE: Currently build gets an error about strong naming?