1.1.0: Standardization, reporting mode, extension of visualizations, and many enhancements

This major release introduces substantial updates to further standardize and extend this software for evaluation of auto-tuning algorithms, in particular by the following:

• Visualization and reporting of results are modular, scores can be obtained programmatically using report_experiments.
• New benchmark_hub repository and submodule hosting brute-forced benchmarking resources.
• Extended and improved visualizations:
  • Heatmaps for comparison per search space and / or over time
  • Head-to-head plots for direct comparisons on practical impact (beta)
  • New color handling based on existing color maps
• Improved Experiments Schema:
  • Strategy grouping and coloring
  • Extended visualization settings such as visual minima and maxima
• Improvements to consistency:
  • Improved and expanded tests
  • File handling modes explicitly define encodings for consistent behavior
  • Ensuring correct aggregation data is used
• Further improvements and changes:
  • Switched to NumPy 2.0
  • Python 3.12 and 3.13 support, 3.9 was dropped
  • Auto-retry on missing data, smart cutoff handling, better error messages
  • Execution Engine Enhancements
  • Time-based cutoffs and runtime conversion
  • Flexible support for optimization direction, cutoffs, objectives, and valid result thresholds
  • Reformatted code with Ruff, improved test coverage
  • Implemented a new baseline strategy execution framework (ExecutedStrategyBaseline)
  • Implemented automatic hiding of strategies used as an executed baseline
  • Updated other plot types to respect the ‘hide’ key for strategies
  • Introduced get_colors function for strategy grouping and improved color assignment

In addition, there are various bug fixes and minor improvements.
For more details, refer to the full changelog: 1.0.0...1.1.0

1.0.0: release with article publication

With the article published this package receives its first stable release.

Full Changelog: 1.0.0b5...1.0.0

1.0.0 beta 5:

Added interface support for setting cutoff_type to time and setting the cutoff_margin for individual strategies.

Full Changelog: 1.0.0b4...1.0.0b5

1.0.0 beta 4: added CFF

This minor update allows the software to be cited with the Citation File Format (CFF).

Full Changelog: 1.0.0b3...1.0.0b4

1.0.0 beta 3: finalization

Finalization before stable release.

Full Changelog: 1.0.0b2...1.0.0b3

1.0.0 beta 2: improvements to documentation

This second beta release brings several improvements to the documentation, most importantly:

• "Contributing" section has been added
• "Getting Started" section has been expanded
• Inheritance diagrams are rendered correctly

Full Changelog: 1.0.0b1...1.0.0b2

1.0.0 beta 1: features & improvements

This pre-release version brings several new features, bug fixes and updates to the documentation in preparation of the first full release.
Most notable changes:

• Support for imported KTT runs
• Histogram plotting
• New statistics functions in searchspace statistics
• Improved smoothing of curves
• Bug fixes for the confidence shades of fictional regions
• A "getting started" section was added to the documentation
• Documentation contains generated inheritence diagrams and has been made more readable

Full Changelog: 0.1.0b1...1.0.0b1

0.1.0 beta 1: initial pre-release

This first released version is in beta.