[Feature] Evaluate using Profile-Guided Optimization (PGO) and LLVM BOLT

[zamazan4ik]

🚀 Feature

Optimize Leo tools (like the compiler) with Profile-Guided Optimization (PGO) and LLVM BOLT.

Motivation

Deliver better performance for the Leo ecosystem. It would be useful for CI/CD pipelines (faster pipelines == cheaper pipelines) and better developer experience (faster write-compile-test feedback loop).

Implementation

Recently I checked Profile-Guided Optimization (PGO) improvements on multiple projects. The results are here. E.g. PGO results for LLVM-related tooling are here. According to the tests, PGO usually helps with the compiler and compiler-like workloads (like static analysis) - e.g. Clang gets +20% compilation speed with PGO, CPython achieves +10% in pybench. So I think optimizing Leo tooling with PGO would be a good idea at least to try.

I can suggest the following action points:

• Perform PGO benchmarks on the Leo tooling. If it shows improvements - add a note to the documentation about possible improvements in Leo's performance with PGO.
• Providing an easier way (e.g. a build option) to build scripts with PGO can be helpful for the end-users and maintainers since they will be able to optimize the Leo tools according to their own workloads.
• Optimize pre-built binaries with PGO

Testing Post-Link Optimization techniques (like LLVM BOLT) would be interesting too (CPython, Clang, and Rustc already use BOLT as an addition to PGO) but I recommend starting from the usual PGO.

For the Rust projects, I recommend starting with cargo-pgo.

Here are some examples of how PGO optimization is integrated in other projects:

• ISPC: CMake scipts
• Rustc: a CI script for the multi-stage build
• GCC:
  • Official docs, section "Building with profile feedback" (even AutoFDO build is supported)
  • A part in a "wonderful" configure script
• Clang: Docs
• Python:
  • CPython: README
  • Pyston: README
• Go: Bash script
• V8: Bazel flag
• ChakraCore: Scripts
• Chromium: Script
• Firefox: Docs
  • Thunderbird has PGO support too
• PHP - Makefile command and old Centminmod scripts
• MySQL: CMake script
• YugabyteDB: GitHub commit
• FoundationDB: Script
• Zstd: Makefile
• Foot: Scripts
• Windows Terminal: GitHub PR
• Pydantic-core: GitHub PR
• file.d: GitHub PR
• OceanBase: CMake flag
• NodeJS: Configure script

Are you willing to open a pull request? (See CONTRIBUTING)

Not yet.

How the Leo compiler performance is critical for the project - I don't know right now. If it's the problem right now - we could leave the issue open for the future improvements.