Skip to content

[ENH]: Addition of initial ASV benchmarking suite with synthetic data#503

Open
KaranSinghDev wants to merge 1 commit intodswah:mainfrom
KaranSinghDev:feature/add-asv-benchmarks
Open

[ENH]: Addition of initial ASV benchmarking suite with synthetic data#503
KaranSinghDev wants to merge 1 commit intodswah:mainfrom
KaranSinghDev:feature/add-asv-benchmarks

Conversation

@KaranSinghDev
Copy link

@KaranSinghDev KaranSinghDev commented Mar 8, 2026

Fixes #471

This PR implements a formal benchmarking infrastructure using Airspeed Velocity (asv). This fulfills the long-standing request mentioned in #99 to provide a mechanism for tracking performance history and preventing speed regressions.

Implementation

  1. Infrastructure: Added asv.conf.json configured for isolated virtualenv builds and direct pip installation for cross-platform stability.
  2. Synthetic Data Strategy: Benchmarks generate data in-memory using numpy to avoid dependencies on pandas or external CSV files and ensures we measure algorithmic throughput rather than disk I/O speed.
  3. Thread Isolation: Explicitly set OMP_NUM_THREADS=1 and MKL_NUM_THREADS=1 to ensure timing remains consistent across different hardware configurations.
  4. Initial Suites:
    • LinearGAMFit: Measures fitting, prediction, and gridsearch latency.
    • PoissonGAMFit: Benchmarks the performance of the iterative PIRLS loop using a stable mathematical signal to ensure rapid convergence and prevent timeouts.

Verification

I have thoroughly verified in an isolated environment. All benchmarks execute in under 300ms, ensuring the suite is fast enough for CI/CD integration.

Sample Local Output:

· Discovering benchmarks
· Running 4 total benchmarks (1 commits * 1 environments * 4 benchmarks)
[ 62.50%] ··· bench_fit.LinearGAMFit.time_fit           83.3±10ms
[ 75.00%] ··· bench_fit.LinearGAMFit.time_gridsearch    280±200ms
[ 87.50%] ··· bench_fit.LinearGAMFit.time_predict       17.8±4ms
[100.00%] ··· bench_fit.PoissonGAMFit.time_fit         202±100ms

The tests have been done on Python 3.10/3/13. Linux (Ubuntu), with CPU: i7 12700h.

@KaranSinghDev KaranSinghDev force-pushed the feature/add-asv-benchmarks branch from 780ec18 to 6ef3213 Compare March 8, 2026 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ENH] Implement automated benchmarking suite

1 participant