Skip to content

Add MKL backends for numpy.fft and scipy.fft and minor fixes - #22

Merged
jharlow-intel merged 4 commits into
IntelPython:masterfrom
vchamarthi:mkl-backend-fixes
Sep 29, 2026
Merged

jharlow-intel merged 4 commits into
IntelPython:masterfrom
vchamarthi:mkl-backend-fixes

Conversation

@vchamarthi

@vchamarthi vchamarthi commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Summary

On the current conda stack, NumPy is the stock conda-forge build, so fft_bench.py measured pocketfft for both numpy.fft and scipy.fft even in an environment with mkl_fft installed. The README environment recipe also resolved to an older oneMKL and mkl_fft generation. This PR adds explicit MKL backend options, records the library versions with every run, and updates the README.

Changes

  • fft_bench.py
    • Add --scipy-backend {stock,mkl}. mkl runs the timed region inside scipy.fft.set_backend(mkl_fft.interfaces.scipy_fft). Rows are tagged python-scipy-stock or python-scipy-mkl.
    • Add --numpy-backend {stock,mkl}. mkl routes numpy.fft through mkl_fft.patch_numpy_fft(). Rows are tagged python-numpy-mkl.
    • Both options exit with an error if mkl_fft is missing or the routing does not take effect.
    • Fix the shape check (len(shape) < 0 could never be true).
    • Use numpy.random.default_rng for input generation.
  • perf.py
    • Print NumPy, mkl_fft and oneMKL versions in the TAG: lines.
    • Add type hints, use a NamedTuple timer, and move environment printing into print_environment_info().
  • fft_bench.c
    • Fix the inverted assertion on the real-input, non-rfft path (ndims != 1 on a path that only runs for 1-D shapes). It was hidden by -DNDEBUG.
    • Add the missing break after case 't'.
  • win_compile_all.bat: use CORE-AVX512 instead of COMMON-AVX512.
  • README.md
    • The environment recipe now installs mkl_fft and mkl-service with --override-channels, so it resolves to the current oneMKL and mkl_fft releases.
    • Document --numpy-backend and --scipy-backend, with examples.
    • Correct the description of the printed measurements: each measurement is printed, not aggregated.
    • Explain what -c changes in the native benchmark timing.
    • Add a "Comparing platforms" section.

Test plan

Linux (Xeon Gold 6338, socket 0 pinned), before = master 298491f, after = this PR. Python env: Intel channel mkl_fft 2.3.2, MKL 2026.1.0, conda-forge NumPy. Native: icx 2026.1.1, MKL 2026.1.0. Timings are medians of 5 samples over 3 interleaved rounds.

  • --numpy-backend mkl / --scipy-backend mkl rejected by master, accepted by this PR
  • numpy.fft c128 1024, T=1: master 13.5 us, PR default 13.5 us, --numpy-backend mkl 4.2 us
  • scipy.fft c128 1024, T=1: master 10.3 us, PR default 10.3 us, --scipy-backend mkl 6.7 us
  • 512x512 c128 T=1 and T=4 with both MKL backends; T=4 is about 1.8x faster than T=1
  • Default output: same header and numpy.fft rows; scipy.fft prefix is now python-scipy-stock
  • Both options exit with a clear error when mkl_fft is not installed
  • TAG: lines report NumPy, mkl_fft and oneMKL versions, None when absent
  • Native NDEBUG builds match master (c128 1024, 512x512, f64 rfft 1024 with -c)
  • Native assert-enabled build: real 1-D input aborts on master, passes on this PR
  • README environment recipe resolves to current mkl_fft and oneMKL; all 8 README commands run as written, including the -P -r example from -P flag appears to be broken in example README command #18
  • Native c128 1024, T=1: 18.8 us without -c, 2.7 us with -c
  • win_compile_all.bat not tested (no Windows build environment)

- fft_bench.py: add --numpy-backend {stock,mkl}. "mkl" routes numpy.fft
  through mkl_fft.patch_numpy_fft() and exits with an error if mkl_fft is
  missing or the patch does not take. Rows are tagged python-numpy-mkl.
- perf.py: print numpy, mkl_fft and oneMKL versions in the TAG lines.
- fft_bench.c: fix inverted assertion on the real-input, non-rfft path,
  which only runs for 1-D shapes.
- README.md: install mkl_fft and mkl-service in the Python environment,
  document --numpy-backend and --scipy-backend, correct the description
  of the printed measurements, explain descriptor caching with -c, and
  add guidance for comparing platforms.
jharlow-intel
jharlow-intel previously approved these changes Sep 28, 2026

@jharlow-intel jharlow-intel left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think approving this sooner rather than later is important, LGTM

Skip patch_numpy_fft() when the installed NumPy already routes numpy.fft
to mkl_fft, and exit with an error when mkl_fft has no patch_numpy_fft().
Replace a non-ASCII dash in a comment.
@vchamarthi

Copy link
Copy Markdown
Contributor Author

I think approving this sooner rather than later is important, LGTM

Fully tested this PR and updated in description.

@jharlow-intel jharlow-intel left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

super appreciate your thorough testing, lgtm

@jharlow-intel
jharlow-intel merged commit 983cc7f into IntelPython:master Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants