v1.5.1

v1.5.1 Changes

• Updated CMSIS-NN to v8.0.0 and aligned Arm runner dependencies.
• Added the public get_thread_count() API.
• Expanded Arm support for symbolic shapes and dynamic convolution and pooling.
• Folded redundant Q/DQ pairs in the shared EXIR to_edge lowering path.

Highlights

---

• Expanded the Python wheels into a C++ SDK, with linkable libraries for the
  CUDA, Core ML, MLX, OpenVINO, and Qualcomm delegates and packaged TorchAO
  kernels.
• Expanded LLM serving with multi-method export, batched request scheduling,
  bounded cancellation, off-graph KV-cache layouts, and new Qwen3.5 MoE, Muse
  Glimmer, Supertonic, and Voxtral workflows.
• Added ROCm support to the AOTI CUDA backend, including packed W4 execution
  and end-to-end Voxtral Realtime coverage.
• Expanded Arm and Cortex-M support with explicit-layout lowering and kernels,
  additional Ethos-U targets and demos, broader VGF profiling, and more model
  and operator coverage.
• Expanded Qualcomm LLM
  quantization-aware distillation, additional operators, Windows guidance, and
  stronger validation and error handling.
• Improved Vulkan quantized execution, dispatch safety, external-memory
  integration, and operator coverage.

Runtime and Core

---

• Added multi-method model support to the export pipeline and per-backend load
  options.
• Added capacity-aware memory planning over target memory maps and support for
  shared allocations with offsets.
• Added schema-version checks for PTE and PTD files and clearer failures for
  invalid device-planned copies and missing delegate data.
• Added shared pre-decomposition and graph-transform hooks, including common
  view/permute propagation and reinplace passes.
• Improved constant propagation, symbolic-shape handling, and device placement
  for memory-planned buffers.

Packaging and Build

---

• Added a C++ SDK to the Python wheel and split the runtime from the Python
  bindings extension.
• Added CUDA wheel publishing on Pytorch.org
• Improved delegate/runtime bundling in the published wheels.
• Added packaged TorchAO and quantized-kernel libraries.
• Added Windows ARM64 build coverage.
• Reduced wheel size and improved portability by stripping local macOS symbols,
  removing unreachable vendored sources, and dropping unusable Linux MKL
  search paths.
• Lowered the supported CMake floor so CMake 3.26 through 3.28 builds work
  again.

Backends

---

Arm

• Added opt-in Cortex-M explicit-layout lowering with convolution and pooling kernels.
• Added Arm export-recipe support, a pre-decomposition partitioner pipeline,
  and broader symbolic-shape and dim-order handling.
• Added Ethos-U65 support, Ethos-U85 KV-cache export, and SmolLM2,
  TinyStories-42M, Silero VAD, MobileSAM, and Stable Diffusion workflows.
• Expanded VGF profiling and environment preflight validation tools, neural statistics, automatic operator-support generation, support for int8 KV cache, FP16,
  BF16, MXFP8, and quantized model coverage.
• Improved Cortex-M memory planning, CMSIS-Pack coverage, in-place activation
  fusion, and arithmetic, pooling, and activation lowering.

Qualcomm

• Added LLM quantization-aware distillation and Gemma 4 text-model enablement.
• Added support for pdist, sort, reflection_pad3d, as_strided,
  scatter_reduce, scatter_add, and Hadamard-transform flows.
• Added QNN-on-Windows guidance and improved SDK setup, import behavior,
  delegate bounds checks, graph-I/O validation, and quantized metadata
  preservation.

CUDA and ROCm

• Added opt-in ROCm support to the AOTI backend, packed W4 execution, AMD
  pointwise examples, and Voxtral Realtime coverage on gfx1100 and gfx950.
• Added shared-memory targeting for cross-architecture CUDA inference and
  INT4, INT5, and INT6 short-query kernels.
• Improved CUDA delegate integration with per-thread streams, a warm memory
  pool, cross-method weight sharing, and clearer missing-weight errors.

Core ML and MLX

• Added linkable Core ML and MLX libraries to Apple frameworks and SwiftPM,
  plus ETDump profiling support.
• Added MLX off-graph KV-cache flat, ring, and cell layouts with shared pools
  and cross-thread execution fixes.
• Expanded MLX lowering with fused attention, native_group_norm,
  upsample_nearest2d, flip, and trunc support.
• Added Metal addmm and split_copy support and improved Core ML handling of
  tied embeddings and causal attention masks.

Vulkan

• Added external image and buffer wrapping with caller-provided dimensions.
• Added device-safe dispatch geometry, two-dimensional dispatch helpers, and
  quantized convolution paths using unsigned dot products.
• Added payload-bounded constant sharding, scalar comparisons, and broader
  arange, clamp, indexing, and batch support.

NXP

• Upgraded the eIQ Neutron SDK to 3.2.1 and added an MCUXpresso example build.
• Added Neutron support for rsqrt, Hardswish, softmax, reflected padding,
  scalar elementary arguments, and AdaptiveAvgPool1d.
• Added Neutron operator-suite and MLPerf Tiny classification coverage.

XNNPACK

• Added BF16 fully connected delegation and BF16 activation support in LLM
  Export.
• Bumped the XNNPACK pin to 92a7ad501b9516f9fecae119e0146dd1f58e54c1 to support the above
• Improved dynamic-quantized
  channels-last handling.
• Fixed pooling with default strides and single-element kernels, convolution
  dimensionality inference, and shared-observer handling for slices.

Operators and Kernels

---

• Added a quantized MoE feed-forward operator with INT4 packing, fused
  projections, source transformations, and numerical coverage.
• Added a quantized stacked-halves RoPE operator and BF16-by-FP32 GEMV decode
  support.
• Optimized BF16 SDPA and fixed NaNs in CUDA split-K attention for Voxtral.
• Added faster contiguous reductions and fixed non-contiguous reduction bounds,
  broadcast rank mismatches, and dim-order validation for several operators.

Notable Fixes

---

• Fixed CUDA low-memory external-weight export, model weight lookup, stream
  selection, and cross-method AOTI library collisions.
• Fixed QNN crashes on AMD hosts, invalid graph I/O, stale SDK bundles, and
  setup side effects during import.
• Fixed Arm convolution, pooling, rescale, grid-sampling, symbolic-shape, and
  quantization edge cases across TOSA, VGF, and Ethos-U.
• Fixed Vulkan delegation beyond supported graph regions and several dispatch,
  resize, and quantization synchronization issues.
• Fixed XNNPACK pooling, channels-last, placeholder-renaming, and BF16 export
  issues.
• Fixed Core ML and MLX attention, thread-affinity, and build-integration
  issues.

Contributors

We welcome 18 first-time contributors to ExecuTorch in this release:
@denizkilinc, @yenhao-huang, @Kilbex, @ruanslv, @yuweix94, @slipstr34m,
@AxelNoun, @adityasingh2400, @rstehle, @sid-rp, @malfet, @cthotti, @kkch3ng,
@cgreenberg, @bharath-123, @ricardoasouz, @synath, @LuozhuZhang

Full Changelog

v1.4.0...v1.5.0

v1.5.0

Highlights

---

• Expanded the Python wheels into a C++ SDK, with linkable libraries for the
  CUDA, Core ML, MLX, OpenVINO, and Qualcomm delegates and packaged TorchAO
  kernels.
• Expanded LLM serving with multi-method export, batched request scheduling,
  bounded cancellation, off-graph KV-cache layouts, and new Qwen3.5 MoE, Muse
  Glimmer, Supertonic, and Voxtral workflows.
• Added ROCm support to the AOTI CUDA backend, including packed W4 execution
  and end-to-end Voxtral Realtime coverage.
• Expanded Arm and Cortex-M support with explicit-layout lowering and kernels,
  additional Ethos-U targets and demos, broader VGF profiling, and more model
  and operator coverage.
• Expanded Qualcomm LLM
  quantization-aware distillation, additional operators, Windows guidance, and
  stronger validation and error handling.
• Improved Vulkan quantized execution, dispatch safety, external-memory
  integration, and operator coverage.

Runtime and Core

---

• Added multi-method model support to the export pipeline and per-backend load
  options.
• Added capacity-aware memory planning over target memory maps and support for
  shared allocations with offsets.
• Added schema-version checks for PTE and PTD files and clearer failures for
  invalid device-planned copies and missing delegate data.
• Added shared pre-decomposition and graph-transform hooks, including common
  view/permute propagation and reinplace passes.
• Improved constant propagation, symbolic-shape handling, and device placement
  for memory-planned buffers.

Packaging and Build

---

• Added a C++ SDK to the Python wheel and split the runtime from the Python
  bindings extension.
• Added CUDA wheel publishing on Pytorch.org
• Improved delegate/runtime bundling in the published wheels.
• Added packaged TorchAO and quantized-kernel libraries.
• Added Windows ARM64 build coverage.
• Reduced wheel size and improved portability by stripping local macOS symbols,
  removing unreachable vendored sources, and dropping unusable Linux MKL
  search paths.
• Lowered the supported CMake floor so CMake 3.26 through 3.28 builds work
  again.

Backends

---

Arm

• Added opt-in Cortex-M explicit-layout lowering with convolution and pooling kernels.
• Added Arm export-recipe support, a pre-decomposition partitioner pipeline,
  and broader symbolic-shape and dim-order handling.
• Added Ethos-U65 support, Ethos-U85 KV-cache export, and SmolLM2,
  TinyStories-42M, Silero VAD, MobileSAM, and Stable Diffusion workflows.
• Expanded VGF profiling and environment preflight validation tools, neural statistics, automatic operator-support generation, support for int8 KV cache, FP16,
  BF16, MXFP8, and quantized model coverage.
• Improved Cortex-M memory planning, CMSIS-Pack coverage, in-place activation
  fusion, and arithmetic, pooling, and activation lowering.

Qualcomm

• Added LLM quantization-aware distillation and Gemma 4 text-model enablement.
• Added support for pdist, sort, reflection_pad3d, as_strided,
  scatter_reduce, scatter_add, and Hadamard-transform flows.
• Added QNN-on-Windows guidance and improved SDK setup, import behavior,
  delegate bounds checks, graph-I/O validation, and quantized metadata
  preservation.

CUDA and ROCm

• Added opt-in ROCm support to the AOTI backend, packed W4 execution, AMD
  pointwise examples, and Voxtral Realtime coverage on gfx1100 and gfx950.
• Added shared-memory targeting for cross-architecture CUDA inference and
  INT4, INT5, and INT6 short-query kernels.
• Improved CUDA delegate integration with per-thread streams, a warm memory
  pool, cross-method weight sharing, and clearer missing-weight errors.

Core ML and MLX

• Added linkable Core ML and MLX libraries to Apple frameworks and SwiftPM,
  plus ETDump profiling support.
• Added MLX off-graph KV-cache flat, ring, and cell layouts with shared pools
  and cross-thread execution fixes.
• Expanded MLX lowering with fused attention, native_group_norm,
  upsample_nearest2d, flip, and trunc support.
• Added Metal addmm and split_copy support and improved Core ML handling of
  tied embeddings and causal attention masks.

Vulkan

• Added external image and buffer wrapping with caller-provided dimensions.
• Added device-safe dispatch geometry, two-dimensional dispatch helpers, and
  quantized convolution paths using unsigned dot products.
• Added payload-bounded constant sharding, scalar comparisons, and broader
  arange, clamp, indexing, and batch support.

NXP

• Upgraded the eIQ Neutron SDK to 3.2.1 and added an MCUXpresso example build.
• Added Neutron support for rsqrt, Hardswish, softmax, reflected padding,
  scalar elementary arguments, and AdaptiveAvgPool1d.
• Added Neutron operator-suite and MLPerf Tiny classification coverage.

XNNPACK

• Added BF16 fully connected delegation and BF16 activation support in LLM
  Export.
• Bumped the XNNPACK pin to 92a7ad501b9516f9fecae119e0146dd1f58e54c1 to support the above
• Improved dynamic-quantized
  channels-last handling.
• Fixed pooling with default strides and single-element kernels, convolution
  dimensionality inference, and shared-observer handling for slices.

Operators and Kernels

---

• Added a quantized MoE feed-forward operator with INT4 packing, fused
  projections, source transformations, and numerical coverage.
• Added a quantized stacked-halves RoPE operator and BF16-by-FP32 GEMV decode
  support.
• Optimized BF16 SDPA and fixed NaNs in CUDA split-K attention for Voxtral.
• Added faster contiguous reductions and fixed non-contiguous reduction bounds,
  broadcast rank mismatches, and dim-order validation for several operators.

Notable Fixes

---

• Fixed CUDA low-memory external-weight export, model weight lookup, stream
  selection, and cross-method AOTI library collisions.
• Fixed QNN crashes on AMD hosts, invalid graph I/O, stale SDK bundles, and
  setup side effects during import.
• Fixed Arm convolution, pooling, rescale, grid-sampling, symbolic-shape, and
  quantization edge cases across TOSA, VGF, and Ethos-U.
• Fixed Vulkan delegation beyond supported graph regions and several dispatch,
  resize, and quantization synchronization issues.
• Fixed XNNPACK pooling, channels-last, placeholder-renaming, and BF16 export
  issues.
• Fixed Core ML and MLX attention, thread-affinity, and build-integration
  issues.

Contributors

We welcome 18 first-time contributors to ExecuTorch in this release:
@denizkilinc, @yenhao-huang, @Kilbex, @ruanslv, @yuweix94, @slipstr34m,
@AxelNoun, @adityasingh2400, @rstehle, @sid-rp, @malfet, @cthotti, @kkch3ng,
@cgreenberg, @bharath-123, @ricardoasouz, @synath, @LuozhuZhang

Full Changelog

v1.4.0...v1.5.0

v1.4.1

1.4.1 Changes

Fixed an issue with incorrect baked in paths within the wheels.
Fixed an issue with XNNPack's PreLU in quantized graphs.

1.4 Highlights

---

• Expanded Cortex-M support across additional CPU variants, model tests,
  scratch-buffer planning, CMSIS-NN integration, and CMSIS Pack builds.
• Added broad Arm backend coverage for TOSA dialect lowering, dynamic shapes,
  FP8/MXFP/BFloat16 paths, VGF profiling, and LLM export workflows.
• Added Vulkan support for Linux and Windows desktop GPUs, opt-in wheel builds,
  automatic build enablement, and a functional backend availability check.
• Expanded Qualcomm QNN support with 2-bit 16a2w quantization, faster VLM/LLM
  PTQ flows, multi-batch quantization and evaluation, LPAI tooling, profiling,
  and additional operators.
• NXP Backend finished MLIR based Neutron Converter flow rollout - all operators using the MLIR flow, Neutron backend now support 48 ATen operators.
• Continued Android API migration to Kotlin and added Android model E2E
  coverage and Kotlin examples.

Runtime and Core

---

• Removed the legacy _use_edge_ops toggles from the runtime/export path.
• Added explicit dim-order legalization for channels-last portable fallbacks.
• Hardened tensor-layout validation and segment loading against null metadata.
• Improved symbolic input metadata preservation during exported-program pass
  replay.
• Added direct bundled-program serialization improvements for embedded flows.

Backends

---

Arm and Cortex-M

• Added builds for more Cortex-M variants on Corstone-300, including M0+.
• Added AoT scratch-buffer planning and broader MLPerf Tiny model coverage.
• Expanded TOSA dialect lowering for activation, binary, conversion, data
  layout, FFT, reduction, shape, and unary operators.
• Added or improved VGF support for profiling, shader segments, BFloat16,
  MXFP, grid sampling, and persistent IO mappings.
• Ethos-U and VGF dependencies can now be installed with ExecuTorch using optional packages “pip install executorch[ethos_u,vgf]”
• Added SmolLM2 and Qwen3-VL export and evaluation coverage.
• Added CMSIS Pack infrastructure and an ExecuTorch 1.4 public API manifest.

Qualcomm

• Added additional QNN operators, including addmm, select_scatter, variance,
  diagonal, fill, cdist, LayerNorm variants, and hyperbolic functions.
• Added 2-bit weight quantization support and multi-batch LLM quantization.
• Decoupled calibration, quantization, compilation, and evaluation stages for
  faster LLM and VLM workflows.
• Added LPAI partitioning, tensor dumping, and runtime heap profiling.

NXP

• Updated the Neutron software stack to 3.1.3.
• Added backend profiling and MLPerf Tiny profiling tests.
• Expanded Neutron operator coverage and removed several broadcasting
  restrictions.
• Fixed partition-output handling, format inference, BatchNorm fusion, and
  multiprocessing conversion issues.

CoreML and MLX

• Added a Core ML compute-plan reporting tool for ANE, GPU, and CPU dispatch.
• Added MLX handlers for hardtanh, bitwise_or, and leaky_relu.
• Improved MLX submodule build isolation.

Vulkan

• Added Vulkan desktop support for Linux and Windows and opt-in wheel builds.

XNNPack

• Added quantized same-padding convolution support in XNNPACK, including fixes
  for even Conv1d kernels and ViT concatenation quantization parameters.
• Expanded RISC-V XNNPACK coverage across multiple RVV QEMU configurations.

Operators and Kernels

---

• Added BFloat16 and FP16 activation support to CPU SDPA.
• Added BFloat16-output quantized embeddings on CPU.
• Improved numerical accuracy by accumulating Half and BFloat16 softmax,
  log-softmax, mean, and sum operations in FP32.
• Fixed portable anti-aliased bilinear upsampling correctness.

LLM, Mobile, and Examples

---

• Added grammar and grammar_type fields to GenerationConfig.
• Added an unoptimized ExecuTorch runner example for Espressif ESP32.
• Migrated Android Module, TrainingModule, SGD, and LLM extension APIs from
  Java to Kotlin.
• Added Kotlin examples and Android model end-to-end testing.

Notable Fixes

---

• Fixed channels-last handling for portable fallbacks and Cortex-M DS-CNN.
• Fixed a QNN runtime null dereference when op-package options are absent.
• Fixed quantized Conv1d same padding for even kernels in XNNPACK.
• Fixed several Arm partitioning, dynamic-shape, quantization, and runner
  integration issues.
• Fixed multiple NXP Neutron partition, format, and conversion issues.

Contributors

We welcome 29 first-time human contributors to ExecuTorch in this release:
@beomwookang, @karan1508, @atharvjairath, @devin-lai, @derekdixu, @jpiat, @doggeral, @arnavnagzirkar, @wirthual, @youxie, @nanookclaw, @blood-orange, @uddeshsingh, @johnny90, @KushalAdhvaryu, @MatthiasHertelArm, @mohankumarkumar, @yctwo, @jansel, @nlutsenko, @Reubend, @goutamadwant, @JaynouOliver, @RanjithRagavan, @arkadip-maitra, @durvesh1992, @corey-cole, @SakshamKapoor2911, @iamorlando

Full Changelog

v1.3.1...v1.4.0

v1.4

Highlights

---

• Expanded Cortex-M support across additional CPU variants, model tests,
  scratch-buffer planning, CMSIS-NN integration, and CMSIS Pack builds.
• Added broad Arm backend coverage for TOSA dialect lowering, dynamic shapes,
  FP8/MXFP/BFloat16 paths, VGF profiling, and LLM export workflows.
• Added Vulkan support for Linux and Windows desktop GPUs, opt-in wheel builds,
  automatic build enablement, and a functional backend availability check.
• Expanded Qualcomm QNN support with 2-bit 16a2w quantization, faster VLM/LLM
  PTQ flows, multi-batch quantization and evaluation, LPAI tooling, profiling,
  and additional operators.
• NXP Backend finished MLIR based Neutron Converter flow rollout - all operators using the MLIR flow, Neutron backend now support 48 ATen operators.
• Continued Android API migration to Kotlin and added Android model E2E
  coverage and Kotlin examples.

Runtime and Core

---

• Removed the legacy _use_edge_ops toggles from the runtime/export path.
• Added explicit dim-order legalization for channels-last portable fallbacks.
• Hardened tensor-layout validation and segment loading against null metadata.
• Improved symbolic input metadata preservation during exported-program pass
  replay.
• Added direct bundled-program serialization improvements for embedded flows.

Backends

---

Arm and Cortex-M

• Added builds for more Cortex-M variants on Corstone-300, including M0+.
• Added AoT scratch-buffer planning and broader MLPerf Tiny model coverage.
• Expanded TOSA dialect lowering for activation, binary, conversion, data
  layout, FFT, reduction, shape, and unary operators.
• Added or improved VGF support for profiling, shader segments, BFloat16,
  MXFP, grid sampling, and persistent IO mappings.
• Ethos-U and VGF dependencies can now be installed with ExecuTorch using optional packages “pip install executorch[ethos_u,vgf]”
• Added SmolLM2 and Qwen3-VL export and evaluation coverage.
• Added CMSIS Pack infrastructure and an ExecuTorch 1.4 public API manifest.

Qualcomm

• Added additional QNN operators, including addmm, select_scatter, variance,
  diagonal, fill, cdist, LayerNorm variants, and hyperbolic functions.
• Added 2-bit weight quantization support and multi-batch LLM quantization.
• Decoupled calibration, quantization, compilation, and evaluation stages for
  faster LLM and VLM workflows.
• Added LPAI partitioning, tensor dumping, and runtime heap profiling.

NXP

• Updated the Neutron software stack to 3.1.3.
• Added backend profiling and MLPerf Tiny profiling tests.
• Expanded Neutron operator coverage and removed several broadcasting
  restrictions.
• Fixed partition-output handling, format inference, BatchNorm fusion, and
  multiprocessing conversion issues.

CoreML and MLX

• Added a Core ML compute-plan reporting tool for ANE, GPU, and CPU dispatch.
• Added MLX handlers for hardtanh, bitwise_or, and leaky_relu.
• Improved MLX submodule build isolation.

Vulkan

• Added Vulkan desktop support for Linux and Windows and opt-in wheel builds.

XNNPack

• Added quantized same-padding convolution support in XNNPACK, including fixes
  for even Conv1d kernels and ViT concatenation quantization parameters.
• Expanded RISC-V XNNPACK coverage across multiple RVV QEMU configurations.

Operators and Kernels

---

• Added BFloat16 and FP16 activation support to CPU SDPA.
• Added BFloat16-output quantized embeddings on CPU.
• Improved numerical accuracy by accumulating Half and BFloat16 softmax,
  log-softmax, mean, and sum operations in FP32.
• Fixed portable anti-aliased bilinear upsampling correctness.

LLM, Mobile, and Examples

---

• Added grammar and grammar_type fields to GenerationConfig.
• Added an unoptimized ExecuTorch runner example for Espressif ESP32.
• Migrated Android Module, TrainingModule, SGD, and LLM extension APIs from
  Java to Kotlin.
• Added Kotlin examples and Android model end-to-end testing.

Notable Fixes

---

• Fixed channels-last handling for portable fallbacks and Cortex-M DS-CNN.
• Fixed a QNN runtime null dereference when op-package options are absent.
• Fixed quantized Conv1d same padding for even kernels in XNNPACK.
• Fixed several Arm partitioning, dynamic-shape, quantization, and runner
  integration issues.
• Fixed multiple NXP Neutron partition, format, and conversion issues.

Contributors

We welcome 29 first-time human contributors to ExecuTorch in this release:
@beomwookang, @karan1508, @atharvjairath, @devin-lai, @derekdixu, @jpiat, @doggeral, @arnavnagzirkar, @wirthual, @youxie, @nanookclaw, @blood-orange, @uddeshsingh, @johnny90, @KushalAdhvaryu, @MatthiasHertelArm, @mohankumarkumar, @yctwo, @jansel, @nlutsenko, @Reubend, @goutamadwant, @JaynouOliver, @RanjithRagavan, @arkadip-maitra, @durvesh1992, @corey-cole, @SakshamKapoor2911, @iamorlando

Full Changelog

v1.3.1...v1.4.0

v1.3.1

ExecuTorch v1.3.1 Release Notes

1.3.1 is the first broadly published 1.3 patch release, following the Maven-only 1.3.0 release.

Highlights

ExecuTorch v1.3.1 expands model and backend coverage across embedded, mobile, and GPU targets. This release adds major Arm, Cortex-M, VGF, NXP, Qualcomm, CUDA, Metal, MLX, Vulkan, and XNNPACK improvements, and continues broadening LLM and multimodal model support.

• Arm backend (Ethos-U / TOSA / VGF / Cortex-M) - Major expansion in TOSA dialect lowering, quantization support, VGF infrastructure, Cortex-M test coverage, Ethos-U compatibility, and CI reliability.
• New and expanded model support - Qwen3.5 MoE, Gemma 4 31B, LFM2.5, Voxtral Realtime/TTS, Llama4 export support, and additional Android LLM runner documentation.
• CUDA backend - Qwen3.5 MoE export/runtime work, fused MoE and INT4 Triton kernels, CUDA graph capture/replay, GPU-side sampling, and memory/stat reporting.
• Metal and MLX backends - Qwen3.5 MoE Metal path, Gemma 4 31B MLX support, MLX delegate progress, integer op support, top-k fallback, gated delta rule kernels, and hardened runtime code signing.
• Vulkan backend - Cooperative matrix dispatch for linear/matmul, safer memory handling, integer overflow and heap-buffer fixes, improved partitioning, and VGF compatibility work.
• Qualcomm AI Engine Direct (QNN) - Expanded ATen op support, LPAI and Direct Mode work, custom op and quantization annotation APIs, debugging improvements, performance dumping, and heap profiling.
• NXP backend - Continued new Neutron flow rollout, eIQ Neutron SDK 3.1.1, stricter graph verification, QAT coverage, and additional operator support.
• Runtime and packaging - Shared library install support, Android lifecycle/error handling fixes, and improved wheel/build reliability.

New Models and Model Enablement

• Qwen3.5 MoE export and runner support across CUDA and Metal, including fused MoE kernels, INT4 paths, CUDA graph support, and runtime stats.
• Gemma 4 31B export and runner support with CUDA and MLX backend support.
• LFM2.5 export and runner support for MLX.
• Voxtral Realtime and Voxtral TTS improvements, including streaming fixes, CUDA backend support, and documentation updates.
• Android LLM runner documentation and HuggingFace-oriented usage docs.

Core Components

EXIR / Export

• Safer ETRecord/export deserialization.
• FP8 placeholder support in ExecuTorch serialization.
• Fixes for torch.split alias annotations and partition grouping.
• Migration cleanup away from deprecated capture config paths.

Runtime

• Backend and runtime option handling improvements.
• Better contextual error reporting for debuggability.

Kernels and Operators

• Portable adaptive average pool 2D kernel.
• fp32 accumulation for Half/BFloat16 grid sampler bilinear paths.
• Fused quantized add, mul, relu, hardswish, and bmm kernels.
• Quantized max pool and quantized conv1d fixes.
• Additional quantization and observer fixes across portable and backend kernels.

Backend Delegates

Arm / Ethos-U / TOSA / VGF / Cortex-M

• Expanded TOSA op coverage including slice, max pool, avg pool, adaptive avg pool, shape ops, resize, LSTM decomposition, and additional dtype/layout handling.
• FP16/BF16 operator coverage is now considered largely complete; most support landed incrementally over prior releases and the 1.3 cycle adds test stabilization for FP16 MobileNetV3 and VGF bilinear paths.
• Refactored dim-order and permute handling. Removed the legacy tosa_dim_order path, added permute-removal for TOSA ops, and improved permute fusion over elementwise ops. This reduced unnecessary transposes in lowered graphs.
• Improved quantization support, including composable quantizer work, constant folding fixes, TOSA/VGF linear quantization modes, and Cortex-M MVE/Helium int16 quantize/dequantize support.
• VGF build/test infrastructure and documentation.
• Cortex-M docs, TinyML validation, CMSIS-NN integration, linting, and CI improvements.
• Ethos-U driver backwards compatibility and more robust FVP/toolchain download handling.

CUDA

• Qwen3.5 MoE export/runtime path with fused MoE and INT4 Triton kernels.
• Gemma 4 31B support on CUDA.
• CUDA graph capture/replay and GPU-side Gumbel-max sampling.
• Memory-efficient model export.
• Runtime cross-method weight sharing.
• GPU memory tracking and structured runtime stats.
• CI and build reliability fixes.

Metal / MLX

• Qwen3.5 MoE Metal export, build target, source transforms, and integration tests.
• MLX delegate progress and integer op support.
• Gemma 4 31B support through MLX.
• Top-k fallback, gated delta rule kernels, SDPA updates, and hardened runtime code signing.

Vulkan

• Cooperative matrix dispatch for linear/matmul.
• Safer memory allocation and heap-buffer handling.
• Integer overflow fixes and improved partitioning behavior.
• VGF compatibility and Parakeet documentation/build fixes.

Qualcomm AI Engine Direct (QNN)

• New QNN op support including isinf, isnan, rand, randn, log2, log10, log1p, trunc, acos, atan2, tan, avg_pool1d, remainder, reflection_pad, and scatter.src.
• LPAI and Direct Mode support.
• Custom op package and quantization annotation APIs.
• Debugging/numeric discrepancy tooling and performance dump improvements.
• Multimodal and LLM quantization guidance updates.

NXP

• eIQ Neutron SDK updated to version 3.1.1. eIQ Neutron SDK 3.1.x introduced a new MLIR-based conversion flow with broader operator support for Neutron-C architectures.
• Operators migrated to the MLIR flow: aten.avg_pool2d, aten.max_pool2d, aten.abs, aten.constant_pad_nd, aten.sigmoid, aten.leaky_relu, and aten.adaptive_avg_pool2d.
• New ops: 1D MaxPool, aten.bmm, and aten.conv_transpose1.
• Unified test pipeline for operator and model testing.

XNNPACK

• Enable weight cache and workspace sharing as runtime options.
• Output padding support for transposed convolution.
• Workspace sharing and weight cache concurrency fixes.

Platforms

Android

• Improved JNI error reporting and lifecycle handling.
• Tensor.copyDataInto added to the Java API.
• LLM module lifecycle/thread-safety fixes and test coverage.
• Modern Clang and ETDump path override fixes.

Apple / CoreML / iOS

• CoreML profiler crash fix.
• CoreML partitioner skips for unsupported random and argmin/argmax patterns.
• iOS 18 quantization error hints.
• Shared library install support for consumers and backends.

Documentation

• Android LLM runner docs.
• Arm, Cortex-M, Ethos-U, and VGF documentation updates.
• NXP QAT and Neutron flow documentation updates.
• Vulkan, Parakeet, Voxtral, and LFM2.5 documentation updates.

Deprecations

The legacy MPS backend remains deprecated and is expected to be removed in a future release.

Contributors

We welcome 62 first-time human contributors to ExecuTorch in this release:

@s09g, @Froskekongen, @MahinAshraful, @qti-mmadhava, @quic-boyuc, @qti-horodnic, @rezaasjd, @ocvh, @twsl, @mvartani-meta, @ifed-ucsd, @tmtrademarked, @berndporr, @leixin, @amov-meta, @xiaodong705, @Hyungkeun-Park-Nota, @AlannaBurke, @kastopia, @zhaoxul-qti, @VasuAgrawal, @JorickvdHoeven, @dveremeevfb, @alexey-sidnev, @Lidang-Jiang, @notaJiminLee, @irtrukhina, @mvsfb, @SAY-5, @Jah-yee, @fgsiveone, @boomitsnoom, @jeanschmidt, @KevinUW114514, @aksharabhardwaj766-commits, @AlessandroVacca, @gkrulce, @AswaniSahoo, @OpenByteDev, @ishangodawatta, @WongJohnson, @ymrohit, @XAheli, @jiawei-lyu, @Nazim-fad, @yadferhad, @darknight054, @xuyanwen2012, @matt-cossins, @zeel2104, @john-rocky, @luhenry, @christine-long-meta, @hboyraz, @telgamal-1, @madhesh60, @omkar-334, @vacu9708, @Vasanthadithya-mundrathi, @ozgecinko, @wliuyx, @tomeuv

Full Changelog

Full Changelog: v1.2.0...v1.3.1

v1.2.0

Highlights

ExecuTorch v1.2.0 expands on-device AI to more models and more hardware. This release adds real-time speech inference with Voxtral Realtime, promotes Cortex-M to a first-class embedded target, delivers major backend improvements, and reduces binary size for resource-constrained deployments.

Aligned with PyTorch 2.11, TorchAudio 2.11, TorchVision 0.26, TorchCodec 0.11, TorchAO 0.17, and PyTorch-Tokenizers 1.2

New model support - Voxtral Realtime (streaming speech), NVIDIA NeMo Sortformer (speaker diarization), Silero VAD (voice activity detection), Qwen3.5, and Parakeet now exportable and runnable across multiple backends.

Cortex-M as a first-class target - Dedicated backend with CMSIS-NN integration, quantized int8 batch matmul and pad, improved pattern matching, and portable kernel usage for broader operator support.

Metal backend - 4-bit quantized inference via MLX-derived GEMM kernels, native causal SDPA with GQA, GPU buffer pool with LRU eviction, dispatch pipelining for lower latency, and mmap weight prefetching for faster model loading.

CUDA backend - SlimTensor integration for lightweight tensor management, CUDA stream sharing across methods for skip-copy optimization, and pybind integration with automatic CUDA detection.

Vulkan backend - Comprehensive int8 quantized inference with new linear, convolution, and fused operators, layout-flexible shaders, and improved device compatibility including fp16 fallback and Vulkan 1.0 support.

Arm backend (Ethos-U / TOSA) - LLM support via TOSA and Ethos-U backends, VGF image classification flow, Ethos-U SDK 26.02 with Vela 5.0.0, aarch64-linux-musl build support, and new operators including grouped transposed convolutions and boolean attention masks.

Qualcomm AI Engine Direct (QNN) - Backend-awareness quantizer for hardware-targeted quantization, CDSP Direct Mode for lower-latency DSP dispatch, SLC allocator, and attention sink for long context support.

NXP backend - Migration to eIQ Neutron SDK, selective kernel registration for smaller footprint, weight prefetching from external memory to SRAM, QAT support, and 14 new operators.

Faster serialization - New flatbuffer Program serialization in EXIR improves export performance.

Smaller binaries - Compile-time optimizations (.eh_frame suppression, constexpr kernel constructors, log disabling) reduce binary size for embedded targets.

LoRA and multi-method - Structured LoraConfig and MultimethodLoraConfig enable on-device model personalization. Multi-method support is now wired through export, runner, and ETRecord.

MPS backend deprecated - The MPS backend is deprecated and will be removed in a future release (v1.4.0).

---

New Model Support

• Voxtral Realtime - Mistral's real-time speech model with true streaming inference on XNNPACK, Metal, and CUDA. Includes int4 quantization and fp32/bf16 support, live microphone input, and pre-exported .pte files for quick start.
• Qwen3.5 - Export support for 0.8B, 2B, and 4B variants. Also added Qwen2.5-Coder.
• NVIDIA NeMo Parakeet - Now XNNPACK by default, with int4 quantization, Metal int4 via HQQ, bfloat16, and CUDA support.
• NVIDIA NeMo Sortformer - Speaker diarization model on XNNPACK and CUDA.
• Silero VAD - Voice Activity Detection on XNNPACK for speech endpoint detection.

---

Core Components

EXIR / Export

• Flatbuffer Program serialization for faster export and model loading.
• CSE (Common Subexpression Elimination) pass eliminates redundant computations in exported models, reducing model size and improving inference speed. Includes hierarchical infrastructure for nested graph optimization.
• Unified fusion passes across backends for more consistent optimization.
• QAT ConvBN fuse pass and quantize fused ConvBN bias pass for improved quantization-aware training workflows.
• Multi-method support in export and ETRecord for edge dialect programs.
• PTE diff utility for comparing exported programs across versions.
• Various other bugfixes and improvements.

Runtime

• MmapUseMlockIgnoreErrors as default load mode for more robust memory-mapped loading.
• LoadBackendOptionsMap - New API for passing backend-specific configuration (performance hints, memory limits) when loading models, without modifying the exported program.
• Shared state in Module for more efficient multi-method execution.
• MemoryFormatOpsPass fix - Preserves input dim_order for clone/to_copy with no memory_format kwarg.

Kernels & Operators

• Constexpr Kernel and KernelKey constructors for compile-time initialization and smaller binaries.
• Direct function pointers for et_copy_index and et_view in prim_ops, reducing dispatch overhead.
• ARM embedded platform compatibility added to all portable operators.
• Fix for int overflow in Tensor.numel().
• Fix for uninitialized memory in portable batch norm kernel.

Binary Size

• About a 10 - 15kb reduction in core runtime code size depending on toolchain.
• Suppress .eh_frame generation when EXECUTORCH_OPTIMIZE_SIZE is ON.
• Export ET_LOG_ENABLED=0 as public compile definition on CMake targets.
• Constexpr constructors enable compile-time initialization, eliminating static constructor overhead.

SDK / Pybindings

• Pybindings for TextLLMRunner for Python-based LLM inference.
• Custom data loader hooks - allowing you to provide custom dataloaders in Python.
• Fixed and completed pybindings.pyi type stubs for better IDE support.

---

Backend Delegates

Arm (Ethos-U / TOSA)

• Ethos-U SDK 26.02, Vela 5.0.0, FVP version bumps for Corstone-300/320.
• VGF image classification flow with Ethos-U support and end-to-end documentation.
• LLM extension - Added TOSA and Ethos-U backend support for running LLMs on Arm.
• New operator support - aten.erfinv, PAD dialect op, TOSA shape ops, boolean attention masks, grouped transposed convolutions.
• aarch64-linux-musl build support for lightweight Linux deployments.
• Zephyr examples reorganized under zephyr/samples/ with STM Nucleo board example.
• Operator documentation for all supported ops.
• Support added for Ethos-U85 Direct drive from a Linux based Cortex-A.

Cadence/HiFi

• Channel-last conv kernels, im2row and transpose kernel fixes.

CoreML

• Deprecation warning added for to_edge + to_backend workflow.

Cortex-M

• Cortex-M as first-class target in aot_arm_compiler with dedicated documentation.
• Quantized int8 batch matmul and pad via CMSIS-NN.
• Improved PatternMatcher algorithm for more efficient quantizer pattern matching.
• Enabled portable kernel usage to broaden operator support.

CUDA

• SlimTensor integration - Replaced ETensor with SlimTensor as the internal tensor representation for cuda backend for lightweight tensor management and improved memory operations.
• CUDA stream sharing - Added use_shared_cuda_stream to share a single stream across multiple methods (e.g., encoder, decoder, sampler) to ensure proper ordering for skip-copy optimization and removal of unnecessary synchronization.
• Pybind integration - Integrated CUDA backend into the pybind build system with automatic CUDA detection.
• Bug fixes and platform support - Fixed Triton SDPA NaN with sparse boolean masks, output stride mismatch during delegate copy-back, and improved Windows/MinGW cross-compilation support.

Metal

• 4-bit quantized linear - Ported quantized GEMM kernels from MLX for int4 inference.
• Causal SDPA - Native causal scaled dot-product attention with GQA and hoisted mask computation, reducing memory and improving throughput for LLM inference.
• Buffer pool - Best-fit matching with LRU eviction for reduced memory allocation overhead.
• Dispatch pipelining - commitAndContinue overlaps GPU command encoding and execution, reducing latency for multi-op models.
• Linear with bias decompose pass for broader coverage.
• Voxtral Realtime with streaming mode and int4, bf16, and fp32 support.
• Build support for macOS SDK < 15.
• mmap weight prefetching - Prefetches weight blobs to eliminate page fault bottleneck, significantly reducing load time for large models.

NXP

• Neutron Kernel Registration: The Neutron backend now supports selective kernel registration, allowing you to register only the kernels needed for your specific models to decrease the memory footprint of the Neutron kernel library.
• Migration to eIQ Neutron SDK: The backend has transitioned to the self-contained eIQ Neutron SDK (eiq_neutron_sdk) package, replacing the legacy neutron_converter_SDK_<MCUX-ver>. The new SDK includes both the Neutron Converter and Neutron runtime for all NXP-supported platforms, simplifying backend deployment and removing the dependency on MCUXPresso SDK.
• Weight Prefetching: The Neutron backend supports fetching weights from external memory (such as flash) on NeutronC platforms. This feature enables model deployment on MCU-class SoCs where models are too large to fit in SRAM.
• Quantization Aware Training (QAT): The Neutron backend now supports Quantization Aware Training.
• Example Runtime for Neutron Backend: Introduced nxp_executor_runner, an example runner for the Neutron backend utilizing the Neutron simulator (eIQ NSYS).
• Channel-Last dim-order: Minor improvements in channel-last dim-order support.
• New Operations: `aten.avg_pool1d.default`, `aten.clamp.default`, `aten.div.Tensor`, `aten.leaky_relu.default`, `aten.neg.default`, `aten.prelu.default`, `aten.slice.default`, `aten._softmax.default`, `aten.squeeze.default`, `aten.squeeze.dim`, `aten.squeeze.dims`, `aten.u...

v1.1.0

ExecuTorch 1.1 Release Notes

Highlights

• CUDA Backend for NVIDIA GPUs: New experimental backend enables GPU inference with AOTInductor compilation, Triton SDPA kernels, INT4 weight quantization, and async memory allocation achieving latency reduction—validated on Voxtral, Gemma3, and Whisper with Windows support
• Metal Backend for Apple GPUs: New experimental backend enables GPU inference on Apple Silicon using Metal Performance Shaders with custom SDPA kernel achieving peedup—validated on Voxtral, Whisper, and Parakeet models
• Vulkan Static Int8 Quantization: Complete infrastructure for statically quantized CNNs with packed int8 tensors, specialized memory layouts for conv/matmul, and fully quantized conv2d variants—includes extensive shader-level performance optimizations and SDPA fusion for LLM inference
• Cadence DSP for Vision and LLMs: New optimized Vision module kernel library for Cadence/Xtensa DSPs with quantized implementations targeting vision workloads; added LLM operator support including RoPE, group-quantized embedding, and batched matmul for transformer models
• 4-bit Weight Quantization: Arm Ethos-U NPU now supports A8W4 quantization (4-bit weights, 8-bit activations) providing 2x weight compression for conv2d, conv3d, depthwise conv, and linear operators with automatic runtime unpacking
• Multimodal Vision-Language Models: Qualcomm backend adds complete infrastructure for VLMs including SmolVLM-500M and InternVL3-1B with vision encoder quantization, multimodal AOT compilation, and Adreno GPU backend support
• Dynamic Control Flow: torch.cond and while loops now supported across Arm (Ethos-U, VGF, TOSA) and LLM pipelines, enabling dynamic execution patterns like early exit, adaptive computation, and iterative refinement in both quantized and floating-point models
• Python 3.13 Support: ExecuTorch now fully supports Python 3.13 (requires coremltools 9.0+) with wheels built and tested for Python 3.10-3.13
• Embedded Platform Expansion: Zephyr RTOS integration enables ExecuTorch as an external module with Kconfig options and Corstone-300/Ethos-U acceleration; new Raspberry Pi Pico2 deployment tutorial for RP2040 microcontrollers with <400KB memory footprint; Cortex-M backend adds CMSIS-NN accelerated kernels for linear, conv2d, depthwise conv2d, avg_pool2d, softmax, and other operations enabling CNNs and transformers on microcontrollers
• Security Hardening: Addressed multiple CVEs including integer overflow in HierarchicalAllocator, stack buffer overflow in resize_tensor, and overflow-safe arithmetic in nbytes/numel using c10::mul_overflows()
• Parakeet ASR Model: Added Parakeet TDT 0.6B automatic speech recognition model to examples with dedicated runner, export scripts, and timestamp support for production-quality speech-to-text applications

Core Components

EXIR/Export

• Scan operator support: Full implementation of torch.scan higher-order op with loop constructs, carry state management, and et_copy_index for output stacking—enables recurrent and stateful model patterns (#16028)
• Shared state memory planning: New option to co-locate shared buffers across entry points; automatically propagates mutability if a buffer is mutable in any entry point (#14230)
• Custom external constant filenames: external_constants config now accepts a Callable[[Node], Optional[str]] to route constants to custom files (e.g., lambda x: "weights" → "weights.ptd") (#15862)
• Dynamic shape fixes: create_arg now handles torch.SymInt arguments; dim_order_from_stride uses guard_or_false for symbolic strides (#16774, #15472)
• Multi-partitioner fix: Each partitioner now processed individually with its specific op requirements, fixing conflicts when partitioners have different decomposition expectations (#14458)
• SpecPropPass guard fix: Fixed double-tracing issue where tensor specs didn't capture shape guards—now generates specs from meta values (#15485)
• Constant deduplication fix: When constants share data but have different FQNs, both FQNs are now properly saved (#15139)
• PTEFile class: deserialize_pte_binary() now returns a PTEFile object; access the program via .program attribute (#15864)
• Removed qnnpack backend: The deprecated qnnpack backend has been deleted; migrate to XNNPACK (#14663)

Runtime

• Security fixes: Addressed CVE in HierarchicalAllocator with integer overflow check, fixed stack buffer overflow in resize_tensor (bounds check for dimension limit), and added overflow-safe arithmetic in nbytes() and numel() using c10::mul_overflows() (#16103, #15626, #16804, #15581)
• Caching CPU memory allocator: New CPUCachingAllocator class that reuses freed memory blocks, reducing allocation overhead for repeated inference (#16120)
• Windows 64-bit mmap support: Fixed mmap to use _fstat64 and uint64_t offsets, enabling files larger than 2GB on Windows (#15538)
• Flexible tensor views: make_tensor_ptr() now supports custom sizes, dim order, and strides for squeeze/unsqueeze operations; views properly keep source tensor alive via custom deleter (#14944, #15056)
• Execution state checking: New in_progress() API detects mid-execution state; execute() returns InvalidState if called during step() and auto-resets on error (#16531)
• Large model support: FreeableBuffer now supports uint64_t data pointers for cross-core memory and models larger than 2GB (#14570)

Kernels

• Quantization performance: Parallelized op_choose_qparams and added ARM NEON SIMD + multithreading for op_quantize, significantly accelerating quantized KV-cache prefill in LLMs (#15767, #15768, #15769)
• Parallel slice_copy: Added multithreading to slice_copy for large workloads, reducing RoPE computation overhead by 5-10% in LLM prefills (#16125)
• Fast clone_dim_order: Added memcpy fast path when dim order is unchanged—benchmarked at 4.4x faster (27.9ms → 6.4ms) (#15815)
• New operator: grid_sampler_2d: Full portable kernel implementation supporting bilinear/nearest interpolation with all padding modes (#16051)
• New operators: bitwise shift: Added bitwise_left_shift and bitwise_right_shift for Tensor and Scalar variants (#15893)
• Quantized embedding int32 indices: Sub-8-bit quantized embedding ops now support int32 indices in addition to int64 (#16518)
• Security fixes: Fixed fuzzer-discovered vulnerabilities including heap-buffer-overflow in constant_pad_nd, integer overflow in pixel_shuffle, division by zero in reduction ops, and out-of-bounds access in pad2d (#16468, #16138, #16462, #15865)

SDK/Pybindings

• Bundled program support: Added Python bindings for loading .bpte (bundled program) and .ptd (program with data) files (#14678)
• Thread introspection: Exposed get_num_threads() to Python for debugging and profiling (#15944)

Backend Delegates

Arm

Ethos-U NPU (U55/U85)

• 16-bit activation quantization (A16W8): New get_symmetric_a16w8_quantization_config() enables 16-bit activations with 8-bit weights for improved accuracy on precision-sensitive models—supports linear, conv2d, sigmoid, tanh, permute, LayerNorm, LSTM, rsqrt, add, sub, and mul operators (#14258, #15101, #15256, #16015, #15524)
• 4-bit weight quantization (A8W4): New get_symmetric_a8w4_quantization_config() enables 4-bit weights with 8-bit activations for 2x weight compression—supports conv2d, conv3d, depthwise conv, and linear; runtime automatically expands packed 4-bit outputs to int8 tensors (#16577, #15588)
• QAT BatchNorm folding: Quantization-Aware Training now supports Conv1d+BatchNorm1d and Conv2d+BatchNorm2d folding with in-place activation fusion (hardtanh, relu) for optimized quantized models (#16001)
• Vela compiler 4.5.0: Updated Ethos-U Vela compiler with bug fixes, removing multiple operator xfails that previously required workarounds (#16075)
• Output layout mismatch handling: Runtime now automatically detects and corrects Vela's output padding and packing, ensuring tensor byte layouts match ExecuTorch expectations without manual intervention (#15588)
• FPU embedded target: Added embedded target variant with floating-point unit support for mixed-precision workloads (#15202)

VGF GPU Backend

• Mixed INT+FP profile as default: VGF backend now defaults to supporting both quantized (INT) and floating-point operations in the same model—enables partial quantization where performance-critical layers stay in float while memory-bound layers use int8, validated on MobileNetV2 and Llama models (#16176, #15773, #16311)
• Conv3d operator support: Added 3D convolution for video and volumetric workloads with FP32, Int8, and A16W8 quantization support (#16093)
• Stable Diffusion model testing: VGF backend validated on Stable Diffusion transformer modules, demonstrating support for generative AI workloads (#14655)
• PyPI package installation: VGF backend dependencies can now be installed via examples/arm/setup.sh for easier setup (#15737)
• ETDump profiling support: DevTools integration enables on-device profiling and performance analysis for VGF-delegated models (#15221)

TOSA Infrastructure

• TOSA 2025.11.0: Updated to latest TOSA specification with improved operator coverage and compatibility (#16424)
• New operators: Added support for fill_.Scalar, bitwise_not, split_copy, remainder, copy, floor_divide, mean, sum, min/max with unset dim, select_scatter, masked_fill_.Scalar, clamp.Tensor, log1p, and tan (#14501, #14460, #14717, #15409, #15363, #15380, #14933, #15972, #16272, #16273)
• 6D tensor and pixel shuffle: Support for rank-6 tensors and pixel shuffle/unshuffle operations for super-resolution and image processing models (#14626)
• Per-channel rescale: TOSA...

v1.0.1

ExecuTorch version 1.0.1 is a small patch release containing fixes for the 1.0 release.

Changes

• Add pytorch-tokenizers as a Python dependency for ExecuTorch. This removes the need to install from source.
• Update coremltools dependency to version 9.0.
• Improve support for bool and int tensors in the Vulkan backend.
• Display a nice error message when building from source on Windows and git symlinks are not enabled.

Bug Fixes

• Fixed start position updates in the text LLM prefill logic.
• Fixed handling of negative indices in the portable index operator.
• Hardened several portable operator implementations against out-of-bounds memory access.

v1.0.0

We're excited to announce the release of ExecuTorch 1.0! This release marks the official transition out of beta status and focuses on stability, ease of use, and polish, as well as expanded support for multi-modal language models .

Highlights

• Expanded platform support for ARM64 Linux and experimental support for native x86 Windows, both for Python model preparation and runtime builds.
• New APIs for running multimodal models on Android, iOS, and desktop.
  1. Supported models include Voxtral, Gemma3, and more.
• LoRA inference capabilities - export multiple LoRA .pte files that share a single set of foundation weights.
• 4-bit HQQ quantization support for improved model accuracy.
• Python and Maven (Android) package variants for Vulkan and QNN backends.
• Experimental JavaScript support for the ExecuTorch runtime via WASM.

API Changes

• Expanded LLM inference APIs in C++, Java, Objective-C, and Swift, including support for vision and audio modalities.
• New ahead-of-time and runtime APIs to support shared weights between .pte files.

Build

• Add support for top-level CMake targets for backends, kernels, and extensions, including kernel selective build via CMake options.
• Remove buck as a dependency for CMake builds.
• Link Vulkan and QNN backends in Python runtime builds when enabled.
• Expose CMake presets for common platforms and use cases.
• Consolidate and clean up CMake options and build structure.

Backend Delegates

Arm

• Ethos-U backend
  • The Ethos-U backend has reached production quality in this release, with stable operator coverage, CI validation, and verified integration for embedded deployments.
• VGF / VKML backend
  • ExecuTorch’s Arm VGF backend is now documented and aligned with Arm’s ML SDK for Vulkan (VKML).
  • Includes end-to-end lowering to VGF and running/extracting VGF from .pte.
  • Correct handling for rank-0 tensors and improved input/output handling in the portable executor runner.
  • Runtime delegate added for Linux targets with the VKML emulator.
  • Comprehensive testing of the VGF AoT flow introduced, ensuring functional and performance validation across representative models
• Partitioner & API polish
  • Partitioner/compile interfaces simplified and refactored for easier extension and maintainability.
• Expanded TOSA coverage
  • Added and verified mappings for arithmetic, pooling, and reduction operators, improving model compatibility and TOSA lowering fidelity.
  • Ahead-of-time (AoT) path extended significantly to support the full set of Arm backend operators (now exceeding 80% Edge dialect coverage).
• Quantization pipeline
  • Broader data type support, refined logging, and clearer documentation.
  • Experimental 16A8W data layout support added, with tests for linear, add, and related INT16 ops.
• Portable runner enhancements
  • Added support for models with non-tensor inputs, broadening applicability to control-flow and hybrid workloads.
• Testing & CI
  • Expanded unit tests, out-of-the-box tests, and model verification scripts.
  • Broader CI coverage for quantized and general workloads.

Cadence

• Extend support for audio operators to 50+, covering most audio models.
• Support the first vision operator (softmax).
• Various improvements to the AoT compiler.
• Optimized operators give 20-50x speedups compared to vanilla on 7 OSS models.

Core ML

• Support block-wise, channel-wise, and pallet/codebook quantization of linear and embedding layers with torchao APIs.
• Support for enumerated shapes for limited support dynamism on the ANE.
• Improve support for dim order.
• Fix crashes for several operators with mixed dtypes.
• Support models and partitions with no user inputs.
• Log when operator nodes are not partitioned and why.
• Add a lower_full_graph partitioner option to assert that all model operators are running on Core ML.
• Add a take_over_constant_data option to tell CoreML delegate to not consume weight data.
• Various small bug fixes and improvements.

MediaTek

• Support weight sharing for llama.
• Support qwen, phi, and gemma models.
• Update MTK tool version.

MPS

• No major updates since 0.7.

NXP

• Support for additional operators: abs, _adaptive_avg_pool2d, addmm, add.Tensor, avg_pool2d, cat, clone, constant_pad_nd, convolution, hardtanh, max_pool2d, max_pool2d_with_indices, mean.dim, mm, relu, tanh, view_copy, and sigmoid.
• Support i.MX RT700 platform with MCUXpresso SDK 25.06.
• Add model support for CNN models – MobileNetV2, CifarNet, and MLPerf Tiny.
• Update documentation to fix links and add a tutorial for on device deployment.

OpenVINO

• New backend delegate for inference acceleration on Intel platforms for CPU, GPU and NPU.
• Support for Intel CPUs, integrated GPUs, and NPUs. Validated on Intel Core Ultra Series 2 AI PCs.
• Run Llama, Stable Diffusion LCM, and Yolo models.
• Expanded aten operator coverage for broader model compatibility.
• Introduce 4-bit (INT4) weight compression for LLMs and 8-bit (INT8) quantization for vision models via OpenVINOQuantizer.
• Add new end-to-end examples for Yolo and Stable Diffusion, instructions for exporting Llama models, and update build documentation to use the official OpenVINO release package.

Qualcomm

• Support 26 new models (Albert, Bert, Distilbert, Eurobert, Roberta, Gemma3-1B, Llama-3.2-1B-instruct, Llama-3.2-3B-instruct, Qwen2.5-0.5B, Qwen2.5-1.5B, Qwen3-0.6B, Qwen3-1.7B, Phi-4-mini-instruct, SmolLM2-135M, SmolLM3-3B, Google's T5k, k, Convolutional Vision Transformer, Distilled Data-efficient Image Transformer, Dit: Document Image Transformer, EfficientNet, Focalnet, Apple's MobileViT and MobileViTv2, Pvt: Pyramid Vision Transformer, Swin: Swin Transformer, Whisper).
  1. Optimization including MaskedSoftMax
  2. SeqMSE
  3. TorchAO SpinQuant
  4. Mixed precision, including linear layers and kv cache.
  5. Support for lookahead decoding.
• Support for custom operators.
• Support for 92/116 HTP operators.
• Support for automatic installation of model preparation dependencies with the executorch pip package.
• Add runtime options to adjust log level, performance mode and profiling level in runtime
• Upgrade to QNN SDK 2.37.

Samsung

• New backend delegate for Samsung Exynos (#13677), which enables inference of ExecuTorch models via the DSP/NPU of Samsung SoCs.
  • Currently supported chipsets:
    • Exynos 2500 (E9955)
  • Includes support for 60 operators (#14106).
• Enable support for statically 8-bit quantized models (#14464).
  • Quantization support included for 34 operators .
• Currently validated models:
  • DeepLab V3, EDSR, Inception V3, Inception V4, MobileNet V2, MobileNet V3, ResNet 18, ResNet 50, ViT, Wav2Letter, MobileBERT

Vulkan

• We have partnered with Samsung's GPU compute team to collaborate on delivering general and Samsung GPU specific optimizations to the ExecuTorch Vulkan backend; the following optimizations/improvements were introduced via this partnership:
  1. Introduce operator fusions for fusing clamp to convolution, binary, and other clamp operators (#14960).
  2. Compute shader optimizations for convolution (#14724).
  3. Add support for 2D reduction operators (#12860).
• Add support for 8da4w (8-bit dynamically per-channel quantized activations, 4-bit per-group quantized weights) quantized linear layers, whose implementation leverages the accelerated integer dot product Vulkan extension
• Introduce an automatic half precision mode; setting the force_fp16 compile option will cause the Vulkan backend to internally convert fp32 tensors to fp16
  1. Inputs are still expected to be fp32 (they will be automatically converted to fp16 within the delegate).
  2. Outputs will be converted back to fp32 as they are returned.
• Add support for high dimensional tensors; the Vulkan backend now supports tensors with <= 8 dimensions; previously only tensors with <= 4 dimensions could be represented.
• Reduce peak memory usage during model loading by ~2-3x via employing lazy allocation of GPU resources - see #13474 for more details.
• Improved operator support:
  1. Add support for aten.expand_copy.
  2. Add support for aten.max_pool2d; previously only aten.max_pool2d_with_indices was supported.
  3. aten.cat can now handle any number of input tensors; previously, only up to 3 tensors were allowed.
  4. Add support for grouped convolutions.
• Operator-level optimizations:
  1. Re-implemented SDPA with optimized compute shaders (#14130).
  2. Improve local work group size selection for matrix multiplication and linear (#13378).

XNNPACK

• Support batch norm fusion with linear operators.
• Use a new group-based partitioning algorithm to resolve occasional errors with partitioning of quantized graphs.
• Support channels-last dim order tensors.
• Optimize transpose pairs.
• Use KleidiAI kernels on ARM platforms for improved GEMM performance.
• Enable SME2 kernels via KleidiAI library by default
• Support runtime control of the workspace sharing option for memory ...

v0.7.0

We're excited to announce the release of ExecuTorch 0.7! This release builds upon the foundation established in the 0.7 release with significant improvements across various components of the framework. ExecuTorch continues to be a powerful solution for on-device AI, powering experiences across a wide range of platforms.

Highlights

• New export_llm API: Unified CLI and API to export supported LLM models, replacing the old export_llama script
• Generic Text LLM Runner: Support for all text-only decoder-only LLM models with full Android & iOS integration
• Program-Data Separation: Experimental APIs to export program and data files separately with new '.ptd' file extension
• KleidiAI Integration: Now enabled by default in XNNPACK backend for optimized low-bit matrix multiplication
• LLM Model Support: Added support for Qwen3, SmolLM3, and Gemma3 models
• Developer Tools: New numerical debugging capabilities and inspector APIs

API Changes

• New export_llm API, repackaged from the old export_llama script, which provides a unified CLI and API to custom export supported LLM models
• New generic text LLM runner that is able to support all text-only decoder-only LLM models. Fully integrated with Android & iOS demo apps.
• Experimental program-data separation APIs to export program and data files separately. Introduce the new ‘.ptd’ file extension for data-only files. Support for portable operators and the XNNPACK backend.
• Added a new runtime API for PAL overrides and log routing.

Deprecated

• Constant buffer will be deprecated in the runtime in coming releases. Please re-export any PTE files generated before ExecuTorch 0.4.

Build

• CMake
  • Model API for data type selective build available.
  • Bumped minimum CMake requirement to 3.29
• Pip Wheel Build: Various improvements to wheel building process and CI infrastructure (link1, link2, link3), XNNPACK enabled by default for wheels
• Build Presets: Introduced CMake presets for common build configurations (iOS, macOS, Linux, pybind, LLM)

Backend Delegates

Arm

• Cortex-M Support: Initial commit for Cortex-M backend with scalar C++ ops for quantize/dequantize operations
• TOSA 1.0 Support: Updated to support TOSA 1.0 specification with improved operator coverage
• New Operators: Added alias_copy, arange.default, aten.mul.Tensor (int32 BI/INT), aten.ne, atan, BatchNorm2D, cosine_similarity, embedding.default, eq.Scalar, erf, ge.Scalar, gelu, GroupNorm, grouped_convolution, gt.Scalar, index.Tensor, index_select, linalg_vector_norm (DecomposeLinalgVectorNorm), lt.Scalar, neg.default, aten.round, scaled_dot_product_attention (SDPA), sinh, sqrt, upsample_bilinear2d, where.self
• Quantization: Enhanced per-channel quantization support and QAT (Quantization Aware Training) improvements
• Performance: Improved pooling operations, broadcasting, and memory allocation strategies
• 5D Tensor Support: Extended backend to handle 5D tensors for more complex models

CoreML

• Model Support: Enhanced transformer support with RoPE and RMSNorm original definitions
• Cross-platform: Improved CoreML export support on Linux platforms
• Documentation: Updated documentation with corrected file names and commands

Qualcomm

• AI Engine Direct: GA enablement for multiple models including Albert, Bert, DistilBert, Eurobert, DeiT, EfficientNet, MobileViT, PVT, CvT, DIT, FocalNet, Swin Transformer, and Roberta
• Multi-method Support: Enhanced support for multiple methods in QNN backend
• Lookahead Decoding: Enabled lookahead decoding capabilities
• Custom Operators: Support for custom operator integration
• QAIRT Visualizer: Integration with QAIRT visualization tools

MediaTek

• CI Integration: Added MediaTek backend CI support
• Buffer Allocator: Decoupled Neuron buffer allocator from ExecuTorch framework
• Documentation: Updated documents for Express SDK and buffer allocator usage
• Platform Config: Introduced platform-config in CompileSpec for better hardware targeting

MPS

• Documentation: Updated MPS documentation with correct file names and commands
• Example Fixes: Fixed MPS example for non-LLM models

NXP

• Added initial version of eIQ Neutron Backend and runtime
• Support for basic NN operators
• Quantization and conversion support for MobileNets and convolution-based networks
• Complete end-to-end support for MobileNetV2 and CifarNet with examples

Vulkan

• Operator Support: New implementations for native_group_norm, permute, var.dim, tan, quantization/dequantization ops
• Performance Optimizations: Improved conv2d operations with specialized shaders and reduced memory usage
• Dynamic Shape Support: Enhanced support for dynamic shapes and symbolic operations
• Testing Infrastructure: Improved operator test codegen system and validation

XNNPACK

• KleidiAI is now enabled by default on the XNNPACK Backend! This provides optimized low-bit matrix multiplication kernels that boost Llama prefill and decode performance
• New SDOT kernels by KleidiAI are introduced for platforms lacking the i8mm extension, ensuring improved compatibility and performance for quantized models on a wider range of devices.
• Weight-sharing support between different methods within the same PTE file.
• Program-data separation support. Export and run an XNNPACK-lowered model with program and data in separate files.

Android / iOS

• LLM Integration: Enhanced Android demo app with generic LLM runner support
• Java APIs: New runtime APIs for registered ops and backends
• Dynamic Image Loading: Add support for loading images from /sdcard and /data partitions, with user-granted permissions for image segmentation use cases. See executor-examples proj Deeplab V3 Demo for latest updates
• Logging: Added Android log implementation for better debugging
• Build Improvements: Updated Android CMake configuration and artifact generation

Devtools

• Numerical Debugging: New calculate_numeric_gap API to compare logged operator-level intermediate outputs from exported graph (ETRecord) with runtime outputs (ETDump)
• Inspector Enhancements: Integrated debug handle to operator name mapping and intermediate output capturing
• Numerical Comparators: Added L1, MSE, and SNR numerical comparators for model validation
• ETDump Integration: Enhanced ETDump support in module APIs

Model Support

• Enabled Qwen3 (0.6B, 1.7B, 4B), SmolLM3, and Gemma3 1B with high out-of-box performance
• LLaVA: Improved multimodal model support with enhanced image and text processing
• Quantization: Enhanced quantization support across various models with improved precision and performance

Ops and kernels

• Portable Kernels: Added randn and rand kernel implementations, native_dropout support
• Scalar Overflow Protection: Enhanced scalar overflow checking across multiple operators
• Vectorization: Improved vectorized math operations and elementwise utilities
• Memory Operations: Enhanced copy utilities and memory planning algorithms
• Broadcasting: Improved broadcast support for various binary operations
• Complex Number Support: Added complex dtype support for mul, sum, bmm, and diagonal_copy operations
• New Operators: Added support for leaky_relu, hardtanh, amax, view_as_real_copy, and FFT operations
• Optimized Implementations: Enhanced ELU implementation and improved elementwise utilities

Contributors

@abeakkas @abhinaykukkadapu @andreanicastro @anijain2305 @jainapurva @desertfire @BujSet @chenweng-quic @Conarnar @ethansfng @fumchin @HDCharles @IshanAryendu @Juntian777 @skywall @mchien512 @leafs1 @michaelmaitland @NeelakshSharma @nil-is-all @nitish2112 @Burton2000 @rmaz @rascani @robert-kalmar @rohansjoshi @psiddh @suvadeep89 @tamird @tbergkvist @phaiting @tobbeebbot @wl1026sun @ynimmaga @ydwu4 @cyyever @keyprocedure

Full Changelog: v0.6.0...v0.7.0