TVM v0.7 Release Note Candidate #6486
Description
Overview
v0.7 is brings many major features. The community works together to refactor the internal code base to bring an unified IR code structure with unified IRModule, type system and pass infrastructure. We have also bought many exciting new features, some highlights include:
- Initial automatic scheduling support
- Initial command line driver interface
- WebGPU and webassembly support
- Better first class rust support in the codebase
- Intial Hexagon support
- Bring your own codegen (BYOC) support
The community also continues to bring high quality improvements to the existing modules including, but not limited to: better frontend coverage, performance, quantization, uTVM and dynamic shape support.
New Features
Automatic Scheduling (Experimental)
- Phase 0: Ansor minimum system for auto schedule generating #5962
- Phase 1: Access Analyzer #6103
- Phase 1: Add follow_split and follow_fused_split steps #6142
- Phase 1: Add pragma/storage_align/rfactor steps #6141
- Phase 1: Add RPC Runner #6077
- Phase 1: Add annotation/compute_at/compute_root/compute_inline steps #6073
- Phase 1: Add cache_read/cache_write steps #6107
- Phase 1: Rename namspace form auto_schedule to auto_scheduler #6059
- Phase 1: The base class for cost models #6187
- Phase 1: The base class for cost models #6187
- Phase 1: feature extraction for cost models #6190
- Phase 1: XGBoost Cost Model #6270
- Phase 2: Basic GPU Sketch Search Policy #6269
- Phase 2: Evolutionary Search #6310
- Phase 2: Update heavy operations with parallel_for #6348
- Parallel the InitPopulation ([ANSOR] Auto-scheduler tutorial for GPU and necessary refactor/fix #6512)
- Tutorial: Using the template-free auto-scheduler on CPU ([TUTORIAL][ANSOR] Using the template-free auto-scheduler on CPU #6488)
BYOC
- External codegen support in Relay (#4482)，(#4544)
- Bring Your Own Codegen Guide -- Part 1 #4602
- Bring Your Own Codegen Guide -- Part 2 #4718
- Relay annotation and partitioning for external compilers #4570
- JSON Runtime with DNNL End-to-End Flow #5919
- Handle one symbol for each runtime #5989
- Run accelerator specific optimizations #6068
- Arm Compute Library integration #5915
- Retire the example json runtime #6177
- json_node.h should include data_type.h #6224
- Improve installation tutorial #6170
- Add support for dense (fully connected) layer #6254
- Introduce the Ethos-N BYOC integration #6222
- Enable remote device via environment variables #6279
- Improved pooling support #6248
- Add support for quantized convolution #6335
- CoreML codegen #5634
Operator Coverage
- Add strided_set operation (#4303)
- Add support for conv3d (#4400), pool3d (#4478), 3d upsampling ops (#4584)
- Add group convolution for VTA (#4421)
- Add 1d deconvolution op (#4476)
- Allow batch matmul to be fused into injective ops (#4537)
- Add native depthtospace and spacetodepth operators (#4566)
- Add CUDNN conv3d support (#4418)
- Dilation2D operator support #5033
- Isfinite operator #4981
- Unravel Index operator #5082
- Add thrust support for nms #5116
- Resize3d, Upsample3d op support #5633
- Add operator Correlation #5628
- affine_grid and grid_sample #5657
- Sparse to dense operator #5447
- Conv3d_transpose op support added #5737
- add op crop_and_resize #4417
- Add bitwise ops #4815
- Sparse to dense operator #5447
- support dynamic NMS(Non Maximum Suppression), symbolic begin, end, and strides for strided_slice #4312
- Conv3d_transpose op support added #5737
- ReverseSequence operator #5495
- Conv1D #4639
- 1D Pooling #4663
Quantization
- Channel wise quantization - Quantize & Requantize #4629
- Support QNN ops. #5066
- Adding support for QNN subtract op #5153
- TFLite QNN Tutorial #5595
- Tutorial: Deploy Quantized Model on CUDA #4667
- Support asymmetric per-layer quantized operators #6109
Relay
- Add convertlayout pass in Relay (#4335, #4600)
- Added Merge Composite pass #4771
- Call graph for relay #4922
- Add inline pass #4927
- Target annotation for external codegen #4933
- GradientCell Relay Pass #5039
- Add MergeCompilerRegions pass #5134
- Non-recursive Graph Vistor and Rewriter (#4886)
- [Blocksparse] Pipeline for lowering dense model to sparse-dense (#5377)
- Relay op strategy #4644
- Static Tensor Array (#5103)
- Memory planner (part 1) #5144
- ONNX codegen #5052
- Add Parser 2.0 #5932, part 2 #6162
- Basic block normal form #6152
- Convert Layout pass. #4664
- Pattern Language, Matcher, Rewriter, and Function Paritioner #5231
Runtime and Backend
- Add ADTObject POD container type (#4346)
- TFLite RPC runtime (#4439)
- Standardized graph runtime export (#4532)
- MISRA-C compliant TVM runtime #3934
- Add String container #4628
- Introduce Virtual Memory Allocator to CRT ([uTVM][Runtime] Introduce Virtual Memory Allocator to CRT #5124)
- Initial implementation of Hexagon runtime support ([RUNTIME] Initial implementation of Hexagon runtime support #5252)
- FastRPC interface for Hexagon runtime ([RUNTIME] FastRPC interface for Hexagon runtime #5353)
- CoreML Runtime ([RUNTIME][CONTRIB] CoreML Runtime #5283)
- AutoTVM + uTVM for Cortex-M7 ([RUNTIME][uTVM] AutoTVM + uTVM for Cortex-M7 #5417)
- Windows Support for cpp_rpc (Windows Support for cpp_rpc #4857)
- Implement TVMDSOOp(TensorFlow custom op) for TVM runtime ([RUNTIME] Implement TVMDSOOp(TensorFlow custom op) for TVM runtime #4459)
- WebGPU support #5545
- TVM WebAssembly JS Runtime #5506
- Hexagon driver for offloading kernels to simulator #5492
- Introduce runtime::Array #5585
- Allow non-nullable ObjectRef, introduce Optional. (#5314)
- Introduce static slots for common objects. (#5423)
- ntroduce RValue reference(move) support to TypedPackedFunc (#5271)
- Introduce MetadataModule to separate code compilation/interpretation and weight initialization #5770
- Support module based interface runtime #5753
- Add TVM application extension with WASM runtime #5892
- Provide guide to user who has difficulty register SEqualReduce (#5300)
Rust Support
- Revive the Rust + SGX refactor #4976
- Improve Rust bindings: Map, Array, String, various IR nodes #6339
- Rust Refactor Stage 4: Rewrite Rust graph runtime to use new APIs #5830
- Second stage of Rust Refactor #5527
tvmcrate stage 3 of Rust refactor #5769- Add first stage of updating and rewriting Rust bindings. #5526
TIR
- Introduce StructuralHash for the Unified IR. #5160
- Introduce StructuralEqual Infra for the unified IR. #5154
- Introduce ExprDeepEqual, Remove IRDeepCompare #5206
- [TIR] Introduce BufferLoad/Store (#5205)
- Improved massive build times caused by tir.floormod and tir.floordiv. Fixed Topi testcase. #5666
- Buffer logger assert removed #6147
- Enhance VerifyGPUCode #6194
- HoistIfThenElse added #6066
- Hybrid Script Support for TIR #6227
- Migrate Low-level Passes to Pass Manager #5198
- HoistIfThenElse added #6066
- Hybrid Script Support for TIR #6227
- Block scope hoisting added #6238
TE
- reverse-mode autodiff without any optimization #5121
- Tensor Expression Debug Display (TEDD) #4651
- Optimize and eliminate the Jacobian tensor for te.autodiff #6078
TVMC(Experimental)
- TVMC - A command line driver for TVM (Part 1) #6112
- TVMC - Linting error on onnx command line driver frontend #6536
- TVMC - Command line driver 'compile' (part 2/4) #6302
- TVMC - Introduce 'tune' subcommand (part 3/4) #6537
- TVMC - Introduce 'run' subcommand (part 4/4) [tvmc] Introduce 'run' subcommand (part 4/4) #6578
- TVMC - Getting started tutorial for TVMC [tvmc][docs] Getting started tutorial for TVMC #6597