Skip to content

[REFACTOR][IR] Unify PrimExpr type mechanism to PrimType instead of DataType#19875

Open
tqchen wants to merge 5 commits into
apache:mainfrom
tqchen:task-use-ty-primtype-for-source-of-dtype
Open

[REFACTOR][IR] Unify PrimExpr type mechanism to PrimType instead of DataType#19875
tqchen wants to merge 5 commits into
apache:mainfrom
tqchen:task-use-ty-primtype-for-source-of-dtype

Conversation

@tqchen

@tqchen tqchen commented Jun 23, 2026

Copy link
Copy Markdown
Member

In the past we have been using DataType in PrimExpr.dtype field to check type information for PrimExpr while still having BaseExpr.ty for richer type information. DataType is also used both in runtime and compiler. This PR streamlines the boundary:

  • PrimExpr.ty now carries PrimType that replaces original use of DataType
  • Runtime use will now favor DLPack DLDataType, removing one layer of indirection.
  • Constants attributes where values are usually runtime values, will use DLDataType
  • DataType will be phased out after this PR

We also brings up helper functions in PrimType, but also limits them to a more concise set so the functions do not grow with the data type codes in DLPack.

This is a major refactor that changes the IR primitive. It helps to bring possible future benefits:

  • Unified type mechanism through Expr.ty
  • Possibility of carry future Type nodes

Migration Guide:

  • Use PrimType when code reasons about compiler expression types, tensor element compiler types, or constructs a PrimExpr/compiler type.
  • Use existing source types such as expr.ty(), ExprOp.expr_ty(), or TE tensor element dtype where possible instead of rebuilding a type from dtype text.
  • Use raw DLDataType for runtime constants, ABI paths, dtype-valued attrs, and storage/runtime helper logic.
  • Prefer direct PrimType equality, MatchesCode(...), MatchesElementType(...), and WithCode(...) over local wrappers or string dtype checks.

Performance:

Using Object type instead of DLDataType would indeed bring some performance impact to the IR. We have done the following performance optimizations:

  • Make sure most of the outputs reuse one of the PrimType from inputs
  • Cache a thread local PrimType based on input so we don't repeatly realloc

We did benchmarks show that rewrite simplify operation stays within +-10% overhead of original one. Which merits the refactor given the benefit the unfication brings

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the codebase to transition from using DataType to PrimType (backed by DLDataType), unifying the type system across IR dialects and improving type safety. The reviewer provided valuable feedback on leveraging the new PrimType APIs (such as bits(), lanes(), and IsScalableVector()) to avoid manual DLDataType unpacking and arithmetic, particularly in tensor allocation, memory view handling, static memory planning, and GPU code verification. Additionally, a bug was identified in src/relax/transform/utils.h where kDLBool was incorrectly checked against a bit width of 1 instead of the standard 8 bits.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/relax/transform/lower_alloc_tensor.cc
Comment thread src/relax/op/memory/view.cc
Comment thread src/relax/transform/static_plan_block_memory.cc Outdated
Comment thread src/relax/transform/utils.h Outdated
Comment thread src/s_tir/analysis/verify_gpu_code.cc Outdated
Comment thread src/s_tir/analysis/verify_gpu_code.cc
Comment thread src/s_tir/analysis/verify_gpu_code.cc Outdated
Comment thread src/s_tir/analysis/verify_gpu_code.cc Outdated
Comment thread src/s_tir/analysis/verify_gpu_code.cc Outdated
@tqchen tqchen force-pushed the task-use-ty-primtype-for-source-of-dtype branch 3 times, most recently from 3e75c55 to 74fba5e Compare June 23, 2026 16:32
@tqchen

tqchen commented Jun 23, 2026

Copy link
Copy Markdown
Member Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the TVM codebase by replacing the usage of tvm::DataType with tvm::PrimType and DLDataType across C++ and Python files to better align with DLPack. The code review feedback is highly consistent, recommending the use of the more idiomatic MatchesCode helper on PrimType instead of directly comparing raw data type codes (such as kDLFloat, kDLInt, and kDLBool). Additionally, it is suggested to simplify redundant checks by passing multiple type codes to a single variadic MatchesCode call.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/relax/op/tensor/index.cc Outdated
Comment thread src/relax/op/nn/nn.cc Outdated
Comment thread src/relax/op/nn/nn.cc Outdated
Comment thread src/relax/op/distributed/nn.cc Outdated
Comment thread src/relax/op/distributed/unary.h Outdated
Comment thread src/arith/transitive_comparison_analyzer.cc Outdated
Comment thread src/arith/int_constraints.cc Outdated
Comment thread src/arith/int_constraints.cc Outdated
Comment thread src/arith/int_set.cc Outdated
Comment thread src/arith/int_set.cc Outdated
@tqchen

tqchen commented Jun 23, 2026

Copy link
Copy Markdown
Member Author

/gemini Please run another review pass on latest head b8c7505. Since the last pass, I applied the Gemini MatchesCode cleanups, fixed the Python PrimType-to-runtime-dtype boundary in Relax gradient constants and TE NumPy test uses, reran full LLVM build plus focused Relax/TE/TIRX validation, and resolved the previous review threads that are addressed.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors TVM's type system by replacing the runtime-specific DataType with a unified compile-time PrimType (backed by DLDataType) across the compiler, runtime, and Python bindings. This extensive refactoring updates buffer declarations, expression nodes, and codegen backends to use PrimType or DLDataType directly. Feedback on the changes highlights a potential division-by-zero bug in Hexagon's GetVectorBytes for sub-byte types, a potential AttributeError in the Python PrimExpr.dtype property when self.ty is null or a pointer, and several opportunities to simplify type checks using the newly introduced IsScalar() helper and PrimType equality operators.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/backend/hexagon/codegen/llvm/codegen_hexagon.cc
Comment thread python/tvm/ir/expr.py Outdated
Comment thread include/tvm/topi/detail/broadcast.h Outdated
Comment thread include/tvm/topi/detail/broadcast.h
Comment thread src/arith/const_fold.h
Comment thread src/s_tir/transform/bound_checker.cc
@tqchen

tqchen commented Jun 23, 2026

Copy link
Copy Markdown
Member Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request performs a major refactoring of TVM's type system by removing the tvm::runtime::DataType wrapper class and replacing its usage with tvm::PrimType and raw DLDataType across the entire codebase, including TIR, Relax, TOPI, and various hardware backends (CUDA, Vulkan, Metal, WebGPU, Hexagon, and Trainium). PrimExpr and PrimExprNode now expose their types via PrimType (ty()) instead of DataType (dtype()), and corresponding Python bindings have been updated to reflect these changes. Since no review comments were provided, there is no specific feedback to address, and the changes appear to successfully unify and simplify the type representation.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@tqchen tqchen force-pushed the task-use-ty-primtype-for-source-of-dtype branch from b42ff43 to 35d4a00 Compare June 23, 2026 20:14
Unify compiler-facing PrimExpr dtype state around PrimType as the source of expression type truth, while keeping runtime/storage dtype values as raw DLDataType at ABI and storage boundaries.

This squashes the PrimType refactor, review/CI follow-ups, raw dtype boundary repairs, and the benchmark-confirmed rewrite hot-path recovery into one PR commit.
@tqchen tqchen force-pushed the task-use-ty-primtype-for-source-of-dtype branch from c66b69e to ffccd86 Compare June 24, 2026 00:12
@tqchen tqchen changed the title [REFACTOR][IR] Use PrimType for compiler dtypes [REFACTOR][IR] Unify PrimExpr type mechanism to PrimType Jun 24, 2026
@tqchen tqchen changed the title [REFACTOR][IR] Unify PrimExpr type mechanism to PrimType [REFACTOR][IR] Unify PrimExpr type mechanism to PrimType instead of DataType Jun 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants