Skip to content

slightly different matmul algorithm causes an error message #512

Closed
Closed
slightly different matmul algorithm causes an error message#512

Description

@derisavi
derisavi
opened on Oct 5, 2017

Hi,
I've rewritten the matmul algorithm as follows by peeling the first iteration of the summation. When I run the following code,

import tvm
M = 600
N = 650
P = 700
# Algorithm
k = tvm.reduce_axis((1, N), 'k') # note the starting value of 1 (instead of 0)
A = tvm.placeholder((M, N), name = 'A')
B = tvm.placeholder((N, P), name = 'B')
C = tvm.compute(
    (M, P),
    lambda x, y: A[x,0]*B[0,y] + tvm.sum(A[x, k] * B[k, y], axis = k),
    name = 'C')

# Default schedule
s = tvm.create_schedule(C.op)
print(tvm.lower(s, [A, B, C], simple_mode=True))

I get the following error message. Am I doing anything wrong? Is there a way to specify the matmul algorithm while explicitly peeling the first iteration of the sum reduction?
The reason I want do this is that I don't want tvm to generate zero-initialization code for C. I want C[x,y] to be initialized to A[x,0]*B[0,k] and then to generate the summation code.

Thanks!

[17:01:51] /hwapwork/s00327669/software/tvm/dmlc-core/include/dmlc/logging.h:308: [17:01:51] src/lang/ir.cc:23: Reduce do not work with old Visitor, use IRFunctor style visitor

Stack trace returned 10 entries:
[bt] (0) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x39) [0x7faae5e37bb9]
[bt] (1) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::ExprNode<tvm::ir::Reduce>::accept(Halide::Internal::IRVisitor*, Halide::Expr const&) const+0x4f) [0x7faae5ed7c9f]
[bt] (2) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::IRMutator::mutate(Halide::Expr)+0x1a) [0x7faae6193cba]
[bt] (3) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::Simplify::visit(Halide::Internal::Add const*, Halide::Expr const&)+0xdf) [0x7faae61319af]
[bt] (4) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::IRMutator::mutate(Halide::Expr)+0x1a) [0x7faae6193cba]
[bt] (5) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::Simplify::visit(Halide::Internal::Store const*, Halide::Internal::Stmt const&)+0xca) [0x7faae6114f1a]
[bt] (6) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::IRMutator::mutate(Halide::Internal::Stmt)+0x1a) [0x7faae6193d2a]
[bt] (7) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::Simplify::visit(Halide::Internal::For const*, Halide::Internal::Stmt const&)+0x15a) [0x7faae6163aca]
[bt] (8) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::IRMutator::mutate(Halide::Internal::Stmt)+0x1a) [0x7faae6193d2a]
[bt] (9) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::Simplify::visit(Halide::Internal::For const*, Halide::Internal::Stmt const&)+0x15a) [0x7faae6163aca]

Traceback (most recent call last):
  File "test_sum.py", line 61, in <module>
    print(tvm.lower(s, [A, B, C], simple_mode=True))
  File "/home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/build_module.py", line 228, in lower
    stmt = ir_pass.Simplify(stmt)
  File "/home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/_ffi/function.py", line 255, in my_api_func
    return flocal(*args)
  File "/home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/_ffi/_ctypes/function.py", line 183, in __call__
    ctypes.byref(ret_val), ctypes.byref(ret_tcode)))
  File "/home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/_ffi/base.py", line 62, in check_call
    raise TVMError(py_str(_LIB.TVMGetLastError()))
tvm._ffi.base.TVMError: [17:01:51] src/lang/ir.cc:23: Reduce do not work with old Visitor, use IRFunctor style visitor

Stack trace returned 10 entries:
[bt] (0) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x39) [0x7faae5e37bb9]
[bt] (1) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::ExprNode<tvm::ir::Reduce>::accept(Halide::Internal::IRVisitor*, Halide::Expr const&) const+0x4f) [0x7faae5ed7c9f]
[bt] (2) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::IRMutator::mutate(Halide::Expr)+0x1a) [0x7faae6193cba]
[bt] (3) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::Simplify::visit(Halide::Internal::Add const*, Halide::Expr const&)+0xdf) [0x7faae61319af]
[bt] (4) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::IRMutator::mutate(Halide::Expr)+0x1a) [0x7faae6193cba]
[bt] (5) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::Simplify::visit(Halide::Internal::Store const*, Halide::Internal::Stmt const&)+0xca) [0x7faae6114f1a]
[bt] (6) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::IRMutator::mutate(Halide::Internal::Stmt)+0x1a) [0x7faae6193d2a]
[bt] (7) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::Simplify::visit(Halide::Internal::For const*, Halide::Internal::Stmt const&)+0x15a) [0x7faae6163aca]
[bt] (8) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::IRMutator::mutate(Halide::Internal::Stmt)+0x1a) [0x7faae6193d2a]
[bt] (9) /home/s00327669/.local/lib/python2.7/site-packages/tvm-0.1.0-py2.7-linux-x86_64.egg/tvm/libtvm.so(Halide::Internal::Simplify::visit(Halide::Internal::For const*, Halide::Internal::Stmt const&)+0x15a) [0x7faae6163aca]

Activity

  1. derisavi
    changed the title [-]slightly different matmul algorithm causes error message[/-] [+]slightly different matmul algorithm causes an error message[/+] on Oct 5, 2017

  2. tqchen commented on Oct 5, 2017

    @tqchen
    tqchen
    on Oct 5, 2017
    Member

    Currently, we do not support inlining of reduction with another stage. What we can do is to declare reduction in one stage and add in another.

  3. derisavi commented on Oct 6, 2017

    @derisavi
    derisavi
    on Oct 6, 2017
    ContributorAuthor

    I see. When I put it in stages, tvm still generates code that initializes C's elements to zero before computing the sum.
    Basically, what I need to do is to guide tvm to peel the first iteration of a summation, i.e., generate something like this (the following is not matmul obviously, just an example to illustrate the idea)

    for (x,0,600) {
  C[x] = A[x,0]*B[x,0]
  for (y,0,599) {
    C[x] = C[x] + A[x,y+1]*B[x,y+1]
  }
}

    instead of

    for (x,0,600) {
  C[x] = 0
  for (y,0,600) {
    C[x] = C[x] + A[x,y]*B[x,y]
  }
}

    Is that possible? Thanks!

  4. tqchen commented on Oct 6, 2017

    @tqchen
    tqchen
    on Oct 6, 2017
    Member

    This requires an update of the reduction operator implementation, which is doable but not supported so far. In general we find that zero initialization makes it easier to support reductions of multiple levels and tiling patterns. So brings more benefit than a peeled version of reduction. If you have evidence that the peeled version works better, we can work to bring this into tvm

  5. derisavi commented on Oct 13, 2017

    @derisavi
    derisavi
    on Oct 13, 2017
    ContributorAuthor

    I don't think it would be bring any measurable benefit in CPU or GPU since the zero initialization happens in registers or the first level cache. However, if the region to be initialized is large (a fairly large tensor) or the memory to be initialized is expensive to update, peeling the first iteration (and therefore, avoiding the zero initialization) could be beneficial.
    For now, this is not on my critical path so I close this issue.

  6. derisavi
    closed this as completedon Oct 13, 2017
  7. tqchen
    added 2 commits that reference this issue on May 26, 2018

    Expose clip to frontend mxnet (apache#512)

    8a573bf

    Expose clip to frontend mxnet (#512)

    e4deaad
  8. tqchen
    added a commit that references this issue on Jul 6, 2018

    Expose clip to frontend mxnet (apache#512)

    31edf3f
  9. sergei-mironov
    added a commit that references this issue on Aug 8, 2018

    Expose clip to frontend mxnet (apache#512)

    5a84bd7
  10. yzh119
    added a commit that references this issue on Nov 29, 2021

    [Meta Schedule] Add Helper Function & Minor Modification (apache#512)

    4e1c53a
  11. zxybazh
    added 4 commits that reference this issue on Jan 7, 2022

    Squashed commit

    f98a8e9

    Squashed commit

    d1fce89

    Squashed commit

    49915c2

    Squashed commit

    05f3aba

  12. 3 remaining items

  13. Hzfengsy
    added 2 commits that reference this issue on Feb 19, 2022

    Squashed commit

    17018cc

    Squashed commit

    623c0a6
  14. junrushao
    added 3 commits that reference this issue on Feb 20, 2022

    Squashed commit

    1d8cc54

    Squashed commit

    893555f

    Squashed commit

    46a98f1
  15. zxybazh
    added 5 commits that reference this issue on Feb 21, 2022

    Squashed commit

    eb8def7

    Squashed commit

    6f04005

    Squashed commit

    ed92495

    Squashed commit

    fc2f258

    Squashed commit

    0af0f71
  16. tlopex
    added a commit that references this issue on Mar 9, 2026

    fix(infra): align tirx pre-commit handling with apache/main (apache#512)

    060fd3a
  17. tqchen
    added 2 commits that reference this issue on Mar 26, 2026

    [FFI] Bump tvm-ffi to 63224e3 and fix regressions

    5caddee

    [FFI] Bump tvm-ffi to 63224e3 and fix regressions (#18938)

    44dbd13
  18. tqchen
    added 2 commits that reference this issue on Oct 16, 2026

    [FFI] Bump tvm-ffi to 63224e3 and fix regressions (#18938)

    02b80e4

    [FFI] Bump tvm-ffi to 63224e3 and fix regressions (#18938)

    765475e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions