Skip to content

[Pass Bug] pto-insert-sync primes both multi_tile slots for a preload + prefetch loop (design doc §7.2): wrong data with an even trip count, hang with an odd one #1519

Description

@Hzfengsy

Pass / pipeline name

pto-insert-sync (--enable-insert-sync): per-slot event priming/draining for pto.multi_tile_get back-edge dependencies

Command line

ptoas double_prefetch.pto -o double_prefetch.cpp --enable-insert-sync --pto-level=level2 --pto-arch=a3

ptoas 0.61 release binary (ptoas-bin-aarch64.tar.gz).

Before IR (input)

The "双 buffer prefetch(动态 slot)" example from docs/designs/ptoas-multi-buffer-explicit-design.md §7.2. Only the elided outs(...) of pto.tadd is filled in (a separate tile, as multi_tile_same_region_two_gets_sync.pto does), and the GM argument uses the lit-test partition_tensor_view form:

module {
  func.func @double_prefetch(%gm : !pto.partition_tensor_view<16x16xf16>, %n : index) {
    %c0 = arith.constant 0 : index
    %c1 = arith.constant 1 : index
    %c2 = arith.constant 2 : index

    %mb = pto.alloc_multi_tile : !pto.multi_tile_buf<vec, 16x16xf16, count=2>
    %acc = pto.alloc_tile : !pto.tile_buf<vec, 16x16xf16>

    // preload iter 0 -> slot0
    %pre = pto.multi_tile_get %mb[%c0] : !pto.multi_tile_buf<vec, 16x16xf16, count=2> -> !pto.tile_buf<vec, 16x16xf16>
    pto.tload ins(%gm : !pto.partition_tensor_view<16x16xf16>) outs(%pre : !pto.tile_buf<vec, 16x16xf16>)

    scf.for %i = %c0 to %n step %c1 {
      %next     = arith.addi  %i,    %c1 : index
      %cur_idx  = arith.remui %i,    %c2 : index
      %next_idx = arith.remui %next, %c2 : index

      // prefetch into the other slot
      %s_next = pto.multi_tile_get %mb[%next_idx] : !pto.multi_tile_buf<vec, 16x16xf16, count=2> -> !pto.tile_buf<vec, 16x16xf16>
      pto.tload ins(%gm : !pto.partition_tensor_view<16x16xf16>) outs(%s_next : !pto.tile_buf<vec, 16x16xf16>)

      %s_cur = pto.multi_tile_get %mb[%cur_idx] : !pto.multi_tile_buf<vec, 16x16xf16, count=2> -> !pto.tile_buf<vec, 16x16xf16>
      pto.tadd ins(%s_cur, %s_cur : !pto.tile_buf<vec, 16x16xf16>, !pto.tile_buf<vec, 16x16xf16>) outs(%acc : !pto.tile_buf<vec, 16x16xf16>)
    }
    return
  }
}

Device-runnable version: the same loop and op order, plus per-chunk GM offsets and a tstore so the output can be checked. a has one extra chunk so the last prefetch stays in bounds. 8 chunks of 32x128 f32, out = 2 * a:

x2_kernel.pto
module attributes {pto.target_arch = "a2a3"} {
  func.func @x2_kernel(%arg0: !pto.ptr<f32>, %arg1: !pto.ptr<f32>) attributes {pto.kernel_kind = #pto.kernel_kind<vector>} {
  %c0 = arith.constant 0 : index
  %c1 = arith.constant 1 : index
  %c2 = arith.constant 2 : index
  %c32 = arith.constant 32 : index
  %c128 = arith.constant 128 : index
  %cN = arith.constant 8 : index
  %cWa = arith.constant 1152 : index
  %cWo = arith.constant 1024 : index
  %out_view = pto.make_tensor_view %arg0, shape = [%c32, %cWo], strides = [%cWo, %c1] {layout = #pto.layout<nd>} : !pto.tensor_view<?x?xf32>
  %a_view = pto.make_tensor_view %arg1, shape = [%c32, %cWa], strides = [%cWa, %c1] {layout = #pto.layout<nd>} : !pto.tensor_view<?x?xf32>
  %mb = pto.alloc_multi_tile valid_row = %c32 valid_col = %c128 : !pto.multi_tile_buf<!pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=128, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>, count=2>
  // preload iter 0 -> slot0
  %pre = pto.multi_tile_get %mb[%c0] : !pto.multi_tile_buf<!pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=128, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>, count=2> -> !pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=128, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>
  %av_pre = pto.partition_view %a_view, offsets = [%c0, %c0], sizes = [%c32, %c128] : !pto.tensor_view<?x?xf32> -> !pto.partition_tensor_view<32x128xf32>
  pto.tload ins(%av_pre : !pto.partition_tensor_view<32x128xf32>) outs(%pre : !pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=128, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>)
  scf.for %i = %c0 to %cN step %c1 {
    %next     = arith.addi  %i,    %c1 : index
    %cur_idx  = arith.remui %i,    %c2 : index
    %next_idx = arith.remui %next, %c2 : index
    // prefetch chunk i+1 into the other slot
    %s_next = pto.multi_tile_get %mb[%next_idx] : !pto.multi_tile_buf<!pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=128, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>, count=2> -> !pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=128, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>
    %aoff = arith.muli %next, %c128 : index
    %av = pto.partition_view %a_view, offsets = [%c0, %aoff], sizes = [%c32, %c128] : !pto.tensor_view<?x?xf32> -> !pto.partition_tensor_view<32x128xf32>
    pto.tload ins(%av : !pto.partition_tensor_view<32x128xf32>) outs(%s_next : !pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=128, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>)
    // compute chunk i from the current slot, then store it
    %s_cur = pto.multi_tile_get %mb[%cur_idx] : !pto.multi_tile_buf<!pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=128, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>, count=2> -> !pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=128, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>
    %y = pto.alloc_tile valid_row = %c32 valid_col = %c128 : !pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=128, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>
    pto.tadd ins(%s_cur, %s_cur : !pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=128, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>, !pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=128, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>) outs(%y : !pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=128, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>)
    %ooff = arith.muli %i, %c128 : index
    %ov = pto.partition_view %out_view, offsets = [%c0, %ooff], sizes = [%c32, %c128] : !pto.tensor_view<?x?xf32> -> !pto.partition_tensor_view<32x128xf32>
    pto.tstore ins(%y : !pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=128, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>) outs(%ov : !pto.partition_tensor_view<32x128xf32>)
  }
  return
  }
}

Expected behavior / expected IR (after)

The kernel computes out[:, 128*i : 128*(i+1)] = 2 * a[:, 128*i : 128*(i+1)] for every chunk and terminates for any trip count.

For that, the V->MTE2 back-edge events need to be primed and drained per slot according to which slot the loop's producer and consumer actually touch. The WAR edge runs from tadd in iteration i (reads slot i % 2) to tload in iteration i + 1 (writes slot (i + 2) % 2 == i % 2), so:

  • only slot 1's event should start primed: the first in-loop write (i = 0) goes to slot 1, which nothing has read yet;
  • slot 0's first in-loop write (i = 1) has to wait for tadd of iteration 0. The preload already wrote slot 0 without consuming a token, so priming slot 0 as well hands it a spare token;
  • the post-loop drain should wait only on releases that are still outstanding.

Actual IR or error

Emitted C++ for the §7.2 example: both slots' events are primed before the loop, the preload TLOAD consumes no V->MTE2 token, and both are drained after the loop (v4 = 0 / v3 = 512 are the slot addresses and v6 = 0, v7 = 1, v8 = 2, so the V->MTE2 event id equals the slot index):

  set_flag(PIPE_V, PIPE_MTE2, EVENT_ID0);
  set_flag(PIPE_V, PIPE_MTE2, EVENT_ID1);
  set_flag(PIPE_MTE2, PIPE_V, EVENT_ID1);
  set_flag(PIPE_MTE2, PIPE_V, EVENT_ID2);
  TLOAD(v11, v1);
  set_flag(PIPE_MTE2, PIPE_V, EVENT_ID0);
  wait_flag(PIPE_MTE2, PIPE_V, EVENT_ID0);
  for (int64_t v13 = v6; v13 < v2; v13 += v7) {
    int64_t v14 = (int64_t) ((uint64_t) v13 % (uint64_t) v8);
    int64_t v15 = (int64_t) ((uint64_t) ((int64_t) (uint64_t) v13 + (uint64_t) v7) % (uint64_t) v8);
    int64_t v16 = v15 == v7 ? v3 : v4;
    uint64_t v18 = (uint64_t) v16;
    TASSIGN(v17, v18);
    bool v19 = (int64_t) ((uint64_t) v15 % (uint64_t) v8) == v7;
    int64_t v20 = v19 ? v7 : v6;
    event_t v21 = (event_t) v20;
    wait_flag(PIPE_V, PIPE_MTE2, v21);
    TLOAD(v17, v1);
    int64_t v22 = v19 ? v8 : v7;
    event_t v23 = (event_t) v22;
    set_flag(PIPE_MTE2, PIPE_V, v23);
    int64_t v24 = v14 == v7 ? v3 : v4;
    uint64_t v26 = (uint64_t) v24;
    TASSIGN(v25, v26);
    bool v27 = (int64_t) ((uint64_t) v14 % (uint64_t) v8) == v7;
    int64_t v28 = v27 ? v8 : v7;
    event_t v29 = (event_t) v28;
    wait_flag(PIPE_MTE2, PIPE_V, v29);
    pipe_barrier(PIPE_V);
    TADD(v9, v25, v25);
    int64_t v30 = v27 ? v7 : v6;
    event_t v31 = (event_t) v30;
    set_flag(PIPE_V, PIPE_MTE2, v31);
  }
  wait_flag(PIPE_V, PIPE_MTE2, EVENT_ID0);
  wait_flag(PIPE_V, PIPE_MTE2, EVENT_ID1);
  wait_flag(PIPE_MTE2, PIPE_V, EVENT_ID1);
  wait_flag(PIPE_MTE2, PIPE_V, EVENT_ID2);

V->MTE2 tokens over n loop iterations:

Event Primed set_flag in loop (after TADD of that slot) wait_flag in loop (before TLOAD into that slot) Drained
EVENT_ID0 (slot 0) 1 i even: ceil(n/2) i odd: floor(n/2) 1
EVENT_ID1 (slot 1) 1 i odd: floor(n/2) i even: ceil(n/2) 1

Slot 0 always holds one token more than the dependency allows, so every TLOAD into slot 0 can start before the TADD still reading that slot has finished. For odd n, slot 1's drain has no matching set_flag and never returns.

On an Ascend 910B1 (a2a3), with a random f32, out starting at zero, and the result compared against 2 * a:

Kernel Loop trips Result
Device-runnable §7.2 loop above 8 finishes; out wrong in chunks 0, 2, 4, 6 (9856 / 32768 elements, max abs diff 12.9)
Same allocation, one pto.multi_tile_get %mb[%i mod 2] per iteration (load and compute on the same slot) 8 correct
§7.2 loop plus an epilogue (loop runs to N-1, last chunk computed after the loop), slot index as arith.remui, arith.remsi or a loop-carried k -> 1-k 8 finishes; wrong in chunks 0, 2, 4, 6
Same prologue/epilogue kernel, slot index as arith.remsi, arith.andi %i, 1, (3*i) mod 2 or a loop-carried k -> 1-k 7 hangs; the runtime reports SCHEDULER_TIMEOUT sub_class=S1:running-stalled (the AICore task never completes)

The wrong chunks are exactly those computed from slot 0 inside the loop (chunk i is read from slot i % 2).

Git commit

5eea919 (tag v0.61)

Target Ascend arch (if relevant)

a3

PTOAS build level (if relevant)

level2

Additional context

  • Hardware: Ascend 910B1 (driver 26.0.rc1), CANN 9.0.0. The kernel was built from the ptoas output and launched through the PyPTO runtime (simpler) on a2a3. The CPU simulator stubs out set_flag / wait_flag, so it does not reproduce this.
  • The same priming and draining is emitted at --pto-level=level3 (pto.alloc_multi_tile addr = ...), checked on the prologue/epilogue variant. Slot indices written as arith.remsi, arith.andi, (3*i) mod 2 or a loop-carried value get a single static MTE2->V event instead of per-slot ones, but their V->MTE2 accounting is identical.
  • The MTE2->V side has the same shape: EVENT_ID1 and EVENT_ID2 (slots 0 and 1) are both primed, so iteration 1's wait_flag(PIPE_MTE2, PIPE_V, <slot 1>) can pass before iteration 0's prefetch of chunk 1 has landed. It did not corrupt data in these runs.
  • No v0.61 lit test combines a preload with two gets per iteration. multi_tile_affine_disjoint_slots.pto (no preload) FileChecks that both slots' events are primed and drained, which is the accounting above.

Related: #1118
Related: #1106

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions