TLOAD

Instruction Diagram

Introduction

Loads data from GlobalTensor (GM) to a tile.

Mathematical Semantics

The symbolic representation depends on the shape/stride of GlobalTensor and the layout of Tile. Conceptually (two-dimensional view, with base offsets r_0 and c_0):

$\mathrm{dst}_{i,j} = \mathrm{src}_{r_0 + i,\; c_0 + j}$

Assembly Syntax

Synchronous form:

%t0 = tload %sv[%c0, %c0] : (!pto.memref<...>, index, index) -> !pto.tile<...>

AS Level 1 (SSA)

%dst = pto.tload %mem : !pto.partition_tensor_view<MxNxdtype> ->
!pto.tile<loc, dtype, rows, cols, blayout, slayout, fractal, pad>

AS Level 2 (DPS)

pto.tload ins(%mem : !pto.partition_tensor_view<MxNxdtype>) outs(%dst : !pto.tile_buf<...>)

C++ Built-in APIs

Declared in include/pto/common/pto_instr.hpp:

The public include header is <pto/pto-inst.hpp>, and the internal declaration is located in pto/common/pto_instr.hpp.

template <typename TileData, typename GlobalData, typename... WaitEvents>
PTO_INST RecordEvent TLOAD(TileData &dst, GlobalData &src, WaitEvents &... events);

Constraints

Implementation check (Atlas A2/A3 training products/Atlas A2/A3 inference products):
- TileData::DType must be one of the following: int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float.
- The destination tile position must be TileType::Vec or TileType::Mat.
- sizeof(TileData::DType) == sizeof(GlobalData::DType).
- Runtime: all src.GetShape(dim) values and dst.GetValidRow()/GetValidCol() must be > 0.
- TileType::Vec loads support only matching layouts: ND->ND, DN->DN, NZ->NZ.
- TileType::Mat loads support: ND->ND, DN->DN, NZ->NZ, as well as ND->NZ and DN->ZN.
- For ND->NZ or DN->ZN: GlobalData::staticShape[0..2] == 1 and TileData::SFractalSize == 512.
- For int64_t/uint64_t, only ND->ND or DN->DN is supported.
- Vec tile (UB path): 1 <= TileData::Rows <= 4095.
- Mat tile (L1 path): 1 <= TileData::Rows <= 16384.
Implementation check (Ascend 950PR/Ascend 950DT):
- sizeof(TileData::DType) must be 1, 2, 4, or 8 bytes, and must match sizeof(GlobalData::DType).
- For int64_t/uint64_t, TileData::PadVal must be PadValue::Null or PadValue::Zero.
- TileType::Vec loading requires one of the following layout pairs:
- ND uses row-major + SLayout::NoneBox (ND->ND),
- DN uses column-major + SLayout::NoneBox (DN->DN),
- NZ uses SLayout::RowMajor (NZ->NZ).
- For row-major ND->ND with compile-time known shapes, TileData::ValidCol must equal GlobalData::staticShape[4], and TileData::ValidRow must equal the product of GlobalData::staticShape[0..3].
- TileType::Mat loads are also subject to TLoadCubeCheck constraints (for example, only specific ND/DN/NZ conversions and L1 size limits).
- TileType::Mat loads also handle mx-format loads, including MX_A_ZZ/MX_A_ND/MX_A_DN to ZZ (for scalarA) and MX_B_NN/MX_B_ND/MX_B_DN to NN (for scalarB).
- For MX_A_ZZ/MX_B_NN: (GlobalData::staticShape[3] == 16 || GlobalData::staticShape[3] == -1) and (GlobalData::staticShape[4] == 2 || GlobalData::staticShape[4] == -1).
- For MX_A_ND/MX_A_DN/MX_B_ND/MX_B_DN: (GlobalData::staticShape[0] == 1 || GlobalData::staticShape[0] == -1) and (GlobalData::staticShape[1] == 1 || GlobalData::staticShape[1] == -1) and (GlobalData::staticShape[4] == 2 || GlobalData::staticShape[4] == -1).
- For scaleA, dst.GetValidCol() % 2 == 0.
- For scaleB, dst.GetValidRow() % 2 == 0.
Valid region:
- The implementation uses dst.GetValidRow()/dst.GetValidCol() as the transfer size.

Examples

Automatic

#include <pto/pto-inst.hpp>

using namespace pto;

template <typename T>
void example_auto(__gm__ T* in) {
  using TileT = Tile<TileType::Vec, T, 16, 16>;
  using GShape = Shape<1, 1, 1, 16, 16>;
  using GStride = BaseShape2D<T, 16, 16, Layout::ND>;
  using GTensor = GlobalTensor<T, GShape, GStride, Layout::ND>;

  GTensor gin(in);
  TileT t;
  TLOAD(t, gin);
}

Manual

#include <pto/pto-inst.hpp>

using namespace pto;

template <typename T>
void example_manual(__gm__ T* in) {
  using TileT = Tile<TileType::Vec, T, 16, 16>;
  using GShape = Shape<1, 1, 1, 16, 16>;
  using GStride = BaseShape2D<T, 16, 16, Layout::ND>;
  using GTensor = GlobalTensor<T, GShape, GStride, Layout::ND>;

  GTensor gin(in);
  TileT t;
  TASSIGN(t, 0x1000);
  TLOAD(t, gin);
}

ASM Examples

Automatic Mode

# Automatic mode: the compiler/runtime handles resource placement and scheduling.
%dst = pto.tload %mem : !pto.partition_tensor_view<MxNxdtype> ->

Manual Mode

# Manual mode: explicitly bind resources first, then issue the instruction.
# Optional (when the instruction contains a tile operand):
# pto.tassign %arg0, @tile(0x1000)
# pto.tassign %arg1, @tile(0x2000)
%dst = pto.tload %mem : !pto.partition_tensor_view<MxNxdtype> ->

PTO Assembly Form

%t0 = tload %sv[%c0, %c0] : (!pto.memref<...>, index, index) -> !pto.tile<...>
# AS Level 2 (DPS)
pto.tload ins(%mem : !pto.partition_tensor_view<MxNxdtype>) outs(%dst : !pto.tile_buf<...>)