Contiguous Aligned Data Move-in

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	x
Atlas A2 training product/Atlas A2 inference product	x
Atlas 200I/500 A2 inference product	x
Atlas inference product AI Core	x
Atlas inference product Vector Core	x
Atlas training product	x

Function Usage

Reg vector computation data movement interface, used to move contiguous aligned data from UB to RegTensor. In the single move-in mode, data can be moved from UB to one destination register. In the double move-in mode, data can be moved from UB to two destination registers.

Prototype

// Under the POST_MODE_NORMAL scenario, the single move-in mode is utilized.
template <typename T = DefaultType, LoadDist dist = LoadDist::DIST_NORM, typename U>
__simd_callee__ inline void LoadAlign(U& dstReg, __ubuf__ T* srcAddr);

// Under the POST_MODE_UPDATE scenario, the single move-in mode is utilized.
template <typename T = DefaultType, PostLiteral postMode, LoadDist dist = LoadDist::DIST_NORM, typename U>
__simd_callee__ inline void LoadAlign(U& dstReg, __ubuf__ T*& srcAddr, int32_t postUpdateStride);

// Use AddrReg to store the offset in the single move-in mode.
template <typename T = DefaultType, LoadDist dist = LoadDist::DIST_NORM, typename U>
__simd_callee__ inline void LoadAlign(U& dstReg, __ubuf__ T* srcAddr, AddrReg offset);

// Under the POST_MODE_NORMAL scenario, the double move-in mode is utilized.
template <typename T = DefaultType, LoadDist dist, typename U>
__simd_callee__ inline void LoadAlign(U& dstReg0, U& dstReg1, __ubuf__ T* srcAddr);

// Under the POST_MODE_UPDATE scenario, the double move-in mode is utilized.
template <typename T = DefaultType, PostLiteral postMode, LoadDist dist, typename U>
__simd_callee__ inline void LoadAlign(U& dstReg0, U& dstReg1, __ubuf__ T*& srcAddr, int32_t postUpdateStride);

// Use AddrReg to store the offset in the double move-in mode.
template <typename T = DefaultType, LoadDist dist, typename U>
__simd_callee__ inline void LoadAlign(U& dstReg0, U& dstReg1, __ubuf__ T* srcAddr, AddrReg offset);

Parameters

**Table 1** LoadDist template parameters (single move-in mode)
LoadDist Value	Description	Movement Alignment Constraint (Byte)
DIST_NORM	Normal mode. VL data is moved.	32
DIST_BRC_B8	A piece of data of b8 type is moved and broadcast to all element positions.	1
DIST_BRC_B16	A piece of data of b16 type is moved and broadcast to all element positions.	2
DIST_BRC_B32	A piece of data of b32 type is moved and broadcast to all element positions.	4
DIST_US_B8	Data is upsampled by a factor of 2, and VL/2 data elements are loaded. Each input element is repeated twice, and the data type is b8.	min(32, VL/2)
DIST_US_B16	Data is upsampled by a factor of 2, and VL/2 data elements are loaded. Each input element is repeated twice, and the data type is b16.	min(32, VL/2)
DIST_DS_B8	Data is downsampled by a factor of 2, and 2x VL data is loaded. Every other data element is retained, and the data type is b8.	32
DIST_DS_B16	Data is downsampled by a factor of 2, and 2x VL data is loaded. Every other data element is retained, and the data type is b16.	32
DIST_UNPACK_B8	Unpack mode: Load VL/2 elements of u8 data, unpack to u16 format with a length of VL, and insert zeros in the middle positions. Example: VL = 256B: src: [0x00, 0x01, 0x02, 0x03, ..., 0xFF, ...] dst:[0x0000, 0x0001, ..., 0x0007F]	min(32, VL/2)
DIST_UNPACK_B16	Unpack mode: Load VL/2 elements of u16 data, unpack to u32 format with a length of VL, and insert zeros in the middle positions. Example: VL = 256B: src: [0x0000, 0x0001, 0x0002, 0x0003, ..., 0x007F, ...] dst:[0x00000000, 0x00000001, ..., 0x000003F]	min(32, VL/2)
DIST_BLK	A data block (32 bytes) is read and sent to the VL.	32
DIST_E2B_B16	Data is loaded from (VL/Data block) B, and each 16-bit element is broadcast into a 32-byte data block.	VL/16
DIST_E2B_B32	Data is loaded from (VL/Data block) B, and each 32-bit element is broadcast into a 32-byte data block.	VL/8
DIST_UNPACK_B32	Unpack mode: Load VL/2 elements of u32 data, unpack to u64 format with a length of VL, and insert zeros in the middle positions. Example: VL = 256B: src: [0x00000000, 0x00000001, 0x00000002, ..., 0x000003F, ...] dst:[0x0000000000000000, 0x0000000000000001, ..., 0x000000000000001F]	min(32, VL/2)
DIST_UNPACK4_B8	Unpack mode: Load VL/4 elements of u8 data, unpack to u32 format with a length of VL, and insert zeros in the middle positions. Example: VL = 256B: src: [0x00, 0x01, 0x02, 0x03, ..., 0xFF, ...] dst:[0x00000000, 0x00000001, ..., 0x000003F]	min(32, VL/4)

**Table 2** LoadDist template parameters (double move-in mode)
LoadDist Value	Description	Movement Alignment Constraint (Byte)
DIST_DINTLV_B8	In the double move-in mode, perform element-based interleaved movement. Read data of length 2 x VL from src, store the elements with even indexes to dst0, and store the elements with odd indexes to dst1. The data type is b8.	32
DIST_DINTLV_B16	In the double move-in mode, perform element-based interleaved movement. Read data of length 2 x VL from src, store the elements with even indexes to dst0, and store the elements with odd indexes to dst1. The data type is b16.	32
DIST_DINTLV_B32	In the double move-in mode, perform element-based interleaved movement. Read data of length 2 x VL from src, store the elements with even indexes to dst0, and store the elements with odd indexes to dst1. The data type is b32.	32

**Table 3** Parameters in the POST_MODE_NORMAL scenario in the single move-in mode
Parameter	Input/Output	Description
T	Input	Operand data type. The supported data types are b8, b16, b32, and b64.
dist	Input	Movement mode, which is of the LoadDist type. For details about the value, see Table 1.
U	Input	RegTensor type of the destination operand, for example, RegTensor<half>. It is automatically inferred by the compiler and does not need to be specified.
dstReg	Output	Destination operand, which is of the RegTensor type.
srcAddr	Input	Start address of the source operand in the UB.

**Table 4** Parameters in the POST_MODE_UPDATE scenario in the single move-in mode
Parameter	Input/Output	Description
T	Input	Template parameter. The supported data types are b8, b16, b32, and b64.
postMode	Input	This parameter controls whether to enable post update. The value is of the PostLiteral type.
dist	Input	Movement mode, which is of the LoadDist type. For details about the value, see Table 1.
U	Input	RegTensor type of the destination operand, for example, RegTensor<half>. It is automatically inferred by the compiler and does not need to be specified.
dstReg	Output	Destination operand, which is of the RegTensor type.
srcAddr	Input/Output	Start address of the source operand in the UB.
postUpdateStride	Input	The actual start address for moving data to the UB is srcAddr. After the movement, the address is updated as follows: srcAddr += postUpdateStride.

**Table 5** Parameters for using AddrReg to store the offset in the single move-in mode
Parameter	Input/Output	Description
T	Input	Template parameter. The supported data types are b8, b16, b32, and b64.
dist	Input	Movement mode, which is of the LoadDist type. For details about the value, see Table 1.
U	Input	RegTensor type of the destination operand, for example, RegTensor<half>. It is automatically inferred by the compiler and does not need to be specified.
dstReg	Output	Destination operand, which is of the RegTensor type.
offset	Input	The actual movement address UB is computed by adding the source address (srcAddr) and the offset.
srcAddr	Input/Output	Start address of the source operand in the UB.

**Table 6** Parameters in the POST_MODE_NORMAL scenario in the double move-in mode
Parameter	Input/Output	Description
T	Input	Template parameter. The supported data types are b8, b16, and b32.
dist	Input	Movement mode, which is of the LoadDist type. For details about the value, see Table 2.
U	Input	RegTensor type of the destination operand, for example, RegTensor<half>. It is automatically inferred by the compiler and does not need to be specified.
dstReg0	Output	The first destination operand, which is of the RegTensor type.
dstReg1	Output	The second destination operand, which is of the RegTensor type.
srcAddr	Input/Output	Start address of the source operand in the UB.

**Table 7** Parameters in the POST_MODE_UPDATE scenario in the double move-in mode
Parameter	Input/Output	Description
T	Input	Template parameter. The supported data types are b8, b16, b32, and b64.
postMode	Input	This parameter controls whether to enable post update. The value is of the PostLiteral type.
dist	Input	Movement mode, which is of the LoadDist type. For details about the value, see Table 2.
U	Input	RegTensor type of the destination operand, for example, RegTensor<half>. It is automatically inferred by the compiler and does not need to be specified.
dstReg0	Output	The first destination operand, which is of the RegTensor type.
dstReg1	Output	The second destination operand, which is of the RegTensor type.
srcAddr	Input/Output	Start address of the source operand in the UB.
postUpdateStride	Input	The actual start address for moving data to the UB is srcAddr. After the movement, the address is updated as follows: srcAddr += postUpdateStride.

**Table 8** Parameters for using AddrReg to store the offset in the double move-in mode
Parameter	Input/Output	Description
T	Input	Template parameter. The supported data types are b8, b16, and b32.
dist	Input	Movement mode, which is of the LoadDist type. For details about the value, see Table 2.
U	Input	RegTensor type of the destination operand, for example, RegTensor<half>. It is automatically inferred by the compiler and does not need to be specified.
dstReg0	Output	The first destination operand, which is of the RegTensor type.
dstReg1	Output	The second destination operand, which is of the RegTensor type.
offset	Input	The actual movement address UB is computed by adding the source address (srcAddr) and the offset.
srcAddr	Input/Output	Start address of the source operand in the UB.

Returns

None

Restrictions

The b64 data type supports only the DIST_NORM mode in LoadDist.

Examples

// Use POST_MODE_NORMAL in the single move-in or single move-out mode.
__simd_vf__ inline void ComputeMode01(__ubuf__ T* dstAddr, __ubuf__ T* srcAddr, uint32_t dstSize,
    uint32_t oneRepeatSize, uint16_t repeatTimes)
{
    AscendC::Reg::RegTensor<T> dstReg;
    AscendC::Reg::MaskReg mask;
    for (uint16_t i = 0; i < repeatTimes; ++i) {
        mask = AscendC::Reg::UpdateMask<T>(dstSize);
        AscendC::Reg::LoadAlign(dstReg, srcAddr + i * oneRepeatSize);
        AscendC::Reg::StoreAlign(dstAddr + i * oneRepeatSize, dstReg, mask);
    }
}

// Use POST_MODE_UPDATE in the single move-in or single move-out mode.
__simd_vf__ inline void ComputeMode02(__ubuf__ T* dstAddr, __ubuf__ T* srcAddr, uint32_t dstSize,
    uint32_t oneRepeatSize, uint16_t repeatTimes)
{
    AscendC::Reg::RegTensor<T> dstReg;
    AscendC::Reg::MaskReg mask;
    for (uint16_t i = 0; i < repeatTimes; ++i) {
        mask = AscendC::Reg::UpdateMask<T>(dstSize);
        AscendC::Reg::LoadAlign<T, AscendC::Reg::PostLiteral::POST_MODE_UPDATE>(dstReg, srcAddr, oneRepeatSize);
        AscendC::Reg::StoreAlign<T, AscendC::Reg::PostLiteral::POST_MODE_UPDATE>(dstAddr, dstReg, oneRepeatSize, mask);
    }
}

// Use AddrReg to store the offset in the single move-in or single move-out mode.
__simd_vf__ inline void ComputeMode03(__ubuf__ T* dstAddr, __ubuf__ T* srcAddr, uint32_t oneRepeatSize, uint16_t repeatTimes)
{
    AscendC::Reg::RegTensor<T> dstReg;
    AscendC::Reg::MaskReg mask = AscendC::Reg::CreateMask<T>();
    AscendC::Reg::AddrReg aReg;
    for (uint16_t i = 0; i < repeatTimes; ++i) {
        aReg = AscendC::Reg::CreateAddrReg<T>(i, oneRepeatSize);
        AscendC::Reg::LoadAlign(dstReg, srcAddr, aReg);
        AscendC::Reg::StoreAlign(dstAddr, dstReg, aReg, mask);
    }
}

// Use POST_MODE_NORMAL in the double move-in or double move-out mode.
__simd_vf__ inline void ComputeMode04(__ubuf__ T* dstAddr, __ubuf__ T* srcAddr, uint32_t oneRepeatSize,
    uint16_t repeatTimes)
{
    AscendC::Reg::RegTensor<T> srcReg0;
    AscendC::Reg::RegTensor<T> srcReg1;
    AscendC::Reg::MaskReg mask = AscendC::Reg::CreateMask<uint8_t, AscendC::Reg::MaskPattern::ALL>();
    for (uint16_t i = 0; i < repeatTimes; ++i) {
        AscendC::Reg::LoadAlign<T, AscendC::Reg::LoadDist::DIST_DINTLV_B8>(srcReg0, srcReg1, srcAddr + i * oneRepeatSize);
        AscendC::Reg::StoreAlign<T, AscendC::Reg::StoreDist::DIST_INTLV_B8>(dstAddr + i * oneRepeatSize, srcReg0, srcReg1, mask);
    }
}

// Use AddrReg to store the offset in the double move-in or double move-out mode.
__simd_vf__ inline void ComputeMode05(__ubuf__ T* dstAddr, __ubuf__ T* srcAddr, uint32_t oneRepeatSize, uint16_t repeatTimes)
{
    AscendC::Reg::RegTensor<T> srcReg0;
    AscendC::Reg::RegTensor<T> srcReg1;
    AscendC::Reg::MaskReg mask = AscendC::Reg::CreateMask<T>();
    AscendC::Reg::AddrReg aReg;
    for (uint16_t i = 0; i < repeatTimes; ++i) {
        aReg = AscendC::Reg::CreateAddrReg<T>(i, oneRepeatSize);
        AscendC::Reg::LoadAlign<T, AscendC::Reg::LoadDist::DIST_DINTLV_B8>(srcReg0, srcReg1, srcAddr, aReg);
        AscendC::Reg::StoreAlign<T, AscendC::Reg::StoreDist::DIST_INTLV_B8>(dstAddr, srcReg0, srcReg1, aReg, mask);
    }
}

Parent topic: Reg Data Movement