Contiguous Aligned Data Move-out

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	x
Atlas A2 training product/Atlas A2 inference product	x
Atlas 200I/500 A2 inference product	x
Atlas inference product AI Core	x
Atlas inference product Vector Core	x
Atlas training product	x

Function Usage

Indicates the Reg vector computation data movement interface, and is used to move contiguous aligned data from RegTensor to UB.

Prototype

// Under the POST_MODE_NORMAL scenario, use the single move-out mode.
template <typename T = DefaultType, StoreDist dist = StoreDist::DIST_NORM, typename U>
__simd_callee__ inline void StoreAlign(__ubuf__ T* dstAddr, U& srcReg, MaskReg& mask);

// Under the POST_MODE_UPDATE scenario, use the single move-out mode.
template <typename T = DefaultType, PostLiteral postMode, StoreDist dist = StoreDist::DIST_NORM, typename U>
__simd_callee__ inline void StoreAlign(__ubuf__ T*& dstAddr, U& srcReg, int32_t postUpdateStride, MaskReg& mask);

// Use AddrReg to store the offset in the single move-out mode.
template <typename T = DefaultType, StoreDist dist = StoreDist::DIST_NORM, typename U>
__simd_callee__ inline void StoreAlign(__ubuf__ T* dstAddr, U& srcReg, AddrReg offset, MaskReg& mask);

// Under the POST_MODE_NORMAL scenario, use the double move-out mode.
template <typename T = DefaultType, StoreDist dist, typename U>
__simd_callee__ inline void StoreAlign(__ubuf__ T* dstAddr, U& srcReg0, U& srcReg1, MaskReg& mask);

// Use AddrReg to store the offset in the double move-out mode.
template <typename T = DefaultType, StoreDist dist, typename U>
__simd_callee__ inline void StoreAlign(__ubuf__ T* dstAddr, U& srcReg0, U& srcReg1, AddrReg offset, MaskReg& mask);

Parameters

**Table 1** StoreDist template parameters (single move-out mode)
StoreDist	Description	Alignment Constraint (Byte)
DIST_NORM_B8	The normal mode is used, and VL data of the b8 type is moved.	32
DIST_NORM_B16	The normal mode is used, and VL data of the b16 type is moved.	32
DIST_NORM_B32	The normal mode is used, and VL data of the b32 type is moved.	32
DIST_FIRST_ELEMENT_B8	The mask is ignored, and the first element of src is moved to dst. The data type is b8.	1
DIST_FIRST_ELEMENT_B16	The mask is ignored, and the first element of src is moved to dst. The data type is b16.	2
DIST_FIRST_ELEMENT_B32	The mask is ignored, and the first element of src is moved to dst. The data type is b32.	4
DIST_PACK_B16	In pack mode, the data type is b16. The lower half of the bits of valid elements in src are contiguously stored in dst based on the mask. For example, if the data type is uint16_t, the mask is configured to indicate that all elements are valid, and the length of dst must be VL/2, the execution result is as follows: src: [0x3210, 0x7654, 0xBA98, 0xFEDC, ..., 0xFEDC, 0xBA98, 0x7654, 0x3210] dst: [0x5410, 0xDC98, ... 0x98DC, 0x1054]	min(32, VL/2)
DIST_PACK_B32	In pack mode, the data type is b32. The lower half of the bits of valid elements in src are contiguously stored in dst based on the mask.	min(32, VL/2)
DIST_PACK_B64	In pack mode, the data type is b64. The lower half of the bits of valid elements in src are contiguously stored in dst based on the mask.	min(32, VL/2)
DIST_PACK4_B32	In pack mode, the data type is b32. The lower 8 bits (one quarter) of valid elements in src are contiguously stored in dst based on the mask.	min(32, VL/4)
DIST_NORM	The normal mode is used, VL data is moved. The data type can be b8, b16, or b32, which is determined based on the template T.	32

**Table 2** StoreDist template parameters (double move-out mode)
StoreDist	Description	Alignment Constraint (Byte)
DIST_INTLV_B8	In double move-out mode, the data type is b8, the mask is ignored, and the elements in src0 and src1 are stored in dst in an interleaved manner. The length of dst must be VL × 2. Example: The data type is uint8_t. src0: [0, 2, 4, 6, ... 254, 0, 2, 4, ... 252, 254] src1: [1, 3, 5, 7, ... 255, 1, 3, 5, ... 253, 255] dst: [0, 1, 2, 3, ..., 254, 255, 0, 1, 2, 3, ... 253, 254, 255]	32
DIST_INTLV_B16	In double move-out mode, the data type is b16, the mask is ignored, and the elements in src0 and src1 are stored in dst in an interleaved manner. The length of dst must be VL × 2.	32
DIST_INTLV_B32	In double move-out mode, the data type is b32, the mask is ignored, and the elements in src0 and src1 are stored in dst in an interleaved manner. The length of dst must be VL × 2.	32

**Table 3** Parameters in the POST_MODE_NORMAL scenario in the single move-out mode
Parameter	Input/Output	Description
T	Input	Template parameter. The supported data types are b8, b16, b32, and b64.
dist	Input	StoreDist template parameter of the enum class type. For details about the value, see Table 1.
U	Input	RegTensor type, for example, RegTensor<half>. It is automatically inferred by the compiler and does not need to be specified.
dstAddr	Output	Start address of the destination operand in the UB.
srcReg	Input	Source operand, which is of the RegTensor type.
mask	Input	MaskReg type, indicating which elements are valid during the movement.

**Table 4** Parameters in the POST_MODE_UPDATE scenario in the single move-out mode
Parameter	Input/Output	Description
T	Input	Template parameter. The supported data types are b8, b16, b32, and b64.
postMode	Input	This parameter controls whether to enable post update. The value is of the PostLiteral type.
dist	Input	StoreDist template parameter of the enum class type. For details about the value, see Table 1.
U	Input	RegTensor type, for example, RegTensor<half>. It is automatically inferred by the compiler and does not need to be specified.
dstAddr	Input/Output	Start address of the destination operand in the UB.
srcReg	Input	Source operand, which is of the RegTensor type.
postUpdateStride	Input	The actual start address for moving data to the UB is dstAddr. After the movement, the address is automatically updated as follows: dstAddr += postUpdateStride.
mask	Input	MaskReg type, indicating which elements are valid during the movement.

**Table 5** Parameters for using AddrReg to store the offset in the single move-out mode
Parameter	Input/Output	Description
T	Input	Template parameter. The supported data types are b8, b16, b32, and b64.
dist	Input	StoreDist template parameter of the enum class type. For details about the value, see Table 1.
U	Input	RegTensor type, for example, RegTensor<half>. It is automatically inferred by the compiler and does not need to be specified.
dstAddr	Input/Output	Destination operand.
srcReg	Input	Source operand, which is of the RegTensor type.
offset	Input	The actual movement address UB is computed by adding the source address (dstAddr) and the offset.
mask	Input	MaskReg type, indicating which elements are valid during the movement.

**Table 6** Parameters in the POST_MODE_NORMAL scenario in the double move-out mode
Parameter	Input/Output	Description
T	Input	Template parameter. The supported data types are b8, b16, b32, and b64.
dist	Input	StoreDist template parameter of the enum class type. For details about the value, see Table 2.
U	Input	RegTensor type, for example, RegTensor<half>. It is automatically inferred by the compiler and does not need to be specified.
dstAddr	Output	Start address of the destination operand in the UB.
srcReg0	Input	The first source operand, which is of the RegTensor type.
srcReg1	Input	The second source operand, which is of the RegTensor type.
mask	Input	MaskReg type, indicating which elements are valid during the movement.

**Table 7** Parameters for using AddrReg to store the offset in the double move-out mode
Parameter	Input/Output	Description
T	Input	Template parameter. The supported data types are b8, b16, and b32.
dist	Input	StoreDist template parameter of the enum class type. For details about the value, see Table 2.
U	Input	RegTensor type, for example, RegTensor<half>. It is automatically inferred by the compiler and does not need to be specified.
dstAddr	Output	Start address of the destination operand in the UB.
srcReg0	Input	The first source operand, which is of the RegTensor type.
srcReg1	Input	The second source operand, which is of the RegTensor type.
offset	Input	The actual movement address UB is computed by adding the source address (dstAddr) and the offset.
mask	Input	MaskReg type, indicating which elements are valid during the movement.

Returns

None

Restrictions

The b64 data type supports only the DIST_NORM mode in StoreDist.

Examples

// Use POST_MODE_NORMAL in the single move-in or single move-out mode.
__simd_vf__ inline void ComputeMode01(__ubuf__ T* dstAddr, __ubuf__ T* srcAddr, uint32_t dstSize,
    uint32_t oneRepeatSize, uint16_t repeatTimes)
{
    AscendC::Reg::RegTensor<T> dstReg;
    AscendC::Reg::MaskReg mask;
    for (uint16_t i = 0; i < repeatTimes; ++i) {
        mask = AscendC::Reg::UpdateMask<T>(dstSize);
        AscendC::Reg::LoadAlign(dstReg, srcAddr + i * oneRepeatSize);
        AscendC::Reg::StoreAlign(dstAddr + i * oneRepeatSize, dstReg, mask);
    }
}

// Use POST_MODE_UPDATE in the single move-in or single move-out mode.
__simd_vf__ inline void ComputeMode02(__ubuf__ T* dstAddr, __ubuf__ T* srcAddr, uint32_t dstSize,
    uint32_t oneRepeatSize, uint16_t repeatTimes)
{
    AscendC::Reg::RegTensor<T> dstReg;
    AscendC::Reg::MaskReg mask;
    for (uint16_t i = 0; i < repeatTimes; ++i) {
        mask = AscendC::Reg::UpdateMask<T>(dstSize);
        AscendC::Reg::LoadAlign<T, AscendC::Reg::PostLiteral::POST_MODE_UPDATE>(dstReg, srcAddr, oneRepeatSize);
        AscendC::Reg::StoreAlign<T, AscendC::Reg::PostLiteral::POST_MODE_UPDATE>(dstAddr, dstReg, oneRepeatSize, mask);
    }
}

// Use AddrReg to store the offset in the single move-in or single move-out mode.
__simd_vf__ inline void ComputeMode03(__ubuf__ T* dstAddr, __ubuf__ T* srcAddr, uint32_t oneRepeatSize, uint16_t repeatTimes)
{
    AscendC::Reg::RegTensor<T> dstReg;
    AscendC::Reg::MaskReg mask = AscendC::Reg::CreateMask<T>();
    AscendC::Reg::AddrReg aReg;
    for (uint16_t i = 0; i < repeatTimes; ++i) {
        aReg = AscendC::Reg::CreateAddrReg<T>(i, oneRepeatSize);
        AscendC::Reg::LoadAlign(dstReg, srcAddr, aReg);
        AscendC::Reg::StoreAlign(dstAddr, dstReg, aReg, mask);
    }
}

// Use POST_MODE_NORMAL in the double move-in or double move-out mode.
__simd_vf__ inline void ComputeMode04(__ubuf__ T* dstAddr, __ubuf__ T* srcAddr, uint32_t oneRepeatSize,
    uint16_t repeatTimes)
{
    AscendC::Reg::RegTensor<T> srcReg0;
    AscendC::Reg::RegTensor<T> srcReg1;
    AscendC::Reg::MaskReg mask = AscendC::Reg::CreateMask<uint8_t, AscendC::Reg::MaskPattern::ALL>();
    for (uint16_t i = 0; i < repeatTimes; ++i) {
        AscendC::Reg::LoadAlign<T, AscendC::Reg::LoadDist::DIST_DINTLV_B8>(srcReg0, srcReg0, srcAddr + i * oneRepeatSize);
        AscendC::Reg::StoreAlign<T, AscendC::Reg::StoreDist::DIST_INTLV_B8>(dstAddr + i * oneRepeatSize, srcReg0, srcReg1, mask);
    }
}

// Use AddrReg to store the offset in the double move-in or double move-out mode.
__simd_vf__ inline void ComputeMode05(__ubuf__ T* dstAddr, __ubuf__ T* srcAddr, uint32_t oneRepeatSize, uint16_t repeatTimes)
{
    AscendC::Reg::RegTensor<T> srcReg0;
    AscendC::Reg::RegTensor<T> srcReg1;
    AscendC::Reg::MaskReg mask = AscendC::Reg::CreateMask<T>();
    AscendC::Reg::AddrReg aReg;
    for (uint16_t i = 0; i < repeatTimes; ++i) {
        aReg = AscendC::Reg::CreateAddrReg<T>(i, oneRepeatSize);
        AscendC::Reg::LoadAlign<T, AscendC::Reg::LoadDist::DIST_DINTLV_B8>(srcReg0, srcReg1, srcAddr, aReg);
        AscendC::Reg::StoreAlign<T, AscendC::Reg::StoreDist::DIST_INTLV_B8>(dstAddr, srcReg0, srcReg1, aReg, mask);
    }
}

Parent topic: Reg Data Movement