Real-time NZ2ND Conversion During Data Transfer

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product AI Core	√
Atlas inference product Vector Core	x
Atlas training product	x

Function Usage

Supports NZ-to-ND format conversion during data transfer.

Prototype

template <typename T>
__aicore__ inline void DataCopy(const GlobalTensor<T>& dst, const LocalTensor<T>& src, const Nz2NdParamsFull& intriParams)

For details about supported transfer paths and data types of each prototype, see Supported Paths and Data Types.

Parameters

**Table 1** Template parameters
Parameter	Description
T	Data type of the source or destination operand. For details about supported data types, see Supported Paths and Data Types.

**Table 2** API parameters
Parameter	Input/Output	Description
dst	Output	Destination operand, which is of the GlobalTensor type.
src	Input	Source operand, which is of the LocalTensor type.
intriParams	Input	Data transfer parameters, which are of the Nz2NdParamsFull type. For details, see ${INSTALL_DIR}/include/ascendc/basic_api/interface/kernel_struct_data_copy.h. Replace ${INSTALL_DIR} with the CANN installation path.

**Table 3** Parameters in the Nz2NdParamsFull structure
Parameter	Description
ndNum	Number of NZ matrices to be transferred. The value range is [0, 4095].
nValue	Number of rows in the NZ matrix. The value range is [1, 8192].
dValue	Number of columns in the NZ matrix. The value range is [1, 8192], and dValue must be a multiple of 16.
srcNdMatrixStride	Head-to-head stride between adjacent source NZ matrices. srcNdMatrixStride ∈ [1, 512]. The unit is 256 (16 × 16) elements.
srcNStride	Head-to-head stride between adjacent Z arrangements of the same source NZ matrix. srcNStride ∈ [0, 4096]. The unit is 16 elements.
dstDStride	Head-to-head stride between adjacent rows of the destination ND matrix. dstDStride ∈ [1, 65535] (unit: element)
dstNdMatrixStride	Head-to-head stride between adjacent source NZ matrices in the destination ND matrix. dstNdMatrixStride ∈ [1, 65535] (unit: element)

The following figure shows the NZ2ND conversion of the half type. The parameter settings in the example are described as follows:

ndNum = 2 indicates that the number of source NZ matrices is 2 (NZ matrix 1 is A1–A4 + B1–B4, and NZ matrix 2 is C1–C4 + D1–D4)
nValue = 4 indicates the number of rows in the NZ matrix, that is, the height of the matrix is 4.
dValue = 32 indicates the number of columns in the NZ matrix, that is, the width of the matrix is 32 elements.
srcNdMatrixStride = 1 indicates the stride between the start addresses of adjacent NZ matrices, that is, the distance between A1 and C1 (16 data blocks × 16 elements = 256 elements).
srcNStride = 4 indicates the stride between adjacent Z-tiles in the same source NZ matrix, that is, the distance between A1 and B1 (4 data blocks × 16 elements per block = 64 elements).
dstDStride = 160 indicates the stride between adjacent rows in a destination ND matrix, that is, the distance between A1 and A2, which is 10 data blocks, with each block containing 16 elements (10 × 16 = 160 elements).
dstNdMatrixStride = 48 indicates the stride between the start addresses of the xth and (x+1)th destination ND matrices in dst, that is, the distance between A1 and C1, which is three data blocks (3 × 16 = 48 elements).

Figure 1 NZ2ND conversion (half type)

The following figure shows the NZ2ND conversion of the float type. The parameter settings in the example are described as follows:

ndNum = 2 indicates that the number of source NZ matrices is 2 (NZ matrix 1 is A1–A8 + B1–B8, and NZ matrix 2 is C1–C8 + D1–D8).
nValue = 4 indicates the number of rows in the NZ matrix, that is, the height of the matrix is 4.
dValue = 32 indicates the number of columns in the NZ matrix, that is, the width of the matrix is 32 elements.
srcNdMatrixStride = 1 indicates the stride between the start addresses of adjacent NZ matrices, that is, the distance from A1 to C1, which is 256 elements (32 data blocks × 8 elements per block).
srcNStride = 4 indicates the stride between adjacent Z-tiles in the same source NZ matrix, that is, the distance between A1 and B1, which is 64 elements (8 data blocks × 8 elements per block).
dstDStride = 144 indicates the stride between adjacent rows in a destination ND matrix, that is, the distance between A1 and A3, which is 18 data blocks (18 × 8 = 144 elements).
dstNdMatrixStride = 40 indicates the stride between the start addresses of the xth and (x+1)th destination ND matrices in dst, that is, the distance between A1 and C1, which is five data blocks (5 × 8 = 40 elements).

Figure 2 NZ2ND conversion (float type)

Returns

None

Restrictions

None

Supported Paths and Data Types

The following data paths are expressed using logical positions TPosition, with the corresponding physical paths noted. For details about the mapping between TPosition and the physical memory, see Table 1.

**Table 4** Specific paths and supported data types of Local Memory -> Global Memory
Product Model	Data Path	Data Types of the Source and Destination Operands (Same)
Atlas 350 Accelerator Card	VECOUT -> GM (UB -> GM)	bool, int8_t, uint8_t, hifloat8_t, fp8_e5m2_t, fp8_e4m3fn_t, fp8_e8m0_t, int16_t, uint16_t, half, bfloat16_t, int32_t, uint32_t, float, complex32, int64_t, uint64_t, double, complex64
Atlas inference product AI Core	VECOUT, CO2 -> GM (UB -> GM)	int16_t, uint16_t, int32_t, uint32_t, half, float
Atlas A2 training product/Atlas A2 inference product	VECOUT -> GM (UB -> GM)	int16_t, uint16_t, int32_t, uint32_t, half, bfloat16_t, float
Atlas A3 training product/Atlas A3 inference product	VECOUT -> GM (UB -> GM)	int16_t, uint16_t, int32_t, uint32_t, half, bfloat16_t, float

Examples

For details about intriParams, see Figure 1.

// dstLocal is a LocalTensor of the half type, and dstGlobal is a GlobalTensor of the half type.
AscendC::Nz2NdParamsFull intriParams{1, 32, 32, 1, 32, 32, 1};
// Local Memory -> Global Memory
AscendC::DataCopy(dstGlobal, dstLocal, intriParams);

Result example:

Input (srcGlobal): [1 2 3 ... 1024]
Output (dstGlobal): [1 2 ... 15 16 513 514 ... 527 528 17 18 ... 31 32 529 530 ... 543 544 ...497 498 ...  511 512  1009 1010... 1023 1024]

Parent topic: Mask Operations