Real-time DN2NZ Conversion During Data Transfer (ISASI)

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	x
Atlas A2 training product/Atlas A2 inference product	x
Atlas 200I/500 A2 inference product	x
Atlas inference product AI Core	x
Atlas inference product Vector Core	x
Atlas training product	x

Function Usage

Converts the data format during data transfer.

Prototype

template <typename T, bool enableSmallC0 = false>
__aicore__ inline void DataCopy(const LocalTensor<T>& dst, const GlobalTensor<T>& src, const Dn2NzParams& intriParams);

Parameters

**Table 1** Template parameters
Parameter	Description
T	Data type of the source or destination operand.
enableSmallC0	SmallC0 mode switch: When dValue is less than or equal to 4, the system will pad C0_SIZE to 4 × sizeof(T) bytes. This function is disabled by default.

**Table 2** Parameters
Parameter	Input/Output	Description
dst	Output	Destination operand, which is of the LocalTensor type.
src	Input	Source operand, which is of the GlobalTensor type.
intriParams	Input	Transfer parameter of the Dn2NzParams type. For details about the parameters, see Table 3. For details about the definition, see ${INSTALL_DIR}/include/ascendc/basic_api/interface/kernel_struct_data_copy.h. Replace ${INSTALL_DIR} with the file storage path after the CANN software is installed.

**Table 3** Parameters in the **Dn2NzParams** structure
Parameter	Description
dnNum	Number of DN matrices to be transferred. Value range: ndNum ∈ [0, 4095].
nValue	Number of rows in the DN matrix. Value range: nValue ∈ [0, 16384].
dValue	Number of columns in the DN matrix. Value range: dValue ∈ [0, 2³² – 1].
srcDnMatrixStride	Stride between the start addresses of adjacent DN matrices of the source operand. Value range: srcDNMatrixStride ∈ [0, 2⁶⁴ – 1] (unit: element).
srcDValue	Stride between the start addresses of adjacent rows in the same DN matrix of the source operand. Value range: srcDValue ∈ [1, 2⁶⁴ – 1] (unit: element).
dstNzC0Stride	After DN is converted to NZ, one column in the source operand is converted to multiple rows in the destination operand. dstNzC0Stride indicates the stride between the start addresses of adjacent rows in the destination NZ matrix converted from the column in the source operand. Value range: dstNzC0Stride ∈ [1, 65535]. The unit is C0_SIZE (32 bytes).
dstNzNStride	Stride between the start addresses of adjacent rows in the Z-matrix of the destination NZ matrix. Value range: dstNzNStride ∈ [1, 65535]. The unit is C0_SIZE (32 bytes).
dstNzMatrixStride	Stride between the start addresses of adjacent NZ matrices in the destination NZ matrix. Value range: dstNzMatrixStride ∈ [1, 2³² – 1] (unit: element).

The figure below shows the DN2NZ conversion. The parameter settings in the figure are described as follows (using the half data type as an example):

dnNum = 2 indicates that the number of DN matrices to be transferred is 2.
nValue = 8 indicates the number of columns in the DN matrix, that is, the width of the matrix is 8.
dValue = 24 indicates the number of rows in the DN matrix, that is, the height of the matrix is 24 elements.
srcDnMatrixStride = 96 indicates the stride between the start addresses of adjacent DN matrices, that is, the interval between A1 and C1 is six data blocks (6 × 16 = 96 elements).
srcDValue = 48 indicates the number of elements in a row, which is three data blocks (3 × 16 = 48 elements).
dstNzC0Stride = 6 indicates that after conversion from DN to NZ format, a single column in the source operand is split into multiple columns in the destination operand. For example, one column in src is divided into two separate columns A1 and A2 in dst. The stride between the start addresses of multiple columns of data is the stride of A1 and A2 in dst, which is six data blocks.
dstNzNStride = 2 indicates the stride between the start addresses of consecutive destination DN matrices, that is, the interval between A1 and B1 (two data blocks).
dstNzMatrixStride = 64 indicates the stride between the start addresses of consecutive destination ND matrices, that is, the distance between A1 and C1 is four data blocks (4 × 16 = 64 elements).

Data Paths

**Table 4** Data paths and data types
Supported Model	Data Path (Expressed Using TPosition)	Data Types of the Source and Destination Operands (Same)
Atlas 350 Accelerator Card	GM -> A1/B1	int8_t, uint8_t, fp4x2_e2m1_t, fp4x2_e1m2_t, int16_t, uint16_t, int32_t, uint32_t, half, bfloat16_t, and float

Returns

None

Restrictions

None

Examples

// dstLocal stores the output tensor after the DataCopy operation. Only A1 or B1 matrices are supported as destinations.
// srcGlobal stores the input tensor for the DataCopy operation. Only the global memory is supported as the source.

AscendC::Dn2NzParams dn2nzParams(
    /* dnNum             */ 1,
    /* nValue            */ 32,
    /* dValue            */ 32,
    /* srcDnMatrixStride */ 0,
    /* srcDValue         */ 32,
    /* dstNzC0Stride     */ 32,
    /* dstNzNStride      */ 1,
    /* dstNzMatrixStride */ 0
);
// Convert the DN-formatted data in the global memory into NZ format according to the rules defined by dn2nzParams, and copy it into A1 or B1.
AscendC::DataCopy(dstLocal, srcGlobal, dn2nzParams);

Parent topic: Mask Operations