Basic Data Transfer

Applicability

Product	Supports Prototypes With Identical Data Types for Source and Destination Operands	Supports Prototypes With Different Data Types for Source and Destination Operands
Atlas 350 Accelerator Card	√	√
Atlas A3 training product/Atlas A3 inference product	√	√
Atlas A2 training product/Atlas A2 inference product	√	√
Atlas 200I/500 A2 inference product	√	x
Atlas inference product AI Core	√	x
Atlas inference product Vector Core	√	x
Atlas training product	√	x

Function Usage

Provides basic data transfer capabilities. The original format and content of data remain unchanged during transfer. Both continuous and discontinuous data transfer are supported.

Prototype

Global Memory -> Local Memory

// Continuous data transfer
template <typename T>
__aicore__ inline void DataCopy(const LocalTensor<T>& dst, const GlobalTensor<T>& src, const uint32_t count)

// Both discontinuous transfer and continuous transfer are supported.
template <typename T>
__aicore__ inline void DataCopy(const LocalTensor<T>& dst, const GlobalTensor<T>& src, const DataCopyParams& repeatParams)

Local Memory -> Local Memory

// Continuous data transfer
template <typename T>
__aicore__ inline void DataCopy(const LocalTensor<T>& dst, const LocalTensor<T>& src, const uint32_t count)

// Both discontinuous transfer and continuous transfer are supported.
template <typename T>
__aicore__ inline void DataCopy(const LocalTensor<T>& dst, const LocalTensor<T>& src, const DataCopyParams& repeatParams)

Local Memory -> Global Memory

// Continuous data transfer
template <typename T>
__aicore__ inline void DataCopy(const GlobalTensor<T>& dst, const LocalTensor<T>& src, const uint32_t count)

// Both discontinuous transfer and continuous transfer are supported.
template <typename T>
__aicore__ inline void DataCopy(const GlobalTensor<T>& dst, const LocalTensor<T>& src, const DataCopyParams& repeatParams)

Local Memory -> Local Memory: supporting different data types for source and destination operands

// Both discontinuous transfer and continuous transfer are supported.
template <typename T, typename U>
__aicore__ inline void DataCopy(const LocalTensor<T>& dst, const LocalTensor<U>& src, const DataCopyParams& repeatParams)

For details about supported transfer paths and data types of each prototype, see Supported Paths and Data Types.

Parameters

**Table 1** Template parameters
Parameter	Description
T, U	Operand data type. For details about supported data types, see Supported Paths and Data Types.

**Table 2** Parameters
Parameter	Input/Output	Description
dst	Output	Destination operand, which is of the LocalTensor or GlobalTensor type. For the LocalTensor in C2, its start address must be 64-byte aligned. For the LocalTensor in C2PIPE2GM, its start address must be 128-byte aligned. All other cases require 32-byte alignment. The start address of the GlobalTensor must be aligned based on the number of bytes occupied by the corresponding data type.
src	Input	Source operand, which is of the LocalTensor or GlobalTensor type. The start address of LocalTensor must be 32-byte aligned. The start address of the GlobalTensor must be aligned based on the number of bytes occupied by the corresponding data type.
repeatParams	Input	Data transfer parameters, which are of the DataCopyParams type. repeatParams can be used to configure the size, number, and interval of the data blocks to be transferred. Both discontinuous and continuous transfer are supported. For details, see ${INSTALL_DIR}/include/ascendc/basic_api/interface/kernel_struct_data_copy.h. Replace ${INSTALL_DIR} with the CANN installation path.
count	Input	Number of elements involved in transfer. NOTE: *count sizeof(T)** must be 32-byte aligned. If not, the amount of data to be transferred is rounded down to 32 bytes.

**Table 3** Parameters in the **DataCopyParams** structure
Parameter	Description
blockCount	Number of data chunks to be transferred, which is of the uint16_t type. The value range is [1, 4095].
blockLen	Length of each data chunk to be transferred, in the unit of a data block (32 bytes). The value is of the uint16_t type. The value range is [1, 65535]. Notes: When dst is located in C2PIPE2GM, the unit is 128 bytes. Generally, dst in C2 indicates the length of a data chunk in the source operand. The unit is 64 bytes. For the Atlas 350 Accelerator Card, dst in C2 indicates the length of a data chunk in the source operand. The unit is 32 bytes.
srcGap	Gap between adjacent data chunks of the source operand (the distance from the tail of one data chunk to the head of the next). The unit is a data block (32 bytes). The value is of the uint16_t type and must not exceed the value range of this data type. In the scenario of L1 Buffer -> Fixpipe Buffer, srcGap refers to the gap between adjacent data chunks of the source operand (the distance from the head of one data chunk to the head of the next). The unit is a data block (32 bytes). The value is of the uint16_t type and must not exceed the value range of this data type.
dstGap	Gap between adjacent data chunks of the destination operand (the distance from the tail of one data chunk to the head of the next). The unit is a data block (32 bytes). The value is of the uint16_t type and must not exceed the value range of this data type. Notes: When dst is located in C2PIPE2GM, the unit is 128 bytes. Generally, dst in C2 indicates the length of a data chunk in the source operand. The unit is 64 bytes. For the Atlas 350 Accelerator Card, dst in C2 indicates the length of a data chunk in the source operand. The unit is 32 bytes. In the scenario of L1 Buffer -> Fixpipe Buffer, dstGap refers to the gap between adjacent data chunks of the source operand (the distance from the head of one data chunk to the head of the next). The unit is a data block (32 bytes). The value is of the uint16_t type and must not exceed the value range of this data type.

The following example shows how to use the DataCopyParams structure. In the example, two data chunks are transferred. Each data chunk contains eight data blocks. There is no gap between adjacent data chunks of the source operand. For the destination operand, the gap between the tail and head of adjacent data chunks of the destination operand is one data block.

Returns

None

Restrictions

If multiple DataCopy instructions need to be executed and the destination addresses overlap, call PipeBarrier(ISASI) to insert synchronization instructions to ensure serialization of multiple instructions and prevent abnormal data. As shown on the left side of the figure, when two DataCopy instructions are executed, their destination global memory addresses overlap. You need to call PipeBarrier<PIPE_MTE3>() between the two instructions to synchronize the MTE3 output pipeline. As shown on the right side of the figure, the destination Unified Buffer addresses of the transfers overlap. Call PipeBarrier<PIPE_MTE2>() between the two instructions to synchronize the MTE2 input pipeline.
For the following product models:
Atlas A2 training product/Atlas A2 inference product

Atlas A3 training product/Atlas A3 inference product

In the development of inter-device communication operators, DataCopy APIs support inter-device data transfer. They only work with HCCS physical links. During development, pay attention to the physical links used for inter-device communication. You can execute npu-smi info -t topo to query the HCCS physical links.

For the Atlas 350 Accelerator Card, you can configure ENABLE_CV_COMM_VIA_SSBUF to select two data transfer paths during data transfer from UB to L1 Buffer. When it is set to true, the SSBuffer is used for communication and data is transferred through the hardware channel between UB and L1 Buffer (recommended). When it is set to false, data is transferred to L1 Buffer through the global memory. In this scenario, the Matmul high-level APIs need to be called for registration.

Supported Paths and Data Types

The following data paths are expressed using logical positions TPosition, with the corresponding physical paths noted. For details about the mapping between TPosition and the physical memory, see Table 1.

**Table 4** Specific paths and supported data types of Global Memory -> Local Memory
Product Model	Data Path	Data Types of the Source and Destination Operands (Same)
Atlas training product	GM -> VECIN (GM -> UB) GM -> A1, B1 (GM -> L1 Buffer)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double
Atlas inference product AI Core	GM -> VECIN (GM -> UB) GM -> A1, B1 (GM -> L1 Buffer)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double
Atlas inference product Vector Core	GM -> VECIN (GM -> UB)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double
Atlas A2 training product/Atlas A2 inference product	GM -> VECIN (GM -> UB) GM -> A1, B1, C1 (GM -> L1 Buffer)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double
Atlas A3 training product/Atlas A3 inference product	GM -> VECIN (GM -> UB) GM -> A1, B1, C1 (GM -> L1 Buffer)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double
Atlas 200I/500 A2 inference product	GM -> VECIN (GM -> UB)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double
Atlas 350 Accelerator Card	GM -> VECIN (GM -> UB) GM -> A1, B1, C1 (GM -> L1 Buffer)	b8, b16, b32, b64

**Table 5** Specific paths and supported data types of Local Memory -> Local Memory
Product Model	Data Path	Data Types of the Source and Destination Operands (Same)
Atlas 350 Accelerator Card	VECIN -> VECCALC or VECCALC -> VECOUT (UB -> UB) VECIN, VECCALC, VECOUT -> TSCM (UB -> L1 Buffer) A1, B1, C1 -> C2PIPE2GM (L1 Buffer -> Fixpipe Buffer)	bool, int8_t, uint8_t, hifloat8_t, fp8_e5m2_t, fp8_e4m3fn_t, fp8_e8m0_t, int16_t, uint16_t, half, bfloat16_t, int32_t, uint32_t, float, complex32, int64_t, uint64_t, double, complex64
Atlas 350 Accelerator Card	C1 -> C2 (L1 Buffer -> BiasTable Buffer)	half, bfloat16_t, int32_t, float
Atlas training product	VECIN -> VECCALC or VECCALC -> VECOUT (UB -> UB)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double
Atlas inference product AI Core	VECIN -> VECCALC or VECCALC -> VECOUT (UB -> UB) VECIN, VECCALC, VECOUT -> A1, B1 (UB -> L1 Buffer)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double
Atlas A2 training product/Atlas A2 inference product	VECIN -> VECCALC or VECCALC -> VECOUT (UB -> UB) VECIN, VECCALC, VECOUT -> TSCM (UB -> L1 Buffer) A1, B1, C1 -> C2PIPE2GM (L1 Buffer -> Fixpipe Buffer)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double
Atlas A2 training product/Atlas A2 inference product	C1 -> C2 (L1 Buffer -> BiasTable Buffer)	int32_t, float
Atlas A3 training product/Atlas A3 inference product	VECIN -> VECCALC or VECCALC -> VECOUT (UB -> UB) VECIN, VECCALC, VECOUT -> TSCM (UB -> L1 Buffer) A1, B1, C1 -> C2PIPE2GM (L1 Buffer -> Fixpipe Buffer)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double
Atlas A3 training product/Atlas A3 inference product	C1 -> C2 (L1 Buffer -> BiasTable Buffer)	int32_t, float

**Table 6** Specific paths and supported data types of Local Memory -> Global Memory
Product Model	Data Path	Data Types of the Source and Destination Operands (Same)
Atlas training product	VECOUT -> GM (UB -> GM)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double
Atlas inference product AI Core	VECOUT, CO2 -> GM (UB -> GM)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double
Atlas inference product Vector Core	VECOUT -> GM (UB -> GM)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double
Atlas A2 training product/Atlas A2 inference product	VECOUT -> GM (UB -> GM) A1, B1 -> GM (L1 Buffer -> GM)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double
Atlas A3 training product/Atlas A3 inference product	VECOUT -> GM (UB -> GM) A1, B1 -> GM (L1 Buffer -> GM)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double
Atlas 200I/500 A2 inference product	VECOUT -> GM (UB -> GM)	int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double
Atlas 350 Accelerator Card	VECOUT -> GM (UB -> GM)	b8, b16, b32, b64

**Table 7** Specific paths and supported data types of Local Memory -> Local Memory (supporting different data types for source and destination operands)
Product Model	Data Path	Data Type of the Source Operand	Data Type of the Destination Operand
Atlas 350 Accelerator Card	C1 -> C2 (L1 Buffer -> BiasTable Buffer)	bfloat16_t, half	float
Atlas A2 training product/Atlas A2 inference product	C1 -> C2 (L1 Buffer -> BiasTable Buffer)	half	float
Atlas A3 training product/Atlas A3 inference product	C1 -> C2 (L1 Buffer -> BiasTable Buffer)	half	float

Examples

Global Memory -> Local Memory

// srcLocal is a LocalTensor of the half type, and srcGlobal is a GlobalTensor of the half type.
// Use the transfer API with the count parameter to perform contiguous data transfer.
AscendC::DataCopy(srcLocal, srcGlobal, 512);
// Use the transfer API with the DataCopyParams parameter to perform contiguous and non-contiguous data transfer.
DataCopyParams intriParams;
intriParams.blockCount = 1; // The number of data chunks is 1.
intriParams.blockLen = 512 * sizeof(half) / 32; // Specify the length of a data chunk, in the unit of a data block. The length here is 512 half elements.
intriParams.srcGap = 0; // The source operand is transferred continuously without gaps.
intriParams.dstGap = 0; // The destination operand is laid out continuously without gaps.
AscendC::DataCopy(srcLocal, srcGlobal, intriParams);

Local Memory -> Local Memory

// srcLocal and dstLocal are LocalTensors of the half type.
// Use the transfer API with the count parameter to perform contiguous data transfer.
AscendC::DataCopy(dstLocal, srcLocal, 512);
// Use the transfer API with the DataCopyParams parameter to perform contiguous and non-contiguous data transfer.
DataCopyParams intriParams;
intriParams.blockCount = 1; // The number of data chunks is 1.
intriParams.blockLen = 512 * sizeof(half) / 32; // Specify the length of a data chunk, in the unit of a data block. The length here is 512 half elements.
intriParams.srcGap = 0; // The source operand is transferred continuously without gaps.
intriParams.dstGap = 0; // The destination operand is laid out continuously without gaps.
AscendC::DataCopy(dstLocal, srcLocal, intriParams);

Local Memory -> Global Memory

// dstLocal is a LocalTensor of the half type, and dstGlobal is a GlobalTensor of the half type.
// Use the transfer API with the count parameter to perform contiguous data transfer.
AscendC::DataCopy(dstGlobal, dstLocal, 512);
// Use the transfer API with the DataCopyParams parameter to perform contiguous and non-contiguous data transfer.
DataCopyParams intriParams;
intriParams.blockCount = 1; // The number of data chunks is 1.
intriParams.blockLen = 512 * sizeof(half) / 32; // Specify the length of a data chunk, in the unit of a data block. The length here is 512 half elements.
intriParams.srcGap = 0; // The source operand is transferred continuously without gaps.
intriParams.dstGap = 0; // The destination operand is laid out continuously without gaps.
AscendC::DataCopy(dstGlobal, dstLocal, intriParams);

Result example:

Input (srcGlobal): [1 2 3 ... 512]
Output (dstGlobal): [1 2 3 ... 512]

Parent topic: Mask Operations