Documentation
Download

Basic Data Transfer

Applicability

Product

Supports

Prototypes With Identical Data Types

for Source and Destination Operands

Supports

Prototypes With Different Data Types

for Source and Destination Operands

Atlas A3 training products/Atlas A3 inference products

√

√

Atlas A2 training products/Atlas A2 inference products

√

√

Atlas 200I/500 A2 inference products

√

x

Atlas inference product's AI Core

√

x

Atlas inference product's Vector Core

√

x

Atlas training products

√

x

Function

Provides basic data transfer capabilities. The original format and content of data remain unchanged during transfer. Both continuous and discontinuous data transfer are supported.

Prototype

  • Global Memory -> Local Memory
    1
2
3
4
5
6
7
    // Continuous data transfer
template <typename T>
__aicore__ inline void DataCopy(const LocalTensor<T>& dst, const GlobalTensor<T>& src, const uint32_t count)

// Both discontinuous transfer and continuous transfer are supported.
template <typename T>
__aicore__ inline void DataCopy(const LocalTensor<T>& dst, const GlobalTensor<T>& src, const DataCopyParams& repeatParams)
  • Local Memory -> Local Memory
    1
2
3
4
5
6
7
    // Continuous data transfer
template <typename T>
__aicore__ inline void DataCopy(const LocalTensor<T>& dst, const LocalTensor<T>& src, const uint32_t count)

// Both discontinuous transfer and continuous transfer are supported.
template <typename T>
__aicore__ inline void DataCopy(const LocalTensor<T>& dst, const LocalTensor<T>& src, const DataCopyParams& repeatParams)
  • Local Memory -> Global Memory
    1
2
3
4
5
6
7
    // Continuous data transfer
template <typename T>
__aicore__ inline void DataCopy(const GlobalTensor<T>& dst, const LocalTensor<T>& src, const uint32_t count)

// Both discontinuous transfer and continuous transfer are supported.
template <typename T>
__aicore__ inline void DataCopy(const GlobalTensor<T>& dst, const LocalTensor<T>& src, const DataCopyParams& repeatParams)
  • Local Memory -> Local Memory: supporting different data types for source and destination operands
    1
2
3
    // Both discontinuous transfer and continuous transfer are supported.
template <typename T, typename U>
__aicore__ inline void DataCopy(const LocalTensor<T>& dst, const LocalTensor<U>& src, const DataCopyParams& repeatParams)

For details about supported transfer paths and data types of each prototype, see Supported Paths and Data Types.

Parameters

Table 1 Template parameters

Parameter

Description

T, U

Data type of the operand. For details about supported data types, see Supported Paths and Data Types.
Table 2 Parameters

Parameter

Input/Output

Description

dst

Output

Destination operand, which is of the LocalTensor or GlobalTensor type.

For the LocalTensor in C2, its start address must be 64-byte aligned. For the LocalTensor in C2PIPE2GM, its start address must be 128-byte aligned. All other cases require 32-byte alignment.

The start address of the GlobalTensor must be aligned based on the number of bytes occupied by the corresponding data type.

src

Input

Source operand, which is of the LocalTensor or GlobalTensor type.

The start address of the LocalTensor must be 32-byte aligned.

The start address of the GlobalTensor must be aligned based on the number of bytes occupied by the corresponding data type.

repeatParams

Input

Transfer parameters, of the DataCopyParams type. repeatParams can be used to configure the size, number, and interval of the data blocks to be transferred. Both discontinuous and continuous transfer are supported.

For details, see ${INSTALL_DIR}/include/ascendc/basic_api/interface/kernel_struct_data_copy.h. Replace ${INSTALL_DIR} with the CANN installation path.

count

Input

Number of elements involved in transfer.

NOTE:

count * sizeof(T) must be 32-byte aligned. If not, the amount of data to be transferred is rounded down to 32 bytes.
Table 3 Parameters in the DataCopyParams structure

Parameter

Description

blockCount

Number of contiguous data blocks to be transferred, which is of the uint16_t type. The value range is [1, 4095].

blockLen

Length of each contiguous data block to be transferred, in the unit of DataBlock (32 bytes). The value is of the uint16_t type. The value range is [1, 65535].

For dst in C2PIPE2GM, the unit is 128 bytes. For dst in C2, it indicates the length of contiguous data blocks for source operand transfer, with the unit being 64 bytes.

srcGap

Interval between adjacent contiguous data blocks of the source operand (the interval between the end of the preceding block and the start of the next block). The unit is DataBlock (32 bytes). The value is of the uint16_t type and must not exceed the value range of this data type.

In the scenario of L1 Buffer -> Fixpipe Buffer, srcGap refers to the interval between adjacent contiguous data blocks of the source operand (the interval between the start of the preceding block and the start of the next block). The unit is DataBlock (32 bytes). The value is of the uint16_t type and must not exceed the value range of this data type.

dstGap

Interval between adjacent contiguous data blocks of the destination operand (the interval between the end of the preceding block and the start of the next block). The unit is DataBlock (32 bytes). The value is of the uint16_t type and must not exceed the value range of this data type.

For dstLocal in C2PIPE2GM, the unit is 128 bytes. For dstLocal in C2, the unit is 64 bytes.

In the scenario of L1 Buffer -> Fixpipe Buffer, dstGap refers to the interval between adjacent contiguous data blocks of the source operand (the interval between the start of the preceding block and the start of the next block). The unit is DataBlock (32 bytes). The value is of the uint16_t type and must not exceed the value range of this data type.

The following example shows how to use the DataCopyParams structure. In the example, two contiguous data blocks are transferred. Each data block contains eight DataBlocks. There is no interval between adjacent data blocks of the source operand. For the destination operand, the interval between the end of the preceding data block and the start of the next data block is one DataBlock.

Returns

None

Restrictions

  • If multiple DataCopy instructions need to be executed and the destination addresses overlap, call PipeBarrier(ISASI) to insert synchronization instructions to ensure serialization of multiple instructions and prevent abnormal data. As shown on the left side of the figure, when two DataCopy instructions are executed, their destination global memory addresses overlap. You need to call PipeBarrier<PIPE_MTE3>() between the two instructions to synchronize the MTE3 output pipeline. As shown on the right side of the figure, the destination Unified Buffer addresses of the transfers overlap. Call PipeBarrier<PIPE_MTE2>() between the two instructions to synchronize the MTE2 input pipeline.

  • For the following product models:

    Atlas A2 training products/Atlas A2 inference products

    Atlas A3 training products/Atlas A3 inference products

    In the development of inter-device communication operators, DataCopy APIs support inter-device data transfer. They only work with HCCS physical links. During development, pay attention to the physical links used for inter-device communication. You can execute the npu-smi info -t topo command to query HCCS physical link information.

Supported Paths and Data Types

The following data paths are expressed using logical positions TPosition, with the corresponding physical paths noted. For details about the mapping between TPosition and the physical memory, see Table 1.

Table 4 Specific paths and supported data types of Global Memory -> Local Memory

Product Model

Data Path

Data Types of the Source and Destination Operands (Same)

Atlas training products
  • GM -> VECIN (GM -> UB)
  • GM -> A1, B1 (GM -> L1 Buffer)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double

Atlas inference product's AI Core
  • GM -> VECIN (GM -> UB)
  • GM -> A1, B1 (GM -> L1 Buffer)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double

Atlas inference product's Vector Core
  • GM -> VECIN (GM -> UB)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double

Atlas A2 training products/Atlas A2 inference products
  • GM -> VECIN (GM -> UB)
  • GM -> A1, B1, C1 (GM -> L1 Buffer)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double

Atlas A3 training products/Atlas A3 inference products
  • GM -> VECIN (GM -> UB)
  • GM -> A1, B1, C1 (GM -> L1 Buffer)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double

Atlas 200I/500 A2 inference products
  • GM -> VECIN (GM -> UB)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double
Table 5 Specific paths and supported data types of Local Memory -> Local Memory

Product Model

Data Path

Data Types of the Source and Destination Operands (Same)

Atlas training products
  • VECIN -> VECCALC or VECCALC -> VECOUT (UB -> UB)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double

Atlas inference product's AI Core
  • VECIN -> VECCALC or VECCALC -> VECOUT (UB -> UB)
  • VECIN, VECCALC, VECOUT -> A1, B1 (UB -> L1 Buffer)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double

Atlas A2 training products/Atlas A2 inference products
  • VECIN -> VECCALC or VECCALC -> VECOUT (UB -> UB)
  • VECIN, VECCALC, VECOUT -> TSCM (UB -> L1 Buffer)
  • A1, B1, C1 -> C2PIPE2GM (L1 Buffer -> Fixpipe Buffer)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double
  • C1 -> C2 (L1 Buffer -> BiasTable Buffer)

int32_t, float

Atlas A3 training products/Atlas A3 inference products
  • VECIN -> VECCALC or VECCALC -> VECOUT (UB -> UB)
  • VECIN, VECCALC, VECOUT -> TSCM (UB -> L1 Buffer)
  • A1, B1, C1 -> C2PIPE2GM (L1 Buffer -> Fixpipe Buffer)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double
  • C1 -> C2 (L1 Buffer -> BiasTable Buffer)

int32_t, float
Table 6 Specific paths and supported data types of Local Memory -> Global Memory

Product Model

Data Path

Data Types of the Source and Destination Operands (Same)

Atlas training products
  • VECOUT -> GM (UB -> GM)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double

Atlas inference product's AI Core
  • VECOUT, CO2 -> GM (UB -> GM)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double

Atlas inference product's Vector Core
  • VECOUT -> GM (UB -> GM)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, float, double

Atlas A2 training products/Atlas A2 inference products
  • VECOUT -> GM (UB -> GM)
  • A1, B1 -> GM (L1 Buffer -> GM)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double

Atlas A3 training products/Atlas A3 inference products
  • VECOUT -> GM (UB -> GM)
  • A1, B1 -> GM (L1 Buffer -> GM)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double

Atlas 200I/500 A2 inference products
  • VECOUT -> GM (UB -> GM)

int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t, half, bfloat16_t, float, double
Table 7 Specific paths and supported data types of Local Memory -> Local Memory (supporting different data types for source and destination operands)

Product Model

Data Path

Data Type of the Source Operand

Data Type of the Destination Operand

Atlas A2 training products/Atlas A2 inference products

C1 -> C2 (L1 Buffer -> BiasTable Buffer)

half

float

Atlas A3 training products/Atlas A3 inference products

C1 -> C2 (L1 Buffer -> BiasTable Buffer)

half

float

Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
AscendC::TPipe pipe;
AscendC::TQue<AscendC::TPosition::VECIN, 1> inQueueSrc;
AscendC::TQue<AscendC::TPosition::VECOUT, 1> outQueueDst;
AscendC::GlobalTensor<half> srcGlobal, dstGlobal;
pipe.InitBuffer(inQueueSrc, 1, 512 * sizeof(half));
pipe.InitBuffer(outQueueDst, 1, 512 * sizeof(half));
AscendC::LocalTensor<half> srcLocal = inQueueSrc.AllocTensor<half>();
AscendC::LocalTensor<half> dstLocal = outQueueDst.AllocTensor<half>();
// Use the transfer API with the count parameter to perform contiguous data transfer.
AscendC::DataCopy(srcLocal, srcGlobal, 512);
AscendC::DataCopy(dstLocal , srcLocal, 512);
AscendC::DataCopy(dstGlobal, dstLocal, 512);
// Use the transfer API with the DataCopyParams parameter to perform contiguous and non-contiguous data transfer.
// DataCopyParams intriParams;
// AscendC::DataCopy(srcLocal, srcGlobal, intriParams);
// AscendC::DataCopy(dstLocal , srcLocal, intriParams);
// AscendC::DataCopy(dstGlobal, dstLocal, intriParams);
Result example:
Input (srcGlobal): [1 2 3 ... 512]
Output (dstGlobal): [1 2 3 ... 512]
Parent topic: DataCopy
This site uses cookies to improve your browsing experience. By continuing to browse this site, you agree to our use of cookies.View Details