Multi-dimensional Data Transfer (ISASI)

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	x
Atlas A2 training product/Atlas A2 inference product	x
Atlas 200I/500 A2 inference product	x
Atlas inference product AI Core	x
Atlas inference product Vector Core	x
Atlas training product	x

Function Usage

Supports more flexible configuration of the dimensions and strides for data transfer.

Prototype

Multi-dimensional data transfer from the global memory to the local memory

template <typename T, uint8_t dim, const NdDmaConfig& config = kDefaultNdDmaConfig>
__aicore__ inline void DataCopy(const LocalTensor<T>& dst, const GlobalTensor<T>& src, const NdDmaParams<T, dim>& params)

Before using DataCopy for data transfer, the NdDma DataCache must be refreshed by calling NdDmaDci to ensure the DataCache is up to date.
1
__aicore__ inline void NdDmaDci()

Parameters

Table 1 Template parameters

Parameter

Description

Data type of the source or destination operand.

dim

Dimension for data transfer. The data type is uint8_t. Value range: [1, 5].

config

Configuration items for data transfer, which are of the NdDmaConfig type. For details, see Table 3.

struct NdDmaConfig {
    static constexpr uint16_t unsetPad = 0xffff;
    bool isNearestValueMode = false;
    uint16_t loopLpSize = unsetPad; // Left padding size of all dimensions, must be less than 256.
    uint16_t loopRpSize = unsetPad; // Right padding size of all dimensions, must be less than 256.
    bool ascOptimize = false;       // used for Ascend C optimization on special senario.
};

Table 2 Parameters

Parameter

Input/Output

Description

dst

Output

Destination operand, which is of the LocalTensor type.

src

Input

Source operand, which is of the GlobalTensor type.

params

Input

Parameters for data transfer, which are of the NdDmaParams type. For details, see Table 4.

template <typename T, uint8_t dim>
struct NdDmaParams  {
    NdDmaLoopInfo<dim> loopInfo;
    T constantValue;  // If left or right padding is applied and NearestValueMode is not enabled, this value will be used as the padding fill.
};

NdDmaLoopInfo type. For details about the parameters, see Table 5. The definition is as follows.

template <uint8_t dim>
struct NdDmaLoopInfo  {
    uint64_t loopSrcStride[dim] = {0}; // src stride info per loop.
    uint32_t loopDstStride[dim] = {0}; // dst stride info per loop.
    uint32_t loopSize[dim] = {0}; // Loop size per loop.
    uint8_t loopLpSize[dim] = {0}; // Left padding size per loop.
    uint8_t loopRpSize[dim] = {0}; // Right padding size per loop.
};
// Note: The valid range of dim is [1, 5].

**Table 3** Parameters in the **NdDmaConfig** structure
Parameter	Description
unsetPad	Indicates that PaddingSize is not set. The value is fixed at 0xFFFF.
isNearestValueMode	Method used for filling padding values. It is of the boolean type. True: Padding uses the nearest value. The leftmost or rightmost value of the current dimension is used for left or right padding, respectively. For details, see Figure 1. False: Padding uses a constant value. All padding values are filled with a fixed constant value defined by NdDmaParams::constantValue. When the data type is b64, the value of isNearestValueMode must be False.
loopLpSize	Left padding size applied to each dimension in a loop. If it is set to a specific value instead of unsetPad, that value becomes the left padding size for all loops and NdDmaLoopInfo::loopLpSize does not take effect. The default value is unsetPad. You can set this parameter to the default value or a value in the range [0, 255].
loopRpSize	Right padding size applied to each dimension in a loop. If it is set to a specific value instead of unsetPad, that value becomes the right padding size for all loops and NdDmaLoopInfo::loopRpSize does not take effect. The default value is unsetPad. You can set this parameter to the default value or a value in the range [0, 255].
ascOptimize	Reserved parameter, not supported currently.

**Table 4** Parameters in the **NdDmaParams** structure
Parameter	Description
loopInfo	Data transfer information for each dimension. The type is NdDmaLoopInfo<dim>. In the NdDmaLoopInfo structure, parameters of array type have their array index values corresponding to the actual dimension information. Indexes 0–4 correspond to dimensions 1–5, respectively. For details, see Table 5.
constantValue	A numeric value of type T. When left or right padding is applied to a dimension and NearestValueMode is disabled, this value will be used to fill the padding. When the data type is b64, the value of constantValue must be 0.

**Table 5** Parameters in the **NdDmaLoopInfo** structure
Parameter	Description
loopSrcStride	Stride between the source operand element and the next element in each dimension. Unit: the number of elements. Data type: uint64_t. Value range of srcStride: [0, 2⁴⁰).
loopDstStride	Stride between the destination operand element and the next element in each dimension. Unit: the number of elements. Data type: uint32_t. Value range of dstStride: [0, 2²⁰).
loopSize	Number of elements processed in each dimension (excluding padding elements). Unit: the number of elements. Data type: uint32_t. Value range of dstStride: [0, 2²⁰).
loopLpSize	Number of elements to be padded on the left in each dimension. Unit: the number of elements. Data type: uint8_t. The value of srcStride cannot exceed the value range of this data type.
loopRpSize	Number of elements to be padded on the right in each dimension. Unit: the number of elements. Data type: uint8_t. The value of srcStride cannot exceed the value range of this data type.

The following uses two dimensions as an example to describe several typical application scenarios.

Figure 1 2D padding scenario

Figure 2 2D transpose scenario

Figure 3 2D broadcast scenario

Figure 4 2D slice scenario

Data Paths

**Table 6** Data paths and data types
Supported Model	Data Path (Expressed Using TPosition)	Data Types of the Source and Destination Operands (Same)
Atlas 350 Accelerator Card	GM -> VECIN	b8, b16, b32, b64

Returns

None

Restrictions

A single instruction can only access data within an address range width of up to 40 bits (1 TB).
The source operand size per loop is (loopLpSize + loopSize + loopRpSize – 1) × loopSrcSize, and the destination operand size per loop is (loopLpSize + loopSize + loopRpSize – 1) × loopDstSize. The sum of all loop sizes must not exceed 2⁴⁰.
When each loop's dstStride is in ascending order, the address spaces of different loops must not overlap or interleave. Take a 2D padding scenario as an example. For the second dimension, the minimum stride values for loopSrcStride and loopDstStride must be 3. This ensures that data element 3 cannot fall into the loop of dimension 1.
This API performs data transfer via NDDMA. The corresponding NDDMA cache size is 32 KB. Before using DataCopy for data transfer, you need to call NdDmaDci to refresh the cache. Without refreshing, in a multi-core scenario, reading and writing to the same global memory address may cause some cores to read incorrect data.

Example

// T: type of the data to be transferred
// xGm: stores the data copied in by DataCopy.
// xLocal: stores the data copied out by DataCopy.

// 2D padding scenario
// xGmShape: [2, 8], transferring 8 columns and 2 rows of data. Left padding: 3. Top padding: 1. Right padding: 5. Bottom padding: 1. xLocalShape: [4, 16].
AscendC::NdDmaLoopInfo<2> loopInfo{{1, 8}, {1, 16}, {8, 2}, {3, 1}, {5, 1}};
AscendC::NdDmaParams<T, 2> params{loopInfo, 0};  // The value of padding is 0.
AscendC::NdDmaDci();  // Refresh the cache.
static constexpr AscendC::NdDmaConfig dmaConfig;  // Use the default parameters or do not pass any parameters.
AscendC::DataCopy<T, 2, dmaConfig>(xLocal, xGm, params);

Result example:

Input data (xGm):
[[ 1.  2.  3.  4.  5.  6.  7.  8.]
 [ 9. 10. 11. 12. 13. 14. 15. 16.]]
Data transferred to the local memory (xLocal):
[[ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  1.  2.  3.  4.  5.  6.  7.  8.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  9. 10. 11. 12. 13. 14. 15. 16.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]]

Parent topic: Mask Operations