Load3D

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product / Atlas A3 inference product	√
Atlas A2 training product / Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	√
Atlas inference product AI Core	√
Atlas inference product Vector Core	x
Atlas training product	√

Function Usage

Performs the image-to-column operation to convert a multi-dimensional feature map into a two-dimensional matrix. The following data paths are supported: A1 -> A2 and B1 -> B2.

Prototype

Load3Dv1 API

        
             template <typename T, const IsResetLoad3dConfig &defaultConfig = IS_RESER_LOAD3D_DEFAULT_CONFIG, typename U = PrimT<T>, typename Std::enable_if<Std::is_same<PrimT<T>, U>::value, bool>::type = true>
__aicore__ inline void LoadData(const LocalTensor<T>& dst, const LocalTensor<T>& src, const LoadData3DParamsV1<U>& loadDataParams)

Load3Dv2 API

        
             template <typename T, const IsResetLoad3dConfig &defaultConfig = IS_RESER_LOAD3D_DEFAULT_CONFIG, typename U = PrimT<T>, typename Std::enable_if<Std::is_same<PrimT<T>, U>::value, bool>::type = true>
__aicore__ inline void LoadData(const LocalTensor<T>& dst, const LocalTensor<T>& src, const LoadData3DParamsV2<U>& loadDataParams)

Load3Dv2Pro API

       
            template <typename T>
__aicore__ inline void LoadData(const LocalTensor<T>& dst, const LocalTensor<T>& src, const LoadData3DParamsV2Pro& loadDataParams)

Parameters

Table 1 Template parameters

Parameter

Description

Data types of the source and destination operands.

Load3Dv1 API:
For the Atlas training product , the supported data types are uint8_t, int8_t, and half.

For the Atlas inference product AI Core, the supported data types are uint8_t, int8_t, and half.
Load3Dv2 API:
For the Atlas inference product AI Core, the supported data types are uint8_t, int8_t, half, and int4b_t.

For the Atlas A2 training product / Atlas A2 inference product :
- When TPosition is A1 or A2, the supported data types are uint8_t, int8_t, half, bfloat16_t, uint32_t, int32_t, float, and int4b_t.
- When TPosition is B1 or B2, the supported data types are half, bfloat16_t, uint32_t, int32_t, and float.
For the Atlas A3 training product / Atlas A3 inference product :
- When TPosition is A1 or A2, the supported data types are uint8_t, int8_t, half, bfloat16_t, uint32_t, int32_t, float, and int4b_t.
- When TPosition is B1 or B2, the supported data types are half, bfloat16_t, uint32_t, int32_t, and float.
For the Atlas 200I/500 A2 inference product :
- When TPosition is A1 or A2, the supported data types are uint8_t, int8_t, half, bfloat16, uint32_t, int32_t, float, and int4b_t.
- When TPosition is B1 or B2, the supported data types are half, bfloat16_t, uint32_t, int32_t, and float.
For the Atlas 350 Accelerator Card, the supported data types are uint8_t, int8_t, fp8_e4m3fn_t, fp8_e5m2_t, hifloat8_t, half, bfloat16_t, uint32_t, int32_t, and float.
Load3Dv2Pro API:
For the Atlas 350 Accelerator Card, the supported data types are uint8_t, int8_t, fp8_e4m3fn_t, fp8_e5m2_t, hifloat8_t, half, bfloat16_t, uint32_t, int32_t, and float.

defaultConfig

Whether to set related attributes in Load3Dv1 or Load3Dv2. This parameter is of the IsResetLoad3dConfig type. The IsResetLoad3dConfig structure is defined as follows:

           
                struct IsResetLoad3dConfig {
   bool isSetFMatrix = true;
   bool isSetPadding = true;
};

If isSetFMatrix is set to true, the attribute description (including l1H, l1W, and padList. For details, see Table 3 and Table 4) of the FeatureMap is set in the API. If this parameter is set to false, the attribute description of the FeatureMap passed in the API does not take effect and you need to set the attribute description by referring to SetFmatrix.

If isSetPadding is set to true, the pad attribute description (that is, the padValue parameter. For details, see Table 3 and Table 4) is set in the API. If this parameter is set to false, the pad attribute passed in the API does not take effect and you need to set the attribute description by referring to SetLoadDataPaddingValue. See Examples for reference example.

The default value of this parameter is as follows:

           
                constexpr IsResetLoad3dConfig IS_RESER_LOAD3D_DEFAULT_CONFIG = {true, true};

Data type of padValue in LoadData3DParamsV1 or LoadData3DParamsV2.

When basic data types are used for dst and src, U must be consistent with the data type T of dst and src. Otherwise, the compilation fails.
When the TensorTrait type is used for dst and src, the LiteType of U must match that of the data type T of dst and src. Otherwise, the compilation fails.

The last template parameter is used only for checking the preceding data types.

**Table 2** Common parameters
Parameter	Input/Output	Description
dst	Output	Destination operand, which is of the LocalTensor type. The sequential arrangement of data is determined by TPosition of the destination operand. The constraints are as follows: A2: ZZ or NZ format B2: ZN format A1 or B1: There is no format restriction. Generally, the data is in NZ format.
src	Input	Source operand, which is of the LocalTensor or GlobalTensor type. Its data type must be the same as that of dst.
loadDataParams	Input	LoadData parameter structure. Supported types are as follows: LoadData3DParamsV1. For details, see Table 3. LoadData3DParamsV2. For details, see Table 4. LoadData3DParamsV2Pro. For details, see Table 5. For details, see ${INSTALL_DIR}/include/ascendc/basic_api/interface/kernel_struct_mm.h. Replace ${INSTALL_DIR} with the actual path for storing files after the CANN software is installed.

**Table 3** Parameters in the **LoadData3DParamsV1** structure
Parameter	Description
padList	Padding list [padding_left, padding_right, padding_top, padding_bottom]. The value range of each element is [0,255]. Default value: {0, 0, 0, 0}.
l1H	Height of the source operand. Value range: l1H ∈ [1, 32767].
l1W	Width of the source operand. Value range: l1W ∈ [1, 32767].
c1Index	Start position of the source tensor along the C1 dimension. Value range: c1Index ∈ [0, 4095]. Default value: 0.
fetchFilterW	Start position of the filter along the W dimension. Value range: fetchFilterW ∈ [0, 254]. Default value: 0.
fetchFilterH	Start position of the filter along the H dimension. Value range: fetchFilterH ∈ [0, 254]. Default value: 0.
leftTopW	Start position of the source operand along the W dimension. Value range: leftTopW ∈ [–255, 32767]. Default value: 0. If padding_left is a, set leftTopW to -a.
leftTopH	Start position of the source operand along the H dimension. Value range: leftTopH ∈ [–255, 32767]. Default value: 0. If padding_top is a, set leftTopH to -a.
strideW	Stride of the convolution kernel along the W dimension of the source operand. Value range: strideW ∈ [1, 63].
strideH	Stride of the convolution kernel along the H dimension of the source operand. Value range: strideH ∈ [1, 63].
filterW	Width of the convolution kernel. Value range: filterW ∈ [1, 255].
filterH	Height of the convolution kernel. Value range: filterH ∈ [1, 255].
dilationFilterW	Width dilation of the convolution kernel. Value range: dilationFilterW ∈ [1, 255].
dilationFilterH	Height dilation of the convolution kernel. Value range: dilationFilterH ∈ [1, 255].
jumpStride	Stride of the destination operand's start address between iterations. Value range: jumpStride ∈ [1, 127].
repeatMode	Repeat mode. Mode 0: In each repeat, points in the filter window are added, corresponding to the increase in the W dimension of the destination matrix. Mode 1: In each repeat, the upper left coordinates of the sliding window are added, corresponding to the increase in the H dimension of the destination matrix. Value range: repeatMode ∈ [0, 1]. Default value: 0.
repeatTime	Number of iteration repeats. The addresses of the source and destination operands change upon every iteration. Value range: repeatTime ∈ [1, 255].
cSize	Whether to enable the optimization of cSize = 4(b16) or cSize = 8(b8). Value range: cSize ∈ [0, 1]. Default value: 0.
padValue	Padding value. Its data type must be the same as that of src. Default value: 0. To disable padding, set all elements of padList to 0.

**Table 4** Parameters in the **LoadData3DParamsV2** structure
Parameter	Description
padList	Padding list [padding_left, padding_right, padding_top, padding_bottom]. The value range of each element is [0,255]. Default value: {0, 0, 0, 0}.
l1H	Height of the source operand. Value range: l1H ∈ [1, 32767].
l1W	Width of the source operand. Value range: l1W ∈ [1, 32767].
channelSize	Number of channels of the source operand. Value range: channelSize ∈ [1, 63]. For the following product models, the values of channelSize are as follows: For half, the value can be 4, 8, 16, N × 16 + 4, or N × 16 + 8. For int8_t and uint8_t, the value can be 4, 8, 16, 32, N × 32 + 4, N × 32 + 8, or N × 32 + 16. For int4b_t, the value can be 8, 16, 32, N × 64, N × 64 + 8, N × 64 + 16, or N × 64 + 32. Atlas inference product AI Core For the following product models, the valid values of channelSize are as follows: For uint32_t, int32_t, and float, the value can be 4, N × 8, or N × 8 + 4. For half and bfloat16, the value can be 4, 8, N × 16, N × 16 + 4, or N × 16 + 8. For int8_t and uint8_t, the value can be 4, 8, 16, 32 × N, N × 32 + 4, N × 32 + 8, or N × 32 + 16. For int4b_t, the value can be 8, 16, 32, N × 64, N × 64 + 8, N × 64 + 16, or N × 64 + 32. Atlas A2 training product / Atlas A2 inference product Atlas A3 training product / Atlas A3 inference product Atlas 200I/500 A2 inference product Atlas 350 Accelerator Card
kExtension	Transfer length along the width dimension of the destination operand. If the rightmost fractal is not covered, the value must be a multiple of 16 for half and a multiple of 32 for int8_t and uint8_t. No multiple requirement applies when the rightmost fractal is covered. Value range: kExtension ∈ [1, 65535].
mExtension	Transfer length along the height dimension of the destination operand. If the bottom fractal is not covered, the value must be a multiple of 16 for half, int8_t, and uint8_t. No multiple requirement applies when the bottom fractal is covered. Value range: mExtension ∈ [1, 65535].
kStartPt	Start position along the width dimension of the destination operand. The value must be a multiple of 16 for half, and a multiple of 32 for int8_t and uint8_t. Value range: [0, 65535]. Default value: 0.
mStartPt	Start position along the height dimension of the destination operand. If the bottom fractal is not covered, the value must be a multiple of 16 for half, int8_t, and uint8_t. No multiple requirement applies when the bottom fractal is covered. Value range: [0, 65535]. Default value: 0.
strideW	Stride of the convolution kernel along the width dimension of the source operand. Value range: strideW ∈ [1, 63].
strideH	Stride of the convolution kernel along the height dimension of the source operand. Value range: strideH ∈ [1, 63].
filterW	Width of the convolution kernel. Value range: filterW ∈ [1, 255].
filterH	Height of the convolution kernel. Value range: filterH ∈ [1, 255].
dilationFilterW	Width dilation of the convolution kernel. Value range: dilationFilterW ∈ [1, 255].
dilationFilterH	Height dilation of the convolution kernel. Value range: dilationFilterH ∈ [1, 255].
enTranspose	A bool specifying whether to transpose the entire target matrix. This parameter is valid only when the destination TPosition is set to A2 and the source operand is of the half type. Default value: false. true: enabled. false: disabled.
enSmallK	Whether to enable the small k feature. The size of each fractal matrix is 16 × 4. This parameter is of the bool type and defaults to false. This feature is no longer supported in the current product form. true: enabled false: disabled
padValue	Padding value. Its data type must be the same as that of src. Default value: 0. To disable padding, set all elements of padList to 0.
filterSizeW	Whether to add 256 elements to the width of the convolution kernel based on filterW. true: yes; false: no.
filterSizeH	Whether to add 256 elements to the height of the convolution kernel based on filterH. true: yes; false: no.
fMatrixCtrl	Whether the LoadData3DV2 instruction retrieves FeatureMap attribute descriptions from the left or right matrix. It works with SetFmatrix and can only be set to false (default). true: Retrieve FeatureMap attribute descriptions from the right matrix. false: Retrieve FeatureMap attribute descriptions from the left matrix.

**Table 5** Parameters in the **LoadData3DParamsV2Pro** structure
Parameter	Description
channelSize	Number of channels of the source operand. Value range: channelSize ∈ [1, 63]. For half, the remainder of channelSize divided by 16 must be 0, 4, or 8. For int8_t and uint8_t, the remainder of channelSize divided by 32 must be 0, 4, 8, or 16. For int4b_t, ChannelSize can be 8, 16, 32, N × 64, N × 64 + 8, N × 64 + 16, or N × 64 + 32, where N is a positive integer.
enTranspose	A bool specifying whether to transpose the entire target matrix. This parameter is valid only when the destination TPosition is set to A2 and the source operand is of the half type. Default value: false. true: enabled. false: disabled.
enSmallK	Whether to enable the small k feature. The size of each fractal matrix is 16 × 4. This parameter is of the bool type and defaults to false. This feature is no longer supported in the current product form. true: enabled false: disabled
filterSizeW	Whether to add 256 elements to the width of the convolution kernel based on filterW. true: yes; false: no.
filterSizeH	Whether to add 256 elements to the height of the convolution kernel based on filterH. true: yes; false: no.
fMatrixCtrl	Whether the LoadData3DV2 instruction retrieves FeatureMap attribute descriptions from the left or right matrix. It works with SetFmatrix and can only be set to false (default). true: Retrieve FeatureMap attribute descriptions from the right matrix. false: Retrieve FeatureMap attribute descriptions from the left matrix.
extConfig	Combined parameter of type uint64_t. The default value is 0. extConfig= ((uint64_t)mStartPt << 48) \| ((uint64_t)kStartPt << 32) \| ((uint64_t)mExtension << 16) \| (uint64_t)kExtension
filterConfig	Combined parameter of type uint64_t. The default value is 0X10101010101. filterConfig= ((uint64_t)dilationFilterH << 40) \| ((uint64_t)dilationFilterW << 32) \| ((uint64_t)filterH << 24) \| ((uint64_t)filterW << 16) \| ((uint64_t)strideH << 8) \| (uint64_t)strideW

Restrictions

For details about the operand address alignment requirements, see General Address Alignment Restrictions.

To enable the LoadData3DParamsV1 cSize feature, ensure that the feature map in A1 and B1 is 4-channel aligned.
To call the Load3Dv2 or Load3Dv2Pro instruction, configure dstStride by referring to SetLoadDataRepeat. If the repeat mode is disabled, the repeat-related parameters in the API adopt default values.

Load3d Data Formats

The input feature map and filter must follow the NC1HWC0 format. C0 is the innermost dimension with a fixed value of 16, or 32 for u8/s8 types. C1 = C/C0.

To simplify the scenario, assume the input feature map has 4 channels, namely Ci = 4. The input feature map in A1 has a shape of (Hi, Wi, Ci). After processing by load3dv1, the data in A2 is shaped as (Wo × Ho, Hk × Wk × Ci). Wo and Ho denote the output dimensions after convolution, while Hk and Wk represent the filter dimensions.

Intuitively, the img2col operation slides the filter across the feature map and unfolds the corresponding feature map data into each row of the output. The filter slides Wo steps along the W dimension, then shifts one step along the H dimension, and repeats this process, generating a total of Wo × Ho rows of output data. In the figure below, the red and yellow data represent the first and second rows respectively. The numbers illustrate the mapping between the original input data, filter, and output data. As demonstrated, load3dv1 first fetches four values corresponding to 00 along the Ci dimension of the input data, followed by four values corresponding to 01. The total length of each row is Hk × Wk × Ci, which equals 3 × 3 × 4 = 36 values.

The following figure shows the format of the feature map.

Figure 1 Feature map format

The following figure shows the filter format:

In the format, n indicates the number of filters, and the dimension layout is (Hk, Wk, Ci, n). Note that this format needs to be converted to match the format of matrix B in MMAD.

Figure 2 Filter format

In practice, due to constraints on storage and compute resources, the entire convolution computation is processed in blocks, with only one block of data loaded and computed per iteration.

Figure 3 Convolution computation processed in blocks

Two blocking schemes are available for the feature map in A2: horizontal blocking and vertical blocking, corresponding to repeatMode 0 and 1, respectively.

Note: The fractal matrix shown in the figure is 4 × 4. The actual size is 16 × 16 (or 16 × 32 for u8 and s8 data types).

When repeatMode is set to 0, each repeat changes the data read position within the filter window, followed by a jump to the next C0 position.

Figure 4 Filter window when repeatMode is set to 0

When repeatMode is set to 1, the data read position within the filter window stays fixed, and each repeat moves forward by C0 elements in the feature map.

Figure 5 Filter window when repeatMode is set to 1

Returns

None

Examples

This example is supported on the Atlas inference product AI Core.

      
           uint16_t C1 = 2;
uint16_t H = 4, W = 4;
uint8_t Kh = 2, Kw = 2;
uint16_t Cout = 16;
uint16_t C0 = 16;
uint8_t dilationH = 2, dilationW = 2;
uint8_t padTop = 1, padBottom = 1, padLeft = 1, padRight = 1;
uint8_t strideH = 1, strideW = 1;
uint8_t padList[4] = {padLeft, padRight, padTop, padBottom};
AscendC::LoadData(featureMapA2, featureMapA1,
    { padList, H, W, 0, 0, 0, -1, -1, strideW, strideH, Kw, Kh, dilationW, dilationH, 1, 0, fmRepeat, 0, (half)(0)});

Parent topic: LoadData