Load2DBitMode

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	x
Atlas A2 training product/Atlas A2 inference product	x
Atlas 200I/500 A2 inference product	x
Atlas inference product AI Core	x
Atlas inference product Vector Core	x
Atlas training product	x

Function Usage

Supports data transfer over the following paths:

GM -> A1; GM -> B1; GM -> A2; GM -> B2;

A1 -> A2; B1 -> B2.

Prototype

template <TPosition Dst, TPosition Src, typename T>
__aicore__ inline void LoadData(const LocalTensor<T>& dst, const LocalTensor<T>& src,const Load2DBitModeParam& loadDataParam)

Parameters

**Table 1** Template parameters
Parameter	Description
T	Data types of the source and destination operands. For the Atlas 350 Accelerator Card, the data can be moved only from A1 to A2 or from B1 to B2, and the supported data types are half, bfloat16_t, uint32_t, int32_t, float, uint8_t, int8_t, fp8_e4m3fn_t, fp8_e5m2_t, hifloat8_t, fp4x2_e2m1_t, and fp4x2_e1m2_t.
Src	Logical position (TPosition) of the source operand, used only by the Load2DBitMode API.
Dst	Logical position (TPosition) of the destination operand, used only by the Load2DBitMode API.

**Table 2** Common parameters
Parameter	Input/Output	Description
dst	Output	Destination operand, which is of the LocalTensor type. The sequential arrangement of data is determined by TPosition of the destination operand. The constraints are as follows: When storing data in A2, use the ZZ or NZ format, with the fractal matrix size of 16 × (32 bytes/sizeof(T)). When storing data in B2, use the ZN format, with the fractal matrix size of (32 bytes/sizeof(T)) × 16. When storing data in A1 or B1, there is no format restriction. Generally, the data is in NZ format, with the fractal matrix size of 16 × (32 bytes/sizeof(T)).
src	Input	Source operand, which is of the LocalTensor or GlobalTensor type. Its data type must be the same as that of dst.
loadDataParams	Input	LoadData parameter structure. Supported types are as follows: Load2DBitModeParam. For details, see Table 3. For details, see ${INSTALL_DIR}/include/ascendc/basic_api/interface/kernel_struct_mm.h. Replace ${INSTALL_DIR} with the actual path for storing files after the CANN software is installed.

**Table 3** Parameters in **Load2DBitModeParam**
Parameter	Description
config0	A parameter of type uint64_t. It forms a union together with a bit-field struct called Load2DBitModeConfig0 (represented as config0BitMode). It is initialized to 0. You can use the class method GetConfig0() to retrieve its value.
config0BitMode	A bit-field struct type called Load2DBitModeConfig0. It forms a union together with config0. For details about the parameter, see Table 4.
config1	A parameter of type uint64_t. It forms a union together with a bit-field struct called Load2DBitModeConfig1 (represented as config1BitMode). It is initialized to 0. You can use the class method GetConfig1() to retrieve its value.
config1BitMode	A bit-field struct type called Load2DBitModeConfig1. It forms a union together with config1. For details about the parameter, see Table 5.
ifTranspose	Whether to enable the transpose function for each fractal matrix. The default value is false. This parameter has the same meaning as the parameter with the same name in the LoadData2DParamsV2 structure. For details, see Table 4. true: enabled. false: disabled. Note: The transpose function can be enabled only when data is transferred from A1 to A2 or from B1 to B2. When the transpose function is enabled, the restrictions on the supported data types are as follows: The source and destination operands support the b4, b8, b16, and b32 data types.

The design principles of the Load2DBitModeParam class parameters are as follows:

A union is a special data structure that allows different data types to be stored in the same memory location. All members of a union share the same memory space, and its size is determined by the largest member. At any given time, only one member can be used.

A bit-field is a special class member that allows precise control over how many bits each variable occupies in memory. In a struct, members are laid out from top to bottom, corresponding to memory from low bits to high bits.

The Load2DBitModeParam class uses both union and bit-field methods. Parameters are expressed using bit values, with the bit-field struct automatically handling the exact number of bits for each input. By leveraging the union, multiple parameters can be combined and passed together. Only one input parameter is needed to contain all required information, and the underlying API only needs to receive a single parameter. Additionally, when you need to modify the value of a specific bit within the parameter, you can do so with loops and bitwise operations, without having to re-pass the entire parameter. This reduces scalar computations and improves performance.

The Load2DBitModeParam class can be directly initialized using an object of the LoadData2DParamsV2 struct type.

LoadData2DParamsV2 loadDataParams;
loadDataParams.mStartPosition = 0;
loadDataParams.kStartPosition = 0;
loadDataParams.mStep = xxx;
loadDataParams.kStep = xxx;
loadDataParams.srcStride = xxx;
loadDataParams.dstStride = xxx;
loadDataParams.sid = 0;
loadDataParams.ifTranspose = false;
Load2DBitModeParam params(loadDataParams);  // Directly use an object of the LoadData2DParamsV2 struct type for initialization.

You can also use the Set function of each parameter to modify the parameter value. Because the union is used, you can directly modify config0 and config1 bit by bit to modify the parameters.

**Table 4** Parameters in the **Load2DBitModeConfig0** structure
Parameter	Description
mStartPosition	For an M × K matrix, this parameter specifies the start position along the M-axis of the source matrix, measured in units of 16 elements. This parameter is the lowest-order field in the bit-field struct. It occupies 16 bits. You can set and obtain its value using the SetMStartPosition() and GetMStartPosition() functions of the Load2DBitModeParam class, respectively. For details, see Table 6.
kStartPosition	For an M × K matrix, this parameter specifies the start position along the K-axis of the source matrix, measured in units of 32 bytes. This parameter is the second-order field in the bit-field struct. It occupies 16 bits. You can set and obtain its value using the SetKStartPosition() and GetKStartPosition() functions of the Load2DBitModeParam class, respectively. For details, see Table 6.
mStep	For an M × K matrix, this parameter specifies the transfer length along the M-axis of the source matrix, measured in units of 16 elements. Value range: mStep ∈ [0, 255]. When the transpose function is enabled by setting ifTranspose, mStep must meet the following additional constraints in addition to the value range of [0, 255]: When the data type is b4, mStep must be a multiple of 4. When the data type is b8, mStep must be a multiple of 2. When the data type is b16, mStep must be a multiple of 1. When the data type is b32, there is no additional constraint on mStep. This parameter is the third-order field in the bit-field struct. It occupies 8 bits. You can set and obtain its value using the SetMStep() and GetMStep() functions of the Load2DBitModeParam class, respectively. For details, see Table 6.
kStep	For an M × K matrix, this parameter specifies the transfer length along the K-axis of the source matrix, measured in units of 32 bytes. Value range: kStep ∈ [0, 255]. When the transpose function is enabled by setting ifTranspose, kStep must meet the following additional constraints in addition to the value range of [0, 255]: When the data type is b4, b8, or b16, there is no additional constraint on kStep. When the data type is b32, kStep must be a multiple of 2. This parameter is the highest-order field in the bit-field struct. It occupies 8 bits. You can set and obtain its value using the SetKStep() and GetKStep() functions of the Load2DBitModeParam class, respectively. For details, see Table 6.

The parameters in the Load2DBitModeConfig0 struct have the same meaning as the parameters with the same names in the LoadData2DParamsV2 structure. For details, see Table 4.

**Table 5** Parameters in the **Load2DBitModeConfig1** structure
Parameter	Description
srcStride	For an M × K matrix, this parameter specifies the interval between the start addresses of consecutive fractal matrices along the K-axis of the source matrix, measured in units of 512 bytes. This parameter is the lowest-order field in the bit-field struct. It occupies 16 bits. You can set and obtain its value using the SetSrcStride() and GetSrcStride() functions of the Load2DBitModeParam class, respectively. For details, see Table 6.
dstStride	For an M × K matrix, this parameter specifies the interval between the start address of one fractal matrix and the start address of the next fractal matrix along the K-axis of the destination matrix. The unit is 512 bytes. This parameter is the highest-order field in the bit-field struct. It occupies 16 bits. You can set and obtain its value using the SetDstStride() and GetDstStride() functions of the Load2DBitModeParam class, respectively. For details, see Table 6.

The parameters in the Load2DBitModeConfig1 struct have the same meaning as the parameters with the same names in the LoadData2DParamsV2 structure. For details, see Table 4.

**Table 6** Member functions of the **Load2DBitModeParam** class
Function	Description
void SetMStartPosition(uint32_t mStartPosition_)	Sets the value of mStartPosition in Load2DBitModeConfig0 to mStartPosition_.
void SetKStartPosition(uint32_t kStartPosition_)	Sets the value of kStartPosition in Load2DBitModeConfig0 to kStartPosition_.
void SetMStep(uint16_t mStep_)	Sets the value of mStep in Load2DBitModeConfig0 to mStep_.
void SetKStep(uint16_t kStep_)	Sets the value of kStep in Load2DBitModeConfig0 to kStep_.
void SetSrcStride(int32_t srcStride_)	Sets the value of srcStride in Load2DBitModeConfig1 to srcStride_.
void SetDstStride(uint16_t dstStride_)	Sets the value of dstStride in Load2DBitModeConfig1 to dstStride_.
uint32_t GetMStartPosition() const	Obtains the value of mStartPosition in Load2DBitModeConfig0.
uint32_t GetKStartPosition() const	Obtains the value of kStartPosition in Load2DBitModeConfig0.
uint16_t GetMStep() const	Obtains the value of mStep in Load2DBitModeConfig0.
uint16_t GetKStep() const	Obtains the value of kStep in Load2DBitModeConfig0.
int32_t GetSrcStride() const	Obtains the value of srcStride in Load2DBitModeConfig1.
uint16_t GetDstStride() const	Obtains the value of dstStride in Load2DBitModeConfig1.

Restrictions

For details about the operand address alignment requirements, see General Address Alignment Restrictions.

Returns

None

Examples

#include "kernel_operator.h"
uint16_t C1 = 2;
uint16_t H = 4, W = 4;
uint8_t Kh = 2, Kw = 2;
uint16_t Cout = 16;
uint16_t C0 = 16;
uint8_t dilationH = 2, dilationW = 2;
uint8_t padTop = 1, padBottom = 1, padLeft = 1, padRight = 1;
uint8_t strideH = 1, strideW = 1;
uint16_t coutBlocks, ho, wo, howo, howoRound;
uint32_t featureMapA1Size, weightA1Size, featureMapA2Size, weightB2Size, dstSize, dstCO1Size;
uint8_t padList[4] = {padLeft, padRight, padTop, padBottom};
featureMapA2Size = howoRound * (C1 * Kh * Kw * C0);

fmRepeat = featureMapA2Size / (16 * C0);

LoadData2DParamsV2 param = { padList, H, W, 0, 0, 0, -1, -1, strideW, strideH, Kw, Kh, dilationW, dilationH, 1, 0, fmRepeat, 0, (half)(0)};
Load2DBitModeParam paramBitMode(param); 

AscendC::LocalTensor<half> featureMapA1 = inQueueFmA1.DeQue<half>();
AscendC::LocalTensor<half> featureMapA2 = inQueueFmA2.AllocTensor<half>();
AscendC::LoadData<A2, A1, half>(featureMapA2, featureMapA1, paramBitMode);
inQueueFmA2.EnQue<half>(featureMapA2);
inQueueFmA1.FreeTensor(featureMapA1);

Parent topic: LoadData