Gemm (Deprecated)

Applicability

Product	Supported
Atlas 350 Accelerator Card	x
Atlas A3 training product/Atlas A3 inference product	x
Atlas A2 training product/Atlas A2 inference product	x
Atlas 200I/500 A2 inference product	x
Atlas inference product AI Core	√
Atlas inference product Vector Core	x
Atlas training product	√

Function Usage

This API has been deprecated and will be removed in later versions. Do not use this API.

Multiplies two tensors and outputs a result tensor. Multiply matrix A and matrix B to obtain matrix C, and output matrix C.

Prototype

Functional API:

template <typename T, typename U, typename S>
__aicore__ inline void Gemm(const LocalTensor<T>& dst, const LocalTensor<U>& src0, const LocalTensor<S>& src1, const uint32_t m, const uint32_t k, const uint32_t n, GemmTiling tilling, bool partialsum = true, int32_t initValue = 0)

Tiling compute API:

template <typename T>
__aicore__ inline GemmTiling GetGemmTiling(uint32_t m, uint32_t k, uint32_t n)

Parameters

Table 1 Parameters

Parameter

Input/Output

Description

dst

Output

Destination operand.

For the Atlas training product, the supported TPosition is CO1 or CO2.

For the Atlas inference product AI Core, the supported TPosition is CO1 or CO2.

src0

Input

Source operand. TPosition is set to A1.

src1

Input

Source operand. TPosition is set to B1.

Input

Valid height of the left matrix Src0Local. The value range is [1, 4096].

Note: m does not need to be rounded up to a multiple of 16.

Input

Valid width of the left matrix Src0Local and valid height of the right matrix Src1Local.

If tensor Src0Local is of type float, the value range is [1, 8192].
If tensor Src0Local is of type half, the value range is [1, 16384].
If tensor Src0Local is of type int8_t, the value range is [1, 32768].

Note: k does not need to be rounded up to a multiple of 16.

Input

Valid width of the right matrix Src1Local. The value range is [1, 4096].

Note: n does not need to be rounded up to a multiple of 16.

tilling

Input

Tilling rule. The type is GemmTiling. The structure is defined as follows:

struct GemmTiling {
    const uint32_t blockSize = 16;
    LoopMode loopMode = LoopMode::MODE_NM;
    uint32_t mNum = 0;
    uint32_t nNum = 0;
    uint32_t kNum = 0;
    uint32_t roundM = 0;
    uint32_t roundN = 0;
    uint32_t roundK = 0;
    uint32_t c0Size = 32;
    uint32_t dtypeSize = 1;
    uint32_t mBlockNum = 0;
    uint32_t nBlockNum = 0;
    uint32_t kBlockNum = 0;
    uint32_t mIterNum = 0;
    uint32_t nIterNum = 0;
    uint32_t kIterNum = 0;
    uint32_t mTileBlock = 0;
    uint32_t nTileBlock = 0;
    uint32_t kTileBlock = 0;
    uint32_t kTailBlock = 0;
    uint32_t mTailBlock = 0;
    uint32_t nTailBlock = 0;
    bool kHasTail = false;
    bool mHasTail = false;
    bool nHasTail = false;
    bool kHasTailEle = false;
    uint32_t kTailEle = 0;
};

For details about the parameter description, see Table 3.

partialsum

Input

When the dst parameter's TPosition is set to CO2, this parameter controls whether the computation results are moved out.

0: move out computation result.
1: The computation result is not moved out but used for subsequent computation.

initValue

Input

Indicates whether dst needs to be initialized.

0: dst needs to be initialized. The dst initial matrix stores the previous conv2d result and will be added up with the new conv2d result.
1: dst does not need to be initialized. The data in the dst initial matrix is meaningless, and the new computation results will directly overwrite the data in dst.

**Table 2** Data type combination of feature_map, weight, and dst
src0.dtype	src1.dtype	dst.dtype
int8_t	int8_t	int32_t
half	half	float
half	half	half

Table 3 Parameters in the GemmTiling structure

Parameter

Input/Output

Description

blockSize

uint32_t

Number of elements stored in a dimension. The value is fixed at 16.

loopMode

LoopMode

Traversal mode. The structure is defined as follows:

enum class LoopMode {
    MODE_NM = 0,
    MODE_MN = 1,
    MODE_KM = 2,
    MODE_KN = 3
};

mNum

uint32_t

Equivalent data length of the M axis. The value range is [1, 4096].

nNum

uint32_t

Equivalent data length of the N axis. The value range is [1, 4096].

kNum

uint32_t

Equivalent data length of the K axis.

If tensor Src0Local is of type float, the value range is [1, 8192].
If tensor Src0Local is of type half, the value range is [1, 16384].
If tensor Src0Local is of type int8_t, the value range is [1, 32768].

roundM

uint32_t

Equivalent data length of the M axis. The value is rounded up to an integer multiple of blockSize. The value range is [1, 4096].

roundN

uint32_t

Equivalent data length of the N axis. The value is rounded up to an integer multiple of blockSize. The value range is [1, 4096].

roundK

uint32_t

Equivalent data length of the K axis. The value is rounded up to a multiple of c0Size.

If tensor Src0Local is of type float, the value range is [1, 8192].
If tensor Src0Local is of type half, the value range is [1, 16384].
If tensor Src0Local is of type int8_t, the value range is [1, 32768].

c0Size

uint32_t

Length of a block. The value can be 16 or 32.

dtypeSize

uint32_t

Length of the input data, in bytes. The value range is [1, 2].

mBlockNum

uint32_t

Number of blocks on the M axis. mBlockNum = mNum/blockSize

nBlockNum

uint32_t

Number of blocks on the N axis. nBlockNum = nNum/blockSize

kBlockNum

uint32_t

Number of blocks on the K axis. kBlockNum = kNum/blockSize

mIterNum

uint32_t

Number of traversed dimensions. The value range is [1, 4096].

nIterNum

uint32_t

Number of traversed dimensions. The value range is [1, 4096].

kIterNum

uint32_t

Number of traversed dimensions. The value range is [1, 4096].

mTileBlock

uint32_t

Number of split blocks on the M axis. The value range is [1, 4096].

nTileBlock

uint32_t

Number of split blocks on the N axis. The value range is [1, 4096].

kTileBlock

uint32_t

Number of split blocks on the K axis. The value range is [1, 4096].

kTailBlock

uint32_t

Number of tail blocks on the K axis. The value range is [1, 4096].

mTailBlock

uint32_t

Number of tail blocks on the M axis. The value range is [1, 4096].

nTailBlock

uint32_t

Number of tail blocks on the N axis. The value range is [1, 4096].

kHasTail

bool

Indicates whether a tail block exists on the K axis.

mHasTail

bool

Indicates whether a tail block exists on the M axis.

nHasTail

bool

Indicates whether a tail block exists on the N axis.

kHasTailEle

bool

Indicates whether the tail block element exists.

kTailEle

uint32_t

Tail block element on the K axis. The value range is [1, 4096].

Restrictions

m, k, and n do not need to be multiples of 16. However, due to hardware restrictions, the shapes of the operands dst, Src0Local, and Src1Local must meet the alignment requirements: the m and n dimensions must be rounded up to multiples of 16, while the k dimension must be rounded up to either 16 or 32 depending on the data type of the operands.
For details about the operand address alignment requirements, see General Address Alignment Restrictions.

Parent topic: Cube Computation