Multi-Core Non-Aligned Tiling

Overview

In the multi-core scenario, when a matrix is tiled and if M, N, or K cannot be exactly tiled by singleCoreM, singleCoreN, or singleCoreK, respectively, the tail block, or multi-core non-aligned scenario, appears, as shown in the matrix blocks in the last row and last column of matrix A and matrix B in the following figure.

In this case, the matrix block R in matrix C is still obtained by accumulating A1 × B1 + A2 × B2 + A3 × B3 + A4 × B4. When tail blocks such as A1 × B1, A2 × B2, A3 × B3, and A4 × B4 are processed, the size of the tail block needs to be set on the kernel. In case that the original tiling cannot be changed, call the SetTail API to reset the singleCoreM, singleCoreN, and singleCoreK for the computation. The tail block is moved and computed based on the set tailM, tailN, and tailK.

Application Scenarios

Tail blocks exist during Matmul matrix computation with multiple cores.

Restrictions

The SetTail API called to process tail blocks should be called before Iterate and IterateAll.

Example

For details about the complete example in the Matmul multi-core non-aligned scenario, see Matmul multi-core non-aligned split operator sample. The key sample code for this scenario is as follows:

1
2
3
4
5
6
7
8
9

    // Process the tail block.
int tailM = tiling.M - mCoreIndex * tiling.singleCoreM;
tailM = tailM < tiling.singleCoreM ? tailM : tiling.singleCoreM;
int tailN = tiling.N - nCoreIndex * tiling.singleCoreN;
tailN = tailN < tiling.singleCoreN ? tailN : tiling.singleCoreN;
// If tailM is less than singleCoreM or tailN is less than singleCoreN, the tail block needs to be processed, and you can call SetTail to set the tail block.
if (tailM < tiling.singleCoreM || tailN < tiling.singleCoreN) {
    matmulObj.SetTail(tailM, tailN);
}