Overview

The Ascend C kernel function is a processing function running on a core. The samples described in Basic Vector Operators and Using TBuf are operators running on a single core, and do not involve tiling implementation on the host. The following figure shows vector operator implementation.

To boost operator execution efficiency, multi-core parallel computing is usually implemented in operators. In other words, input data is tiled, and data blocks are allocated across cores for processing. For details about parallel computing, see the SPMD model. The local memory on a single core is limited, and sometimes cannot completely store the operator input and output data at once. Therefore, it is necessary to move some inputs for computation multiple times until the complete final result is obtained. This process of data tiling and block computation is called tiling. The algorithm for splitting data is called the tiling algorithm or tiling strategy. Then, a computation program, called tiling implementation or tiling function, determines tiling parameters (such as the block size transferred each time and the total number of cycles) based on operator information such as shape. The AI Core is not good at scalar computation in the tiling implementation. Therefore, this computation is executed on the host CPU independently. In kernel functions, the tiling structure transferred from the host is parsed to obtain the tiling information, which is used to control the process of transferring data in and out of the local memory. The operator logic is implemented by calling the computing, data movement, memory management, and task synchronization APIs.

Due to hardware restrictions, input data tiling must comply with the following principles:

1. Due to physical restrictions of AI Core on Unified Buffer, the data storage space on Unified Buffer must be 32-byte aligned.
   • If the input data is not 32-byte aligned, its total length is rounded up to the nearest multiple of 32 bytes.
   • The tiling computation is implemented in the minimum unit of 32 bytes.
2. Make sure Unified Buffer is fully utilized.

   Performance overhead may be generated when the AI Core interacts with the global memory, and frequent data movement may cause performance bottlenecks. Therefore, Unified Buffer should be fully utilized to reduce the number of data movements from the Global Memory.

3. The Ascend AI Processor contains multiple AI Cores. You should make full use of the multi-core computing capability and evenly allocate the computing to the AI Cores.

This section describes several typical scenarios based on the preceding principles. For details about the complete code, see the multi-core Add operator sample.

As shown in the preceding figure, the length of the operator input allocated to multiple cores for computation is TOTAL_LENGTH, and the length of the data computed by each core is BLOCK_LENGTH. The computation data of each core is further tiled based on the size of the local memory. The number of tiled data blocks is TILE_NUM, and the length of each data block is TILE_LENGTH.

Based on whether each core computes the same amount of data and whether each data block in the core has the same amount of data, the tiling strategy may vary in the following scenarios:

1. Same amount of data in a core or across cores: Each core processes the same amount of data, and each data block in the core has the same amount of data. In this scenario, multi-core tiling is used to evenly allocate data across cores for execution, and the length of data computed each time by each core is the same.
2. Same amount of data across cores only: Each core processes the same amount of data, but the data blocks in a core may have different amounts of data. In this scenario, multi-core tiling is used. Data in a core cannot be divided into multiple 32-byte-aligned data blocks with the same amount of data. Tail block tiling is required to compute the data in the tail block.
3. Same amount of data in a core only: Cores compute different amounts of data, and each data block in a core has the same amount of data. In this scenario, tail core tiling is used to solve the problem that data cannot be evenly allocated across cores.
4. Different amounts of data in a core and across cores: Cores process different amounts of data, and the data blocks in a core may have different amounts of data. In this scenario, tail cores and tail blocks need to be considered to properly tile data across cores and within a core.