Inter-core Load Balancing

[Priority] Medium

[Description] The number of physical cores of the AI Processor is fixed. After the L2 cache is tiled, some cores may have compute tailing. That is, the compute amount of all cores divided by the amount of data processed by each core cannot be exactly divided by the number of cores. As a result, some tail cores are required to compute the tail block data. However, during tail core compute, some cores are always in an idle state, leading to deteriorated operator performance. As shown in Figure 1, if the total data size is TotalSize, the L2 cache is divided into two parts (TotalSize/2). The compute amount of each core is TotalSize/2/25, that is, 25 cores are required for processing. Because the number of cores of the AI Processor is 20, during each compute operation, each of cores 1 to 5 needs to compute one more piece of data. As a result, tailing occurs.

[Negative Example]

[Positive Example]

For the foregoing tiling strategy, global load optimization can be achieved after relocation of the tail cores, as shown in Figure 2. When all computes are complete, one more data block is computed for 1 to 10 cores, achieving optimal global load.