SetNextTaskStart

This API is for trial use and may be adjusted or updated in later versions. Compatibility is not guaranteed. Pay attention to updates in later versions.

Applicability

Product	Supported	Remarks
Atlas 350 Accelerator Card	√	This API takes effect.
Atlas A3 training product / Atlas A3 inference product	√	This API takes effect.
Atlas A2 training product / Atlas A2 inference product	√	Only compilation compatibility is guaranteed. The actual function does not take effect.
Atlas 200I/500 A2 inference product	√	Only compilation compatibility is guaranteed. The actual function does not take effect.
Atlas inference product AI Core	√	Only compilation compatibility is guaranteed. The actual function does not take effect.
Atlas inference product Vector Core	√	Only compilation compatibility is guaranteed. The actual function does not take effect.
Atlas training product	√	Only compilation compatibility is guaranteed. The actual function does not take effect.

Function Usage

Is called in the subkernels of SuperKernel. After the call, the instructions can be executed in parallel with subsequent subkernels, improving the overall performance. As shown in Figure 1, SuperKernel calls subkernels in sequence. To ensure that data of subkernels does not interfere with each other, inter-operator synchronization is inserted between subkernels to preserve the order. After the subkernel_N-1 calls this API, subsequent instructions are executed in parallel with the subsequent subkernel_N.

SuperKernel is a binary fusion technology for operators. Different from source code fusion, SuperKernel focuses on the binary scheduling solution of kernel functions and performs in-depth optimization. Based on the compiled binary code, a super kernel function (SuperKernel) is created to call multiple other kernel functions (subkernels) by calling sub-functions. Compared with single-operator delivery, the SuperKernel technology can reduce the task scheduling waiting time and scheduling overhead, and further optimize the operator header overhead by utilizing the task gap resources.

You need to ensure that the instructions after this API is called do not interfere with the subsequent operators, as the interference may cause accuracy issues. You are advised to call this API after the last transfer instruction of the entire operator.

Figure 1 Parallelism implemented by SetNextTaskStart

Prototype

This prototype is supported by the following product models:

Atlas 350 Accelerator Card

Atlas A3 training product / Atlas A3 inference product

Atlas A2 training product / Atlas A2 inference product

Atlas 200I/500 A2 inference product

        
             template<pipe_t AIV_PIPE = PIPE_MTE3, pipe_t AIC_PIPE = PIPE_FIX>
__aicore__ inline void SetNextTaskStart()

This prototype is supported by the following product models:

Atlas inference product AI Core

Atlas training product

       
            template<pipe_t AIV_PIPE = PIPE_MTE3, pipe_t AIC_PIPE = PIPE_MTE3>
__aicore__ inline void SetNextTaskStart()

Parameters

**Table 1** Template parameters
Parameter	Description
AIV_PIPE	Instruction executed after SetNextTaskStart is called. If the instruction is in the AIV_PIPE pipeline on the AIV, it can be executed in parallel with subsequent operators. The value of AIV_PIPE can be PIPE_MTE2, PIPE_MTE3, PIPE_S, or PIPE_V. For details about the pipeline types, see Pipelines.
AIC_PIPE	Instruction executed after SetNextTaskStart is called. If the instruction is in the AIC_PIPE pipeline on the AIC, it can be executed in parallel with subsequent operators. The value of AIC_PIPE can be PIPE_MTE1, PIPE_MTE2, PIPE_MTE3, PIPE_FIX, or PIPE_M. For details about the pipeline types, see Pipelines.

Returns

None

Constraints

This API is applicable to the TorchAir graph development scenario and takes effect only after the SuperKernel feature is enabled. For details, see "GE Graph Mode > GE Graph Mode Functions > Calibrating the SuperKernel Range in a Graph" in PyTorch Graph Mode User Guide (TorchAir).
During operator execution, you need to ensure that this API is called on each core and is called only once on each core.
If this API is called in a TilingKey branch of a subkernel, you need to ensure that this API is called for all TilingKeys that may be run by the current operator. Otherwise, the execution may be suspended due to the mismatch of the number of synchronization instructions.

Example

      
           #include "kernel_operator.h"

AscendC::LocalTensor<half> dstLocal = outQueueDst.DeQue<half>();
AscendC::DataCopy(dstGlobal, dstLocal, 512);
// Inserted after the last transfer instruction of the operator. Ensure that the API is called only once.
AscendC::SetNextTaskStart();
outQueueDst.FreeTensor(dstLocal);

Parent topic: Inter-Task Synchronization