[object Object][object Object][object Object]undefined[object Object]

• Description: Scans data in Attention when Attention and FFN are deployed separately. This operator receives the output data from the FFNToAttention operator and scans the data step by step to ensure that the data is ready.

  It is recommended that this operator be used together with the FFNToAttention and AttentionWorkerCombine operators to form a complete workflow.

  1. Receives data sent by the FFNToAttention operator. The data is stored in the ScheduleContext structure. For details, see . This structure contains CommonArea, ControlArea, AttentionArea, and FfnArea. This API involves CommonArea (used to store configuration information, such as [object Object], [object Object], [object Object], and [object Object]), ControlArea (used to determine whether the upper-layer control process exits), and AttentionArea (used to manage the core data buffer and status information required during operator computation. The [object Object] parameter stores the data related to the input).

  2. Reads the flag information stored in [object Object] in ScheduleContext.AttentionArea to check whether the communication data is ready.

  3. Allows data to be used by the AttentionWorkerCombine operator after all data is ready.

• Formula:

$$\text{Initialize:} \quad \text{ready\_count} = 0, \quad \text{flag\_num} = \text{micro\_batch\_size} \times \text{selected\_expert\_num}$$ $$\text{Check if run\_flag is 0:} \quad \text{if run\_flag} = 0, \quad \text{exit and log}$$ $$\text{Loop:} \quad \text{while run\_flag} \neq 0: \quad \text{ready\_count} = \sum_{i=1}^{\text{flag\_num}} \mathbf{1}\_{\{ \text{flag}[i] = 1 \}}; \quad \text{if ready\_count} = \text{flag\_num}, \quad \text{break}$$ $$\text{Reset flags:} \quad \text{flag}[i] = 0 \quad \text{for} \quad i = 1, 2, \dots, \text{flag\_num}$$ $$\text{Set micro\_batch\_id:} \quad \text{micro\_batch\_id} = (\text{micro\_batch\_id} + 1) \% \text{micro\_batch\_num}$$

Note: [object Object], [object Object], [object Object], and [object Object] are parameters of the ScheduleContext structure, which is described in .

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• By default, [object Object] defaults to deterministic implementation . The non-deterministic implementation is not supported, and the deterministic computation configuration does not take effect.

[object Object]

The following example is for reference only. For details, see .