[object Object][object Object][object Object]undefined[object Object]

• Description:

  The function is equivalent to the implementation of Megatron's matmul and fused_vocab_parallel_cross_entropy. It supports splitting and fusion of matmul and celoss in the vocabulary_size dimension. Based on the communication, the intermediate operations require and , which need to be called in sequence for complete function implementation.

• Formula:

  1. The matrix multiplication of $input$ and $wight^T$ is as follows:$$vocabParallelLogitsOutOptional = input @ weight^T$$
  2. Calculate the maximum value of each row in $vocabParallelLogitsOutOptional$.$$logitsMaxLocalOut = max(vocabParallelLogitsOutOptional, dim=-1)$$
  3. Calculate the difference between $vocabParallelLogitsOutOptional$ and $logitsMaxLocalOut$.$$subRes[b][n] = vocabParallelLogitsOutOptional[b][n] - logitsMaxLocalOut[b]$$
  4. Calculate the sum of each row after the exponential operation on $subRes$.$$sumExpLogitsLocalOut = sum(exp(subRes), dim=-1)$$
  5. Calculate the mask where $target$ is less than $vocabStartIndex$ or $target$ is greater than $vocabEndIndex$.$$targetMask = (target < vocabStartIndex) | (target > vocabEndIndex)$$
  6. Calculate $maskedTargetOut$.$$maskedTargetOut[b] = \begin{cases} 0 & \text{targetMask[b]=true}\\ target[b] - vocabStartIndex & \text{targetMask[b]=false} \end{cases}$$
  7. Calculate $predictedLogitsLocalOut$.$$predictedLogitsLocalOut[b] = \begin{cases} 0 & \text{targetMask[b]=true}\\ subRes[b][maskedTargetOut[b]] & \text{targetMask[b]=false} \end{cases}$$
  8. Calculate $targetMaskOut$.$$alignNum = (input.size(0) + 7) / 8 * 8\\ maskBit[p] = \begin{cases} uint8(targetMask[p]) & \text{p < input.size(0)}\\ 1 & \text{input.size(0) <= p < alignNum} \end{cases} \\ targetMaskOut[k] = 0b(maskBit[8*k:8*k+8])$$

  In the preceding information, $0 \le b \lt input.size(0), 0 \le n \lt weight.size(0), 0 \le p \lt alignNum, and 0 \le k \lt alignNum / 8$.

[object Object]

Each operator has calls. First, aclnnFusedLinearOnlineMaxSumGetWorkspaceSize is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, aclnnFusedLinearOnlineMaxSum is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see . The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

[object Object]

• Deterministic description:
  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]: The aclnnFusedLinearOnlineMaxSum is implemented in a deterministic manner by default.

[object Object]

The following example is for reference only. For details, see .