[object Object]

[object Object][object Object]undefined[object Object]

• Description: This operator is a part of the cross entropy computation module in the vocabulary parallelism scenario. It solves the video memory and computing efficiency problems in the case of ultra-large vocabulary. This part involves the gradient computation implementation, which is used to compute the gradients of leaf nodes [object Object] and [object Object]. The outputs of [object Object] and [object Object], along with the global communication result related to [object Object], need to be obtained as the input of this API.
• Formula:

   High-performance mode, where the value of softmaxOptional is not nullptr:

$$\text{softmax} \in \mathbb{R}^{BT \times V}$$ $$\text{arange\_1d} = [0, 1, \dots, BT-1] \in \mathbb{N}^{BT}$$ $$\text{softmax\_update} = \mathbf{1} - \text{target\_mask}.view(-1) \in \mathbb{R}^{BT}$$ $$\text{softmax}[\text{arange\_1d}, \text{masked\_target}] \leftarrow \text{softmax}[\text{arange\_1d}, \text{masked\_target}] - \text{softmax\_update}$$ $$\text{softmax} \leftarrow \text{softmax} \odot \text{grad}.unsqueeze(-1) \in \mathbb{R}^{BT \times V}$$ $$\text{grad\_input} = \text{softmax} \cdot \text{weight}^T \in \mathbb{R}^{BT \times H}$$ $$\text{grad\_weight} = \text{softmax}^T \cdot \text{input} \in \mathbb{R}^{V \times H}$$

   Video memory–saving mode, where the value of softmaxOptional is nullptr:

$$\text{vocab\_parallel\_logits} = \text{input} \cdot \text{weight}^T \quad \in \mathbb{R}^{BT \times V}$$ $$\text{logits\_sub} = \text{vocab\_parallel\_logits} - \text{logits\_max}.unsqueeze(-1) \quad \in \mathbb{R}^{BT \times V}$$ $$\text{exp\_logits} = \exp(\text{logits\_sub}) \quad \in \mathbb{R}^{BT \times V}$$ $$\text{exp\_logits} \gets \frac{\text{exp\_logits}}{\text{sum\_exp\_logits}.unsqueeze(-1)} \quad \in \mathbb{R}^{BT \times V}$$ $$\text{grad\_logits} = \text{exp\_logits} \quad \in \mathbb{R}^{BT \times V}$$ $$\text{grad\_2d} = \text{grad\_logits}.view(-1, \text{partition\_vocab\_size}) \quad \in \mathbb{R}^{BT \times V}$$ $$\text{arange\_1d} = [0, 1, \dots, BT-1] \quad \in \mathbb{N}^{BT}$$ $$\text{softmax\_update} = 1 - \text{target\_mask}.view(-1) \quad \in \mathbb{R}^{BT}$$ $$\text{grad\_2d}[\text{arange\_1d}, \text{masked\_target\_1d}] \gets \text{grad\_2d}[\text{arange\_1d}, \text{masked\_target\_1d}] - \text{softmax\_update}$$ $$\text{grad\_logits} \gets \text{grad\_logits} \odot \text{grad}.unsqueeze(-1) \quad \in \mathbb{R}^{BT \times V}$$ $$\text{grad\_input} = \text{grad\_logits} \cdot \text{weight} \quad \in \mathbb{R}^{BT \times H}$$ $$\text{grad\_weight} = \text{grad\_logits}^T \cdot \text{input} \quad \in \mathbb{R}^{V \times H}$$ [object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

• [object Object]
• [object Object]

[object Object]

• Parameters:

  • grad (aclTensor*, computation input): gradient of the current node, grad in the formula, and aclTensor on the device. The data type can be FLOAT32, and the shape can be 1D. and empty tensors are supported. The can be ND.
  • input (aclTensor*, computation input): input matrix of matrix multiplication, input in the formula, and aclTensor on the device. The data type can be FLOAT16 or BFLOAT16. The shape can be 2D, and the length of the first dimension is the same as that of grad. and empty tensors are supported. The can be ND.
  • weight (aclTensor*, computation input): weight matrix of matrix multiplication, weight in the formula, and aclTensor on the device. The data type can be FLOAT16 or BFLOAT16, which is the same as that of input. The shape can be 2D. The length of the first dimension cannot be less than 128, and the length of the second dimension is the same as that of the second dimension of input. and empty tensors are supported. The can be ND.
  • targetMask (aclTensor*, computation input): intermediate variable, target_mask in the formula, and aclTensor on the device. This parameter indicates whether the corresponding word ID is within the target range. The data type can be UINT8. Each bit represents a Boolean value, where 0 indicates false and 1 indicates true. The shape can be 1D, and the length multiplied by 8 must be greater than or equal to the length of grad. and empty tensors are supported. The can be ND.
  • maskedTarget (aclTensor*, computation input): intermediate variable, masked_target in the formula, and aclTensor on the device. This parameter indicates the local index of the corresponding word ID mapped to the vocabulary shard of the current device. Invalid targets are processed by targetMask. The data type can be INT64 or INT32. The shape can be 1D, and the length is the same as that of grad. and empty tensors are supported. The can be ND.
  • labelSmoothing (float, computation input): label smoothing coefficient, which is used to alleviate overfitting. Currently, the value can only be 0. This is an attribute parameter.
  • logitsMaxOptional (aclTensor*, optional input): intermediate variable, maximum value of global logits, logits_max in the formula, and aclTensor on the device. This parameter is optional, with value nullptr supported. If this parameter is set to nullptr, a valid value of softmaxOptional must be provided. The data type can be FLOAT32. The shape can be 1D, and the length is the same as that of grad. and empty tensors are supported. The can be ND.
  • sumExpLogitsOptional (aclTensor*, optional input): intermediate variable, processed logits, sum_exp_logits in the formula, and aclTensor on the device. This parameter is optional, with value nullptr supported. If this parameter is set to nullptr, a valid value of softmaxOptional must be provided. The data type can be FLOAT32. The shape can be 1D, and the length is the same as that of grad. and empty tensors are supported. The can be ND.
  • softmaxOptional (aclTensor*, computation input): intermediate variable, result of matrix multiplication, softmax in the formula, and aclTensor on the device. This parameter is optional, with value nullptr supported. If this parameter is set to nullptr, valid values of logitsMaxOptional and sumExpLogitsOptional must be provided. If this parameter is set to any other value, the values of logitsMaxOptional and sumExpLogitsOptional are invalid. The data type can be FLOAT32, and the shape can be 2D. The length of the first dimension is the same as that of grad, and the length of the second dimension is the same as that of the first dimension of weight. and empty tensors are supported. The can be ND.
  • inputGradOut (aclTensor*, computation output): gradient of the leaf node input, grad_input in the formula, and aclTensor on the device. The data type can be FLOAT16 or BFLOAT16, which is the same as that of input. The shape can be 2D. The length of the first dimension is the same as that of grad, and the length of the second dimension is the same as that of the second dimension of weight. The can be ND.
  • weightGradOut (aclTensor*, computation output): gradient of the leaf node weight, grad_weight in the formula, and aclTensor on the device. The data type can be FLOAT16 or BFLOAT16, which is the same as that of input. The shape can be 2D. The length of the first dimension is the same as that of the first dimension of weight, and the length of the second dimension is the same as that of grad. The can be ND.
  • workspaceSize (uint64_t*, output): size of the workspace to be allocated on the device.
  • executor (aclOpExecutor**, output): operator executor, containing the operator computation process.

• Returns:

  aclnnStatus: status code. For details, see .

[object Object]

• Parameters:

  • workspace (void*, input): address of the workspace to be allocated on the device.
  • workspaceSize (uint64_t, input): size of the workspace to be allocated on the device, which is obtained by calling the first-phase API aclnnFusedLinearCrossEntropyLossGradGetWorkspaceSize.
  • executor (aclOpExecutor*, input): operator executor, containing the operator computation process.
  • stream (aclrtStream, input): stream for executing the task.

• Returns:

  aclnnStatus: status code. For details, see .

[object Object]

• Deterministic description:
  • [object Object]Atlas training series products[object Object] and [object Object]Atlas inference series products[object Object]: aclnnFusedLinearCrossEntropyLossGrad defaults to a deterministic implementation.

[object Object]

The following example is for reference only. For details, see .