SoftmaxGradFront

Applicability

Function Usage

If the product of non-last axis lengths of the input tensor [m₀, m₁, ..., m_t, n] (t ≥ 0) is considered as m, the shape of the input tensor is [m, n]. This API performs GradFront backward propagation on input tensor[m, n]. The formula is as follows.

When the input shape is in ND format, the internal reduction process is performed along the last axis. When the input shape is in NZ format, the internal reduction process is performed along the last and first axes. For details about the reduction process, see the figures in SoftMax.

For ease of understanding, the formula expressed through a Python script is as follows, where dx and y are the source operands (input), and d is the destination operand (output).

Principles

The following figure shows the internal algorithm diagram of the SoftmaxGradFront high-level APIs by taking the input tensor of the float type, in ND format, and with shape [m, k] as an example.

Figure 1 Diagram of the SoftmaxGradFront algorithm

The computation process is divided into the following steps, all of which are performed on vectors:

1. mul: Multiply all data of input x and y. The computation result is saved to a temporary space temp.
2. reducesum: Sum up each row of temp data [m, k] to obtain [m, 1]. The computation result is saved to the temporary space.
3. broadcast: Pad [m, 1] by data block. For example, for the float type, extend [m, 1] to [m, 8] and output z.

Prototype

• Allocate the temporary space through the API framework.

  1 2	template <typename T, bool isBasicBlock = false, bool isDataFormatNZ = false> __aicore__ inline void SoftmaxGradFront(const LocalTensor<T>& dstTensor, const LocalTensor<T>& gradTensor, const LocalTensor<T>& srcTensor, const SoftMaxTiling& tiling, const SoftMaxShapeInfo& softmaxShapeInfo = {})

• Pass the temporary space through the sharedTmpBuffer input parameter.

  1 2

  template <typename T, bool isBasicBlock = false, bool isDataFormatNZ = false> __aicore__ inline void SoftmaxGradFront(const LocalTensor<T>& dstTensor, const LocalTensor<T>& gradTensor, const LocalTensor<T>& srcTensor, const LocalTensor<uint8_t>& sharedTmpBuffer, const SoftMaxTiling& tiling, const SoftMaxShapeInfo& softmaxShapeInfo = {})

Due to the complex computation involved in the internal implementation of this API, additional temporary space is required to store intermediate variables generated during computation. The temporary space can be allocated through the API framework or passed by developers through the sharedTmpBuffer input parameter.

• When the API framework is used for temporary space allocation, developers do not need to allocate the space, but must reserve the required size for the space.

• When the sharedTmpBuffer input parameter is used for passing the temporary space, the tensor serves as the temporary space. In this case, the API framework is not required for temporary space allocation. This enables developers to manage the sharedTmpBuffer space and reuse the buffer after calling the API, so that the buffer is not repeatedly allocated and deallocated, improving the flexibility and buffer utilization.

If the API framework is used, developers must reserve the temporary space. If sharedTmpBuffer is used, developers must allocate space for the tensor. The method of obtaining the temporary space size (BufferSize) is as follows: Obtain the required maximum and minimum temporary space sizes using the GetSoftMaxGradMaxTmpSize/GetSoftMaxGradMinTmpSize API provided in SoftmaxGrad Tiling. The minimum space can ensure correct functionality, while the maximum space is used to improve performance.

Parameters

1
2
3
4
5
6

struct SoftMaxShapeInfo {
uint32_t srcM; // Product of lengths of non-last axes.
uint32_t srcK; // Length of the last axis, which must be 32-byte aligned.
uint32_t oriSrcM; // Product of lengths of original non-last axes.
uint32_t oriSrcK; // Length of the original last axis.
};

Returns

None

Restrictions

• For details about the operand address alignment requirements, see General Address Alignment Restrictions.
• The address of sharedTmpBuffer cannot overlap that of the source or destination operand.
• When srcM ! is set to oriSrcM or srcK ! is set to oriSrcK in softmaxShapeInfo, for the original input (oriSrcM, oriSrcK) on the GM, you need to pad data to (srcM, srcK) in the M or K direction. The padded data will be involved in some computation. In the scenario where the input and output are reused, the computation result of the API will overwrite the original data padded to the srcTensor. In the scenario where the input and output are not reused, the computation result of the API will overwrite the data in dstTensor corresponding to the padded position of srcTensor.

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

// dstLocal: tensor for storing the SoftmaxGradFront computation result
// gradLocal: input tensor for storing data for SoftmaxGradFront computation
// srcLocal: input tensor for storing data for SoftmaxGradFront computation
// sharedTmpBuffer: tensor for storing the temporary buffer during SoftmaxGradFront computation
// softmaxTiling: structure for storing the tiling information required for SoftmaxGradFront computation, which can be obtained through the SoftMaxGradTilingFunc API

AscendC::SoftMaxShapeInfo softmaxInfo(
    /* Product of lengths of non-last axes          */ srcM, 
    /* Length of the last axis, which must be 32-byte aligned */ srcK, 
    /* Product of lengths of original non-last axes      */ oriSrcM, 
    /* Length of the original last axis              */ oriSrcK
);

// Pass the temporary space through the sharedTmpBuffer input parameter.
AscendC::SoftmaxGradFront<T>(dstLocal, gradLocal, srcLocal, sharedTmpBuffer, softmaxTiling, softmaxInfo);
// Allocate the temporary space through the API framework.
AscendC::SoftmaxGradFront<T>(dstLocal, gradLocal, srcLocal, softmaxTiling, softmaxInfo);

Result example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

Input (gradLocal):
[[-100.     -80.     -60.     -50.     -30.     -20.     -15.     -10.   ]
 [  -9.      -8.      -7.      -6.      -5.      -4.      -3.      -2.   ]
 [  -1.5     -1.      -0.8     -0.6     -0.5     -0.45    -0.4     -0.35 ]
 [  -0.3     -0.25    -0.2     -0.15    -0.1     -0.05    -0.01    -0.001]
 [   0.       0.001    0.01     0.05     0.1      0.15     0.2      0.25 ]
 [   0.3      0.35     0.4      0.45     0.5      0.6      0.8      1.   ]
 [   1.5      2.       3.       4.       5.       6.       7.       8.   ]
 [   9.      10.      15.      20.      30.      50.      60.      80.   ]]
Input (srcLocal):
[[2. 2. 2. 2. 2. 2. 2. 2.]
 ...
 [2. 2. 2. 2. 2. 2. 2. 2.]]
Output (dstLocal):
[[-730.   ]
 [ -88.   ]
 [ -11.2  ]
 [  -2.122]
 [   1.522]
 [   8.8  ]
 [  73.   ]
 [ 548.   ]]