AdjustSoftMaxRes

Applicability

Product	Supported
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	x
Atlas inference product's AI Core	√
Atlas inference product's Vector Core	x
Atlas training products	x

Function

Adjusts the computation result of SoftMax to a specified value. It is mainly used to postprocess SoftMax computation results. If a specified value exists in the input max, the result in the corresponding softmaxres is adjusted to a user-defined value. The preceding adjustment is performed by row. That is, when the value of max in a row is specified, the value of softmaxres in the corresponding row is adjusted to a user-defined value.

For ease of understanding, the formula expressed through a Python script is as follows, where res is both input and output, and max, from, to, and res_shape are inputs.

def adjust_softmax_res(res, max, from, to, res_shape):
    for i in res_shape[0]:
        if max[i] == from:
            for j in res_shape[1]:
                res[i][j] = to
    return

Prototype

template <typename T1, typename T2, bool isDataFormatNZ = false, uint8_t stepSizeMode = 0>
__aicore__ inline bool AdjustSoftMaxRes(const LocalTensor<T1>& softMaxRes, const LocalTensor<T2>& maxTensor, const uint32_t from, const T1 to, const SoftMaxShapeInfo& softmaxShapeInfo)

Parameters

**Table 1** Template parameters
Parameter	Description
T1	Data type of softMaxRes. For the Atlas A3 training products/Atlas A3 inference products, the supported data types are half and float. For the Atlas A2 training products/Atlas A2 inference products, the supported data types are half and float. For the Atlas inference product's AI Core, the supported data types are half and float.
T2	Data type of maxTensor. For the Atlas A3 training products/Atlas A3 inference products, the supported data types are half and float. For the Atlas A2 training products/Atlas A2 inference products, the supported data types are half and float. For the Atlas inference product's AI Core, the supported data types are half and float.
isDataFormatNZ	Whether the current input and output data is in NZ format. The default data format is ND, that is, the default value of this parameter is false.
stepSizeMode	Mode of the step length for obtaining elements from maxTensor. The values are as follows: 0: default value. In each BlockSize (32 bytes), the value of the first element is compared with the value of the input from. That means, according to the input shape (m, 8), when the data type of maxTensor is float, one element is obtained from every eight elements. According to the input shape (m, 16), when the data type of maxTensor is half, one element is obtained every 16 elements. Non-zero: The value of each element in maxTensor is compared with the value of the input from. That means, according to the input shape (m, 1), the value of one element is compared with the value of the input from. If this parameter is set to a non-zero value, maxTensor can only be in ND format.

Table 2 API parameters

Parameter

Input/Output

Description

softMaxRes

Input/Output

Source operand and destination operand.

The type is LocalTensor, and the supported TPosition is VECIN, VECCALC, or VECOUT.

For details about the definition of the LocalTensor data structure, see LocalTensor.

The length of the last axis must be 32-byte aligned.

Its value is the output result of softmax computation.

maxTensor

Input

Source operand.

The type is LocalTensor, and the supported TPosition is VECIN, VECCALC, or VECOUT.

Result of reducemax during softmax computation.

The length of the last axis of maxTensor is fixed at 32 bytes. It is the length of a data block. All data in this data block has the same value. For example, in the half data type, all 16 numbers in this data block possess an identical reducemax value.
The length of each non-last axis is the same as that of softMaxRes.

from

Input

Source operand of the uint32_t type.

Value of maxTensor to be checked. Note that the values of maxTensor are floating point numbers. Therefore, you need to input a hexadecimal value corresponding to a floating point number. For example, if you want to check whether the value of maxTensor is 1.0, set from to the hexadecimal value 0x3f800000 corresponding to 1.0.

Input

Source operand. The data type is the same as that of softMaxRes.

Value to be padded in softMaxRes.

softmaxShapeInfo

Input

Shape information of softMaxRes. The structure is defined as follows:

struct SoftMaxShapeInfo {
uint32_t srcM; // Product of lengths of non-last axes.
uint32_t srcK; // Length of the last axis, which must be 32-byte aligned.
uint32_t oriSrcM; // Product of lengths of original non-last axes.
uint32_t oriSrcK; // Length of the original last axis.
};

Currently, only ND input is supported.

Returns

A Boolean value is returned. true indicates that there is a value to be checked exists in maxTensor. false indicates that there is no value to be checked exists in maxTensor.

Restrictions

For details about the operand address alignment requirements, see General Address Alignment Restrictions.
When srcM ! is set to oriSrcM or srcK ! is set to oriSrcK in softmaxShapeInfo, for the original input (oriSrcM, oriSrcK) on the GM, you need to pad data to (srcM, srcK) in the M or K direction. The padded data will be involved in some computation. In the scenario where the input and output are reused, the computation result of the API will overwrite the original data padded to the srcTensor. In the scenario where the input and output are not reused, the computation result of the API will overwrite the data in dstTensor corresponding to the padded position of srcTensor.

Example

In this sample, postprocessing is performed on the SoftMax computation result to check whether 0xFF7FFFFF exists in maxTensor. If 0xFF7FFFFF exists, the result is updated to 0. In this example, the operator is AdjustSoftMaxResCustom with the fixed shape of input x[32, 32] and output y[32, 32]. The shape of the input softMaxRes is [32, 32], and the shape of the maxTensor is [32, 8]. The data type is float. For a complete operator example, see adjustsoftmaxres operator sample.

AscendC::LocalTensor<float> srcLocal = inQueueSrc.DeQue<float>();
AscendC::LocalTensor<float> maxLocal = inQueueMax.DeQue<float>();
AscendC::LocalTensor<float> dstLocal = outQueueDst.AllocTensor<float>();
AscendC::LocalTensor<float> tmpTensor = calcBuf.Get<float>();
AscendC::SoftMaxShapeInfo srcShape = {height, width, height, width};
AscendC::AdjustSoftMaxRes<float, float>(srcLocal, maxLocal, FROM, TO, srcShape);
AscendC::DataCopy(tmpTensor, srcLocal, height * width);
AscendC::DataCopy(dstLocal, tmpTensor, height * width);

Parent topic: SoftMax APIs