aclnnBatchNormBackward

📄 View Source Code

Applicable Products

Function

• Description: Backward propagation of aclnnBatchNorm. Computes the gradient of the input tensor to update model parameters during backward propagation.

• Formula:

  • When training is true:

    $$gradInput = \frac{weight}{ n{\sqrt{saveVar + eps}} }(n * gradOut - \sum^m_{i=0}{gradOut} - \frac{x-saveMean}{ {\sqrt{saveVar + eps}} }\sum^m_{i=0}({gradOut} *\frac{x-saveMean}{ {\sqrt{saveVar + eps}} } ))$$ $$gradWeight = \sum^m_{i=0}[{gradOut} * (x - saveMean)] * \frac{1}{ {\sqrt{saveVar + eps}} }$$ $$gradBias = \sum^m_{i=0}{gradOut}$$

  • When training is false:

    $$gradInput = gradOut * \frac{1}{ {\sqrt{runningVar + eps}} } * weight$$ $$gradWeight = \sum^m_{i=0}[{gradOut} * (x - runningMean)] * \frac{1}{ {\sqrt{runningVar + eps}} }$$ $$gradBias = \sum^m_{i=0}{gradOut}$$

Function Prototype

Each operator is divided into a two-phase API. You must first call the "aclnnBatchNormBackwardGetWorkspaceSize" API to obtain the input parameters and calculate the required workspace size based on the process, and then call the "aclnnBatchNormBackward" API to perform the computation.

aclnnStatus aclnnBatchNormBackwardGetWorkspaceSize(
  const aclTensor    *gradOut,
  const aclTensor    *input,
  const aclTensor    *weight,
  const aclTensor    *runningMean,
  const aclTensor    *runningVar,
  const aclTensor    *saveMean,
  const aclTensor    *saveInvstd,
  bool                training,
  double              eps,
  const aclBoolArray *outputMask,
  aclTensor          *gradInput,
  aclTensor          *gradWeight,
  aclTensor          *gradBias,
  uint64_t           *workspaceSize,
  aclOpExecutor     **executor)

aclnnStatus aclnnBatchNormBackward(
  void                *workspace,
  uint64_t             workspaceSize,
  aclOpExecutor       *executor,
  const aclrtStream    stream)

aclnnBatchNormBackwardGetWorkspaceSize

• Parameters

  Parameter	Input/Output	Description	Instruction	Data Type	Data Format	Dimension (Shape)	Non-contiguous Tensor
  gradOut (aclTensor*)	Input	Represents the gradient tensor, corresponding to `gradOut` in the formula.	Supports empty tensors. Supported shapes and formats include: 2D (format: NC), 3D (format: NCL), 4D (formats: NCHW, NHWC), 5D (formats: NCDHW, NDHWC), and 6-8D (format: ND, where the second dimension is fixed as the channel axis).	FLOAT32, FLOAT16, BFLOAT16	NC, NCL, NCHW, NHWC, NCDHW, NDHWC, ND	2-8	√
  input (aclTensor*)	Input.	Represents the forward input tensor, corresponding to `x` in the formula.	Supports empty tensors. The data type, shape, and data format must be consistent with `gradOut`.	FLOAT32, FLOAT16, BFLOAT16	NC, NCL, NCHW, NHWC, NCDHW, NDHWC, ND	2-8	✓
  weight (aclTensor*)	Input	Represents the weight tensor, corresponding to `weight` in the formula.	Supports empty tensors. The shape length equals the length of the channel axis in the input parameter `input`.	FLOAT32, FLOAT16, BFLOAT16	ND	1	√
  runningMean (aclTensor*)	Input	Mean value computed during training, corresponding to `runningMean` in the formula.	Supports empty tensor. The shape length equals the length of the channel axis in the input parameter `input`.	FLOAT32, FLOAT16, BFLOAT16	ND	1	√
  runningVar (aclTensor*)	Input	Indicates the variance computed during training, corresponding to `runningVar` in the formula.	Supports empty tensors. The value is non-negative. The shape length equals the length of the channel axis in the input parameter input.	FLOAT32, FLOAT16, BFLOAT16	ND	1	√
  saveMean (aclTensor*)	Input	Represents the saved mean, corresponding to `saveMean` in the formula.	Supports empty tensors. The shape length equals the length of the channel axis in the input parameter `input`, or is 0 in inference scenarios.	FLOAT32, FLOAT16, BFLOAT16	ND	1	√
  saveInvstd (aclTensor*)	Input	Represents the reciprocal of the saved standard deviation, corresponding to the reciprocal of the square root of (Var(x) + eps) in the formula.	Supports empty tensors. The value is non-negative. The shape length equals the length of the channel axis in the input parameter `input`, or is 0 in inference scenarios.	FLOAT32, FLOAT16, BFLOAT16	ND	1	√
  training (bool)	Input	Indicates whether it is a training scenario, corresponding to `training` in the formula description.	true indicates a training scenario, and false indicates an inference scenario.	-	-	-	-
  eps (double)	Input	Indicates the value added to the variance to avoid division by zero. Corresponds to `eps` in the formula.	-	-	-	-	-
  outputMask (aclBoolArray*)	Input	Output mask.	Size is 3. Indicates whether to output `gradInput`, `gradWeight`, and `gradBias` respectively. If true, the corresponding output is produced; otherwise, the corresponding position returns null.	BoolArray	-	-	-
  gradInput (aclTensor*)	Output	Indicates the gradient of the input tensor, corresponding to `gradInput` in the formula.	Supports empty tensors. Optional output. If outputMask[0] is True, this output is required; otherwise, it is not output. The data type, shape, and data format must be consistent with `gradOut`.	FLOAT32, FLOAT16, BFLOAT16	NC, NCL, NCHW, NHWC, NCDHW, NDHWC, ND	2-8	√
  gradWeight (aclTensor*)	Output	Represents the gradient of the scaling parameter, corresponding to `gradWeight` in the formula.	Supports empty tensors. Optional output. Output is required if outputMask[1] is True; otherwise, no output is produced. The length equals the length of the channel axis in the input `input`.	FLOAT32, FLOAT16, BFLOAT16	ND	1	√
  gradBias (aclTensor*)	Output	Represents the gradient of the bias parameter, corresponding to `gradBias` in the formula.	Supports empty tensors. Optional output. If outputMask[2] is True, this output is required; otherwise, it is not output. The shape length equals the length of the channel axis in the input parameter `input`.	FLOAT32, FLOAT16, BFLOAT16	ND	1	√
  workspaceSize (uint64_t*)	Output	Returns the workspace size that the user needs to apply for on the Device side.	-	-	-	-	-
  executor (aclOpExecutor**)	Output	Returns the op executor, which contains the operator computation flow.	-	-	-	-	-

  • Atlas training products, Atlas inference products:

    • The data type of the parameters gradOut, input, weight, runningMean, runningVar, saveMean, saveInvstd, gradInput, gradWeight, and gradBias does not support BFLOAT16.

    • The data type of the parameters weight, runningMean, runningVar, saveMean, saveInvstd, gradWeight, and gradBias is consistent with that of gradOut.

    • The data format of the parameters gradOut, input, and gradInput does not support NHWC or NDHWC.

  • Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products:

    • The data type of the parameters weight, runningMean, runningVar, saveMean, saveInvstd, gradWeight, and gradBias is consistent with that of gradOut.

    • The data format of the parameters gradOut, input, and gradInput does not support NHWC or NDHWC.

  • Ascend 950PR/Ascend 950DT:

    • When gradOut is FLOAT32, weight, runningMean, runningVar, saveMean, and saveInvstd must be FLOAT32.

    • When gradOut is FLOAT16, weight, terrunningMean, runningVar, saveMean, and saveInvstd must be FLOAT16 or FLOAT32.

    • When gradOut is BFLOAT16, weight, runningMean, runningVar, saveMean, and saveInvstd must be BFLOAT16 or FLOAT32.

    • When weight is FLOAT32, gradWeight must be FLOAT32; when weight is FLOAT16, gradWeight must be FLOAT16 or FLOAT32; when weight is BFLOAT16, gradWeight must be BFLOAT16 or FLOAT32.

    • The data type of runningVar must be the same as that of runningMean.

    • The data type of saveInvstd must be the same as that of saveMean.

• Return Value

  aclnnStatus: return code. For details, see aclnn Return Code.

  The first-phase API performs input parameter validation and returns an error in the following scenarios:

  Return Code	Error Code	Description
  ACLNN_ERR_PARAM_NULLPTR	161001	The passed gradOut or input pointer is null.
  		When outputMask[0] is true, the passed gradInput pointer is null.
  		When outputMask[1] is true, the passed gradWeight pointer is null.
  		When outputMask[2] is true and the passed gradBias is a null pointer.
  ACLNN_ERR_PARAM_INVALID	161002	The data type, data format, or shape of input or gradOut is not within the supported range.
  		When weight, runningMean, runningVar, saveMean, or saveInvstd is not null, its data type, data format, or shape is not within the supported range.
  		The length of outputMask is not 3.
  		When outputMask[0] is true, the data type, data format, or shape of gradInput is not within the supported range.
  		When outputMask[1] is true, the data type, data format, or shape of gradWeight is not within the supported range.
  		When outputMask[2] is true, the data type, data format, or shape of gradBias is not within the supported range.
  		The shape length of weight, runningMean, runningVar, saveMean, saveInvstd, gradWeight (when non-null), or gradBias (when non-null) is not equal to the length of the channel axis in the input shape.
  		The data formats of input, gradOut, and gradInput (when non-null) are inconsistent.
  		The data types of input, gradOut, and gradInput (when non-null) are inconsistent.
  		The shapes of input, gradOut, and gradInput (when non-null) are inconsistent, or the shape has more than 8 dimensions or fewer than 2 dimensions.

aclnnBatchNormBackward

• Parameters

  Parameter	Input/Output	Description
  workspace	Input	Address of the workspace memory applied for on the Device side.
  workspaceSize	Input	Size of the workspace applied for on the Device side, obtained through the first-phase API aclnnBatchNormBackwardGetWorkspaceSize.
  executor	Input	The op executor, which contains the operator computation flow.
  stream	Input	Specifies the stream for task execution.

• Return Value

  aclnnStatus: return code. For details, see aclnn Return Code.

Constraints

• Deterministic computation:

  • aclnnBatchNormBackward defaults to a non-deterministic implementation. Deterministic computation can be enabled through aclrtCtxSetSysParamOpt.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_batch_norm_backward.h"

#define CHECK_RET(cond, return_expr) \
    do {                             \
        if (!(cond)) {               \
            return_expr;             \
        }                            \
    } while (0)

#define LOG_PRINT(message, ...)         \
    do {                                \
        printf(message, ##__VA_ARGS__); \
    } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape)
{
    int64_t shapeSize = 1;
    for (auto i : shape) {
        shapeSize *= i;
    }
    return shapeSize;
}

int Init(int32_t deviceId, aclrtStream* stream)
{
    // Boilerplate: resource initialization.
    auto ret = aclInit(nullptr);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
    ret = aclrtSetDevice(deviceId);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
    ret = aclrtCreateStream(stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
    return 0;
}

template <typename T>
int CreateAclTensor(
    const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr, aclDataType dataType,
    aclTensor** tensor)
{
    auto size = GetShapeSize(shape) * sizeof(T);
    // Call aclrtMalloc to apply for device memory.
    auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);

    // Copy data from the host to the device memory by calling aclrtMemcpy.
    ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

    // Calculate the strides of a contiguous tensor.
    std::vector<int64_t> strides(shape.size(), 1);
    for (int64_t i = shape.size() - 2; i >= 0; i--) {
        strides[i] = shape[i + 1] * strides[i + 1];
    }

    // Call the aclCreateTensor API to create an aclTensor.
    *tensor = aclCreateTensor(
        shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND, shape.data(), shape.size(),
        *deviceAddr);
    return 0;
}

int main()
{
    // 1. (Boilerplate) Initialize the device/stream. See the ACL API manual.
    // Set deviceId based on the actual device.
    int32_t deviceId = 0;
    aclrtStream stream;
    auto ret = Init(deviceId, &stream);
    // Handle the check based on your own needs.
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);

    // 2. Construct the input and output. Customize the construction based on the API interface.
    std::vector<int64_t> gradOutShape = {1, 2, 4};
    std::vector<int64_t> selfShape = {1, 2, 4};
    std::vector<int64_t> weightShape = {2};
    std::vector<int64_t> rMeanShape = {2};
    std::vector<int64_t> rVarShape = {2};
    std::vector<int64_t> sMeanShape = {2};
    std::vector<int64_t> sVarShape = {2};
    std::vector<int64_t> gradInShape = {1, 2, 4};
    std::vector<int64_t> gradWeightShape = {2};
    std::vector<int64_t> gradBiasShape = {2};
    void* gradOutDeviceAddr = nullptr;
    void* selfDeviceAddr = nullptr;
    void* weightDeviceAddr = nullptr;
    void* rMeanDeviceAddr = nullptr;
    void* rVarDeviceAddr = nullptr;
    void* sMeanDeviceAddr = nullptr;
    void* sVarDeviceAddr = nullptr;
    void* outMaskDeviceAddr = nullptr;
    void* gradInDeviceAddr = nullptr;
    void* gradWeightDeviceAddr = nullptr;
    void* gradBiasDeviceAddr = nullptr;
    aclTensor* gradOut = nullptr;
    aclTensor* self = nullptr;
    aclTensor* weight = nullptr;
    aclTensor* rMean = nullptr;
    aclTensor* rVar = nullptr;
    aclTensor* sMean = nullptr;
    aclTensor* sVar = nullptr;
    aclBoolArray* outMask = nullptr;
    aclTensor* gradIn = nullptr;
    aclTensor* gradWeight = nullptr;
    aclTensor* gradBias = nullptr;
    std::vector<float> gradOutHostData = {0, 1, 2, 3, 4, 5, 6, 7};
    std::vector<float> selfHostData = {0, 1, 2, 3, 4, 5, 6, 7};
    std::vector<float> weightHostData = {1, 1};
    std::vector<float> rMeanHostData = {0, 0};
    std::vector<float> rVarHostData = {1, 1};
    std::vector<float> sMeanHostData = {0, 0};
    std::vector<float> sVarHostData = {1, 1};
    std::vector<float> gradInHostData(8, 0);
    std::vector<float> gradWeightHostData(2, 0);
    std::vector<float> gradBiasHostData(2, 0);

    bool training = true;
    double eps = 1e-5;
    // Create the gradOut aclTensor.
    ret = CreateAclTensor(gradOutHostData, gradOutShape, &gradOutDeviceAddr, aclDataType::ACL_FLOAT, &gradOut);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the self aclTensor.
    ret = CreateAclTensor(selfHostData, selfShape, &selfDeviceAddr, aclDataType::ACL_FLOAT, &self);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the weight aclTensor.
    ret = CreateAclTensor(weightHostData, weightShape, &weightDeviceAddr, aclDataType::ACL_FLOAT, &weight);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the rMean aclTensor.
    ret = CreateAclTensor(rMeanHostData, rMeanShape, &rMeanDeviceAddr, aclDataType::ACL_FLOAT, &rMean);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the rVar aclTensor.
    ret = CreateAclTensor(rVarHostData, rVarShape, &rVarDeviceAddr, aclDataType::ACL_FLOAT, &rVar);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the sMean aclTensor.
    ret = CreateAclTensor(sMeanHostData, sMeanShape, &sMeanDeviceAddr, aclDataType::ACL_FLOAT, &sMean);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the sVar aclTensor.
    ret = CreateAclTensor(sVarHostData, sVarShape, &sVarDeviceAddr, aclDataType::ACL_FLOAT, &sVar);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the outMask aclBoolArray.
    bool maskData[3] = {true, true, true};
    outMask = aclCreateBoolArray(&(maskData[0]), 3);
    // Create gradIn aclTensor.
    ret = CreateAclTensor(gradInHostData, gradInShape, &gradInDeviceAddr, aclDataType::ACL_FLOAT, &gradIn);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create gradWeight aclTensor.
    ret = CreateAclTensor(
        gradWeightHostData, gradWeightShape, &gradWeightDeviceAddr, aclDataType::ACL_FLOAT, &gradWeight);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create gradBias aclTensor.
    ret = CreateAclTensor(gradBiasHostData, gradBiasShape, &gradBiasDeviceAddr, aclDataType::ACL_FLOAT, &gradBias);
    CHECK_RET(ret == ACL_SUCCESS, return ret);

    uint64_t workspaceSize = 0;
    aclOpExecutor* executor;

    // Example of aclnnBatchNormBackward API call.
    // 3. Call the CANN operator library API. Replace it with the specific API name.
    // Call the first-phase API of aclnnBatchNormBackward.
    ret = aclnnBatchNormBackwardGetWorkspaceSize(
        gradOut, self, weight, rMean, rVar, sMean, sVar, training, eps, outMask, gradIn, gradWeight, gradBias,
        &workspaceSize, &executor);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnBatchNormBackwardGetWorkspaceSize failed. ERROR: %d\n", ret);
              return ret);
    // Apply for device memory based on the workspaceSize calculated by the first-phase API.
    void* workspaceAddr = nullptr;
    if (workspaceSize > 0) {
        ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
        CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
    }
    // Call the second-phase API of aclnnBatchNormBackward.
    ret = aclnnBatchNormBackward(workspaceAddr, workspaceSize, executor, stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnBatchNormBackward failed. ERROR: %d\n", ret); return ret);

    // 4. (Boilerplate) Synchronize and wait for task execution to complete.
    ret = aclrtSynchronizeStream(stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

    // 5. Obtain the output value and copy the result from the device memory to the host memory. Modify based on the specific API definition.
    auto size = GetShapeSize(gradInShape);
    std::vector<float> resultData(size, 0);
    ret = aclrtMemcpy(
        resultData.data(), resultData.size() * sizeof(resultData[0]), gradInDeviceAddr, size * sizeof(resultData[0]),
        ACL_MEMCPY_DEVICE_TO_HOST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
    for (int64_t i = 0; i < size; i++) {
        LOG_PRINT("result[%ld] is: %f\n", i, resultData[i]);
    }

    // 6. Release aclTensor and aclScalar. Modify based on the specific API definition.
    aclDestroyTensor(gradOut);
    aclDestroyTensor(self);
    aclDestroyTensor(weight);
    aclDestroyTensor(rMean);
    aclDestroyTensor(rVar);
    aclDestroyTensor(sMean);
    aclDestroyTensor(sVar);
    aclDestroyBoolArray(outMask);
    aclDestroyTensor(gradIn);
    aclDestroyTensor(gradWeight);
    aclDestroyTensor(gradBias);

    // 7. Release device resources. Modify based on the specific API definition.
    aclrtFree(gradOutDeviceAddr);
    aclrtFree(selfDeviceAddr);
    aclrtFree(weightDeviceAddr);
    aclrtFree(rMeanDeviceAddr);
    aclrtFree(rVarDeviceAddr);
    aclrtFree(sMeanDeviceAddr);
    aclrtFree(sVarDeviceAddr);
    aclrtFree(gradInDeviceAddr);
    aclrtFree(gradWeightDeviceAddr);
    aclrtFree(gradBiasDeviceAddr);
    if (workspaceSize > 0) {
        aclrtFree(workspaceAddr);
    }
    aclrtDestroyStream(stream);
    aclrtResetDevice(deviceId);
    aclFinalize();
    return 0;
}