aclnnFastBatchNormBackward

Applicable Products

Product	Supported
Ascend 950PR/Ascend 950DT	×
Atlas A3 training products/Atlas A3 inference products	×
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	×
Atlas inference products	×
Atlas training products	×

Function

Description: backward propagation (high-performance version) of aclnnBatchNorm. Computes the gradient of the input tensor for updating model parameters during backward propagation.
Formula:
- When training is true:
  $gradInput = \frac{weight}{ n{\sqrt{saveVar + eps}} }(n * gradOut - \sum^m_{i=0}{gradOut} - \frac{x-saveMean}{ {\sqrt{saveVar + eps}} }\sum^m_{i=0}({gradOut} *\frac{x-saveMean}{ {\sqrt{saveVar + eps}} } ))$ $gradWeight = \sum^m_{i=0}[{gradOut} * (x - saveMean)] * \frac{1}{ {\sqrt{saveVar + eps}} }$ $gradBias = \sum^m_{i=0}{gradOut}$
- When training is false:
  $gradInput = gradOut * \frac{1}{ {\sqrt{runningVar + eps}} } * weight$ $gradWeight = \sum^m_{i=0}[{gradOut} * (x - runningMean)] * \frac{1}{ {\sqrt{runningVar + eps}} }$ $gradBias = \sum^m_{i=0}{gradOut}$

Function Prototype

Each operator is divided into a two-phase API. You must first call the "aclnnFastBatchNormBackwardGetWorkspaceSize" API to obtain the input parameters and calculate the required workspace size based on the process, and then call the "aclnnFastBatchNormBackward" API to perform the computation.

aclnnStatus aclnnFastBatchNormBackwardGetWorkspaceSize(
  const aclTensor    *gradOut,
  const aclTensor    *input,
  const aclTensor    *weight,
  const aclTensor    *runningMean,
  const aclTensor    *runningVar,
  const aclTensor    *saveMean,
  const aclTensor    *saveInvstd,
  bool                training,
  double              eps,
  const aclBoolArray *outputMask,
  int                 version,
  aclTensor          *gradInput,
  aclTensor          *gradWeight,
  aclTensor          *gradBias,
  uint64_t           *workspaceSize,
  aclOpExecutor     **executor)

aclnnStatus aclnnFastBatchNormBackward(
  void                *workspace,
  uint64_t             workspaceSize,
  aclOpExecutor       *executor,
  const aclrtStream    stream)

aclnnFastBatchNormBackwardGetWorkspaceSize

Parameters

Parameter	Input/Output	Description	Instruction	Data Type	Data Format	Dimension (Shape)	Non-Contiguous Tensor
gradOut (aclTensor*)	Input	Represents the gradient tensor, corresponding to `gradOut` in the formula.	Nullable tensor is supported. Supported shapes and formats include: 2D (corresponding format is NC), 3D (corresponding format is NCL), 4D (corresponding format is NCHW), 5D (corresponding format is NCDHW), and 6–8D (corresponding format is ND, where the second dimension is fixed as the channel axis).	FLOAT32, FLOAT16, BFLOAT16	NC, NCL, NCHW, NCDHW, ND	2-8	√
input (aclTensor*)	Input	Indicates the forward input tensor, corresponding to `x` in the formula.	Nullable tensor supported. The data type, shape, and data format must be consistent with `gradOut`.	FLOAT32, FLOAT16, BFLOAT16	NC, NCL, NCHW, NCDHW, ND	2–8	√
weight (aclTensor*)	Input	Represents the weight tensor, corresponding to `weight` in the formula.	Nullable support. The shape length is equal to the channel axis length in the incoming `input`.	FLOAT32, FLOAT16, BFLOAT16	ND	1	√
runningMean (aclTensor*)	Input	Indicates the mean computed during training, corresponding to `runningMean` in the formula.	Supports null tensor. The shape length equals the length of the channel axis in the input `input`.	FLOAT32, FLOAT16, BFLOAT16	ND	1	√
runningVar (aclTensor*)	Input	Indicates the variance computed during training, corresponding to `runningVar` in the formula.	Nullable tensor supported. The value is non-negative. The shape length is equal to the length of the channel axis in the input parameter `input`.	FLOAT32, FLOAT16, BFLOAT16	ND	1	✓
saveMean (aclTensor*)	Input	Indicates the saved mean, corresponding to `saveMean` in the formula.	Nullable tensor supported. The shape length is equal to the axis length of the channel dimension in the incoming `input`.	FLOAT32, FLOAT16, BFLOAT16	ND	1	√
saveInvstd (aclTensor*)	Input	Represents the reciprocal of the saved standard deviation, corresponding to the reciprocal of the square root of (Var(x) + eps) in the formula.	Supports null Tensor. The value is a non-negative number. The shape length is equal to the length of the channel axis in the input parameter `input`.	FLOAT32, FLOAT16, BFLOAT16	ND	1	√
training (bool)	Input	Indicates whether it is a training scenario, corresponding to `training` in the formula description.	true indicates a training scenario, and false indicates an inference scenario.	-	-	-	-
eps (double)	Input.	Value added to the variance to avoid division by zero. Corresponds to `eps` in the formula.	-	-	-	-	-
outputMask (aclBoolArray*)	Input	Mask indicating the outputs.	The size is 3. Indicates whether to output `gradInput`, `gradWeight`, and `gradBias` respectively. If true, the corresponding output is returned; otherwise, null is returned at the corresponding position.	BoolArray	-	-	-
version (int)	Input	Indicates the algorithm version number used internally by the operator.	Currently supports optional values: 0, 1. Default value: 0.	-	-	-	-
gradInput (aclTensor*)	Output	Represents the gradient of the input tensor, corresponding to `gradInput` in the formula.	Nullable tensor supported. Optional output. If outputMask[0] is True, then output required; otherwise no output. The data type, shape, and data format must be consistent with `gradOut`.	FLOAT32, FLOAT16, BFLOAT16	NC, NCL, NCHW, NCDHW, ND	2-8	√
gradWeight (aclTensor*)	Output	Represents the gradient of the scaling parameter, corresponding to `gradWeight` in the formula.	Supports null tensor. Optional output. If outputMask[1] is True, then output required; otherwise no output. The length is equal to the channel axis length in the input `input`.	FLOAT32, FLOAT16, BFLOAT16	ND	1	√
gradBias (aclTensor*)	Output	Represents the gradient of the bias parameter, corresponding to `gradBias` in the formula.	Nullable tensor supported. Optional output. If outputMask[2] is True, then output is required; otherwise, no output. The shape length is equal to the length of the channel axis in the input parameter `input`.	FLOAT32, FLOAT16, BFLOAT16	ND	1	√
workspaceSize (uint64_t*)	Output	Returns the workspace size that the user needs to request on the Device side.	-	-	-	-	-
executor (aclOpExecutor**)	Output	Returns the operator executor, which contains the operator computation flow.	-	-	-	-	-

Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products:
- The data types of weight, runningMean, runningVar, saveMean, saveInvstd, gradWeight, and gradBias must be consistent with that of gradOut.

Return Value

aclnnStatus: return code. For details, see aclnn Return Code.

The first-phase API performs input parameter validation and returns an error in the following scenarios:

Return Code	Error Code	Description
ACLNN_ERR_PARAM_NULLPTR	161001	The incoming gradOut and input pointers are null pointers.
		When outputMask[0] is true, the incoming gradInput is a null pointer.
		When outputMask[1] is true and the incoming gradWeight is a null pointer.
		When outputMask[2] is true and the incoming gradBias is a null pointer.
ACLNN_ERR_PARAM_INVALID	161002	The data type, data format, or shape of input or gradOut is not within the supported range.
		When weight, runningMean, runningVar, saveMean, or saveInvstd is non-null, the data type, data format, or shape is not within the supported range.
		The length of outputMask is not 3.
		When outputMask[0] is true and the data type, data format, or shape of gradInput is not within the supported range.
		When outputMask[1] is true and the data type, data format, or shape of gradWeight is not within the supported range.
		When outputMask[2] is true and the data type, data format, or shape of gradBias is not within the supported range.
		The shape length of weight, runningMean, runningVar, saveMean, saveInvstd, gradWeight (when non-null), and gradBias (when non-null) is not equal to the channel axis length in the input shape.
		The data formats of input, gradOut, and gradInput (when non-null) are inconsistent.
		The data types of input, gradOut, and gradInput (when non-null) are inconsistent.
		The shapes of input, gradOut, and gradInput (when non-null) are inconsistent, or the shape dimension is greater than 8 or less than 2.

aclnnFastBatchNormBackward

Parameters

Parameter	Input/Output	Description
workspace	Input	Memory address of the workspace applied on the device side.
workspaceSize	Input	Size of the workspace applied on the device side, obtained from the first-phase API aclnnFastBatchNormBackwardGetWorkspaceSize.
executor	Input	Operator executor, which contains the operator computation flow.
stream	Input	Specifies the stream for task execution.

Return Value

aclnnStatus: return code. For details, see aclnn Return Code.

Constraints

Deterministic computation:
- aclnnFastBatchNormBackward defaults to a deterministic implementation.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_fast_batch_norm_backward.h"

#define CHECK_RET(cond, return_expr) \
  do {                               \
    if (!(cond)) {                   \
      return_expr;                   \
    }                                \
  } while (0)

#define LOG_PRINT(message, ...)     \
  do {                              \
    printf(message, ##__VA_ARGS__); \
  } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape) {
  int64_t shapeSize = 1;
  for (auto i : shape) {
    shapeSize *= i;
  }
  return shapeSize;
}

int Init(int32_t deviceId, aclrtStream* stream) {
  // Boilerplate: resource initialization.
  auto ret = aclInit(nullptr);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
  ret = aclrtSetDevice(deviceId);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
  ret = aclrtCreateStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
  return 0;
}

template <typename T>
int CreateAclTensor(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr,
                    aclDataType dataType, aclTensor** tensor) {
  auto size = GetShapeSize(shape) * sizeof(T);
  // Call aclrtMalloc to request device-side memory.
  auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);

  // Copy data from the host to the device memory by calling aclrtMemcpy.
  ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

  // Compute the strides of a contiguous tensor.
  std::vector<int64_t> strides(shape.size(), 1);
  for (int64_t i = shape.size() - 2; i >= 0; i--) {
    strides[i] = shape[i + 1] * strides[i + 1];
  }

  // Create an aclTensor by calling aclCreateTensor.
  *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND,
                            shape.data(), shape.size(), *deviceAddr);
  return 0;
}

int main() {
  // 1. (Boilerplate) Initialize the device/stream. See the ACL API manual.
  // Enter the actual device ID.
  int32_t deviceId = 0;
  aclrtStream stream;
  auto ret = Init(deviceId, &stream);
  // Handle the check based on your own needs.
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);

  // 2. Construct the input and output. Customize the construction based on the API.
  std::vector<int64_t> gradOutShape = {1, 2, 4};
  std::vector<int64_t> selfShape = {1, 2, 4};
  std::vector<int64_t> weightShape = {2};
  std::vector<int64_t> rMeanShape = {2};
  std::vector<int64_t> rVarShape = {2};
  std::vector<int64_t> sMeanShape = {2};
  std::vector<int64_t> sVarShape = {2};
  std::vector<int64_t> gradInShape = {1, 2, 4};
  std::vector<int64_t> gradWeightShape = {2};
  std::vector<int64_t> gradBiasShape = {2};
  void* gradOutDeviceAddr = nullptr;
  void* selfDeviceAddr = nullptr;
  void* weightDeviceAddr = nullptr;
  void* rMeanDeviceAddr = nullptr;
  void* rVarDeviceAddr = nullptr;
  void* sMeanDeviceAddr = nullptr;
  void* sVarDeviceAddr = nullptr;
  void* outMaskDeviceAddr = nullptr;
  void* gradInDeviceAddr = nullptr;
  void* gradWeightDeviceAddr = nullptr;
  void* gradBiasDeviceAddr = nullptr;
  aclTensor* gradOut = nullptr;
  aclTensor* self = nullptr;
  aclTensor* weight = nullptr;
  aclTensor* rMean = nullptr;
  aclTensor* rVar = nullptr;
  aclTensor* sMean = nullptr;
  aclTensor* sVar = nullptr;
  aclBoolArray* outMask = nullptr;
  aclTensor* gradIn = nullptr;
  aclTensor* gradWeight = nullptr;
  aclTensor* gradBias = nullptr;
  std::vector<float> gradOutHostData = {0, 1, 2, 3, 4, 5, 6, 7};
  std::vector<float> selfHostData = {0, 1, 2, 3, 4, 5, 6, 7};
  std::vector<float> weightHostData = {1, 1};
  std::vector<float> rMeanHostData = {0, 0};
  std::vector<float> rVarHostData = {1, 1};
  std::vector<float> sMeanHostData = {0, 0};
  std::vector<float> sVarHostData = {1, 1};
  std::vector<float> gradInHostData(8, 0);
  std::vector<float> gradWeightHostData(2, 0);
  std::vector<float> gradBiasHostData(2, 0);
  bool training = true;
  double eps = 1e-5;
  // Create the gradOut aclTensor.
  ret = CreateAclTensor(gradOutHostData, gradOutShape, &gradOutDeviceAddr, aclDataType::ACL_FLOAT, &gradOut);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create the self aclTensor.
  ret = CreateAclTensor(selfHostData, selfShape, &selfDeviceAddr, aclDataType::ACL_FLOAT, &self);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create the weight aclTensor.
  ret = CreateAclTensor(weightHostData, weightShape, &weightDeviceAddr, aclDataType::ACL_FLOAT, &weight);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create rMean aclTensor.
  ret = CreateAclTensor(rMeanHostData, rMeanShape, &rMeanDeviceAddr, aclDataType::ACL_FLOAT, &rMean);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create rVar aclTensor.
  ret = CreateAclTensor(rVarHostData, rVarShape, &rVarDeviceAddr, aclDataType::ACL_FLOAT, &rVar);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create sMean aclTensor.
  ret = CreateAclTensor(sMeanHostData, sMeanShape, &sMeanDeviceAddr, aclDataType::ACL_FLOAT, &sMean);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create sVar aclTensor.
  ret = CreateAclTensor(sVarHostData, sVarShape, &sVarDeviceAddr, aclDataType::ACL_FLOAT, &sVar);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create outMask aclBoolArray.
  bool maskData[3] = {true, true, true};
  outMask = aclCreateBoolArray(&(maskData[0]), 3);
  // Create gradIn aclTensor.
  ret = CreateAclTensor(gradInHostData, gradInShape, &gradInDeviceAddr, aclDataType::ACL_FLOAT, &gradIn);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create gradWeight aclTensor.
  ret = CreateAclTensor(gradWeightHostData, gradWeightShape, &gradWeightDeviceAddr, aclDataType::ACL_FLOAT, &gradWeight);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create gradBias aclTensor.
  ret = CreateAclTensor(gradBiasHostData, gradBiasShape, &gradBiasDeviceAddr, aclDataType::ACL_FLOAT, &gradBias);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  uint64_t workspaceSize = 0;
  aclOpExecutor* executor;

  // aclnnBatchNormBackward API call example
  // 3. Call the CANN operator library API. Replace with the specific API name.
  // Call the first-phase API of aclnnFastBatchNormBackward.
  ret = aclnnFastBatchNormBackwardGetWorkspaceSize(gradOut, self, weight, rMean, rVar, sMean, sVar, training, eps, outMask, 0, gradIn, gradWeight, gradBias, &workspaceSize, &executor);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnBatchNormBackwardGetWorkspaceSize failed. ERROR: %d\n", ret); return ret);
  // Apply for device memory based on the workspaceSize calculated by the first-phase API.
  void* workspaceAddr = nullptr;
  if (workspaceSize > 0) {
    ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
  }
  // Call the second-phase API of aclnnFastBatchNormBackward.
  ret = aclnnFastBatchNormBackward(workspaceAddr, workspaceSize, executor, stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnBatchNormBackward failed. ERROR: %d\n", ret); return ret);

  // 4. (Boilerplate) Synchronize and wait for task execution to complete.
  ret = aclrtSynchronizeStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

  // 5. Obtain the output values and copy the results from the device memory to the host memory. Modify based on the specific API definition.
  auto size = GetShapeSize(gradInShape);
  std::vector<float> resultData(size, 0);
  ret = aclrtMemcpy(resultData.data(), resultData.size() * sizeof(resultData[0]), gradInDeviceAddr,
                    size * sizeof(resultData[0]), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < size; i++) {
    LOG_PRINT("result[%ld] is: %f\n", i, resultData[i]);
  }

  // 6. Release aclTensor and aclScalar. Modify based on the specific API definition.
  aclDestroyTensor(gradOut);
  aclDestroyTensor(self);
  aclDestroyTensor(weight);
  aclDestroyTensor(rMean);
  aclDestroyTensor(rVar);
  aclDestroyTensor(sMean);
  aclDestroyTensor(sVar);
  aclDestroyBoolArray(outMask);
  aclDestroyTensor(gradIn);
  aclDestroyTensor(gradWeight);
  aclDestroyTensor(gradBias);

  // 7. Release device resources. Modify based on the specific API definition.
  aclrtFree(gradOutDeviceAddr);
  aclrtFree(selfDeviceAddr);
  aclrtFree(weightDeviceAddr);
  aclrtFree(rMeanDeviceAddr);
  aclrtFree(rVarDeviceAddr);
  aclrtFree(sMeanDeviceAddr);
  aclrtFree(sVarDeviceAddr);
  aclrtFree(gradInDeviceAddr);
  aclrtFree(gradWeightDeviceAddr);
  aclrtFree(gradBiasDeviceAddr);
  if (workspaceSize > 0) {
    aclrtFree(workspaceAddr);
  }
  aclrtDestroyStream(stream);
  aclrtResetDevice(deviceId);
  aclFinalize();
  return 0;
}