aclnnGroupNormSwish

Supported Products

Product	Supported
Atlas A3 training series products/Atlas A3 inference series products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	×
Atlas inference series products	×
Atlas training series products	×

Function

Description: Computes the group normalization result out, mean value meanOut, reciprocal rstdOut of the standard deviation, and Swish output of the input x.
Formula:
- GroupNorm: Assume $E[x] = \bar{x}$ indicates the mean value of $x$ , and $Var[x] = \frac{1}{n} * \sum_{i=1}^n(x_i - E[x])^2$ indicates the variance of $x$ . Then:
  $\left\{ \begin{array} {rcl} yOut& &= \frac{x - E[x]}{\sqrt{Var[x] + eps}} * \gamma + \beta \\ meanOut& &= E[x]\\ rstdOut& &= \frac{1}{\sqrt{Var[x] + eps}}\\ \end{array} \right.$
- Swish:
  $yOut = \frac{x}{1+e^{-scale * x}}$
  When activateSwish is set to True, Swish is computed. In this case, x in the Swish formula is out obtained by using the GroupNorm formula.

Prototype

Each operator has two-phase API calls. First, aclnnGroupNormSwishGetWorkspaceSize is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, aclnnGroupNormSwish is called to perform computation.

aclnnStatus aclnnGroupNormSwishGetWorkspaceSize(const aclTensor *x, const aclTensor *gamma, const aclTensor *beta, int64_t numGroups, char *dataFormatOptional, double eps, bool activateSwish, double swishScale, const aclTensor *yOut, const aclTensor *meanOut, const aclTensor *rstdOut, uint64_t *workspaceSize, aclOpExecutor **executor)
aclnnStatus aclnnGroupNormSwish(void *workspace, uint64_t workspaceSize, aclOpExecutor *executor, aclrtStream stream)

aclnnGroupNormSwishGetWorkspaceSize

Parameters:
- x (aclTensor*, computation input): target tensor to be group normalized, $x$ in the yOut computation formula, and aclTensor on the device. The shape must be greater than 1D. The 0th and 1st dimensions of x must be greater than 0, and the 1st dimension must be exactly divided by group. The data type can be FLOAT32, FLOAT16, or BFLOAT16. The data format can be ND. Non-contiguous tensors are supported.
- gamma (aclTensor*, computation input): gamma parameter in group normalization, $\gamma$ in the yOut computation formula, and aclTensor on the device. The shape is 1D. The number of elements must be the same as that of the 1st dimension of the input $x$ . The data type of gamma must be the same as that of beta, which must be FLOAT or the same as that of x. The data type can be FLOAT32, FLOAT16, or BFLOAT16. The data format can be ND. Non-contiguous tensors are supported.
- beta (aclTensor*, computation input): beta parameter in group normalization, $\beta$ in the yOut computation formula, and aclTensor on the device. The shape is 1D. The number of elements must be the same as that of the 1st dimension of the input $x$ . The data type of gamma must be the same as that of beta, which must be FLOAT or the same as that of x. The data type can be FLOAT32, FLOAT16, or BFLOAT16. The data format can be ND. Non-contiguous tensors are supported.
- numGroups (int64_t, computation input): integer on the host, indicating that the first dimension of the input $x$ is divided into groups. The value is greater than 0.
- dataFormatOptional (char*, computation input): character type on the host, indicating the data format. Only NCHW is supported in the current version.
- eps (double, computation input): $eps$ in the yOut and rstdOut computation formulas, used to prevent the offset of dividing by 0. The value is greater than 0. It is of the DOUBLE type on the host.
- activateSwish (bool, computation input): indicates whether to support Swish computation. If this parameter is set to true, Swish computation is performed after groupnorm computation. It is of the BOOL type on the host.
- swishScale (double, computation input): $scale$ of Swish computation. It is of the DOUBLE type on the host.
- yOut (aclTensor*, computation output): group normalization result and aclTensor on the device. The data type and shape are the same as those of $x$ . The data type can be FLOAT32, FLOAT16, or BFLOAT16. The data format can be ND. Non-contiguous tensors are supported.
- meanOut (aclTensor*, computation output): mean value after x is grouped and aclTensor on the device. The data type is the same as that of $gamma$ . The shape is (N, numGroups), where N indicates the size of the 0th dimension of $x$ , and numGroups is the computation input, indicating that the first dimension of the input $x$ is divided into groups. The data type can be FLOAT32, FLOAT16, or BFLOAT16. The data format can be ND. Non-contiguous tensors are supported.
- rstdOut (aclTensor*, computation output): reciprocal of the standard deviation after x is grouped and aclTensor is on the device. The data type is the same as that of $gamma$ . The shape is (N, numGroups), where N indicates the size of the 0th dimension of $x$ , and numGroups indicates the computation input, indicating that the first dimension of the input $x$ is divided into groups. The data type can be FLOAT32, FLOAT16, or BFLOAT16. The data format can be ND. Non-contiguous tensors are supported.
- workspaceSize (uint64_t*, output): size of the workspace to be allocated on the device.
- executor (aclOpExecutor **, output): operator executor, containing the operator computation process.
Returns:

aclnnStatus: status code. For details, see aclnn Return Codes.

The first-phase API implements input parameter verification. The following errors may be thrown.
161001 (ACLNN_ERR_PARAM_NULLPTR): 1. The passed x, gamma, beta, yOut, meanOut, or rstdOut is a null pointer.
161002 (ACLNN_ERR_PARAM_INVALID): 1. The data type of x, gamma, beta, yOut, meanOut, or rstdOut is not supported.

aclnnGroupNormSwish

Parameters:
- workspace (void*, input): address of the workspace to be allocated on the device.
- workspaceSize (uint64_t, input): size of the workspace to be allocated on the device, which is obtained by calling the first-phase API aclnnGroupNormSwishGetWorkspaceSize.
- executor (aclOpExecutor*, input): operator executor, containing the operator computation process.
- stream (aclrtStream, input): stream for executing the task.
Returns:

aclnnStatus: status code. For details, see aclnn Return Codes.

Constraints

Deterministic computation
- aclnnGroupNormSwish defaults to a deterministic implementation.

Example

The following example is for reference only. For details, see Compilation and Running Sample.

#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_group_norm_swish.h"

#define CHECK_RET(cond, return_expr) \
  do {                               \
    if (!(cond)) {                   \
      return_expr;                   \
    }                                \
  } while (0)

#define LOG_PRINT(message, ...)     \
  do {                              \
    printf(message, ##__VA_ARGS__); \
  } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape) {
  int64_t shape_size = 1;
  for (auto i : shape) {
    shape_size *= i;
  }
  return shape_size;
}

int Init(int32_t deviceId, aclrtStream* stream) {
  // (Fixed writing) Initialize resources.
  auto ret = aclInit(nullptr);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
  ret = aclrtSetDevice(deviceId);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
  ret = aclrtCreateStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
  return 0;
}

template <typename T>
int CreateAclTensor(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr,
                    aclDataType dataType, aclTensor** tensor) {
  auto size = GetShapeSize(shape) * sizeof(T);
  // Call aclrtMalloc to allocate memory on the device.
  auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);

  // Call aclrtMemcpy to copy the data on the host to the memory on the device.
  ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

  // Compute the strides of the contiguous tensor.
  std::vector<int64_t> strides(shape.size(), 1);
  for (int64_t i = shape.size() - 2; i >= 0; i--) {
    strides[i] = shape[i + 1] * strides[i + 1];
  }

  // Call aclCreateTensor to create an aclTensor.
  *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND,
                            shape.data(), shape.size(), *deviceAddr);
  return 0;
}

int main() {
  // 1. (Fixed writing) Initialize the device and stream. For details, see the list of external ACL APIs.
  // Set the device ID in use.
  int32_t deviceId = 0;
  aclrtStream stream;
  auto ret = Init(deviceId, &stream);
  // Handle the check as required.
  CHECK_RET(ret == 0, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);
  // 2. Construct inputs and outputs based on the API definition.
  std::vector<int64_t> xShape = {2, 3, 4};
  std::vector<int64_t> gammaShape = {3};
  std::vector<int64_t> betaShape = {3};
  std::vector<int64_t> outShape = {2, 3, 4};
  std::vector<int64_t> meanOutShape = {2, 1};
  std::vector<int64_t> rstdOutShape = {2, 1};
  void* xDeviceAddr = nullptr;
  void* gammaDeviceAddr = nullptr;
  void* betaDeviceAddr = nullptr;
  void* outDeviceAddr = nullptr;
  void* meanOutDeviceAddr = nullptr;
  void* rstdOutDeviceAddr = nullptr;
  aclTensor* x = nullptr;
  aclTensor* gamma = nullptr;
  aclTensor* beta = nullptr;
  aclTensor* yOut = nullptr;
  aclTensor* meanOut = nullptr;
  aclTensor* rstdOut = nullptr;
  std::vector<float> xHostData = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0,
                                     13.0, 14.0, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0};
  std::vector<float> gammaHostData = {2.0, 2, 2};
  std::vector<float> betaHostData = {2.0, 2, 2};
  std::vector<float> outHostData = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0,
                                    13.0, 14.0, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0};
  std::vector<float> meanOutHostData = {2.0, 2};
  std::vector<float> rstdOutHostData = {2.0, 2};

  int64_t numGroups = 1;
  double eps = 0.00001;
  bool activateSwish = true;
  double scale = 1.0;
  char* dataFormatOptional = "NCHW";
  // Create an x aclTensor.
  ret = CreateAclTensor(xHostData, xShape, &xDeviceAddr, aclDataType::ACL_FLOAT, &x);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create a gamma aclTensor.
  ret = CreateAclTensor(gammaHostData, gammaShape, &gammaDeviceAddr, aclDataType::ACL_FLOAT, &gamma);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create a beta aclTensor.
  ret = CreateAclTensor(betaHostData, betaShape, &betaDeviceAddr, aclDataType::ACL_FLOAT, &beta);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create an out aclTensor.
  ret = CreateAclTensor(outHostData, outShape, &outDeviceAddr, aclDataType::ACL_FLOAT, &yOut);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create a meanOut aclTensor.
  ret = CreateAclTensor(meanOutHostData, meanOutShape, &meanOutDeviceAddr, aclDataType::ACL_FLOAT, &meanOut);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create a rstdOut aclTensor.
  ret = CreateAclTensor(rstdOutHostData, rstdOutShape, &rstdOutDeviceAddr, aclDataType::ACL_FLOAT, &rstdOut);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  // 3. Call the CANN operator library API, which needs to be replaced with the actual API.
  uint64_t workspaceSize = 0;
  aclOpExecutor* executor;
  // Call the first-phase API of aclnnGroupNormSwish.
  ret = aclnnGroupNormSwishGetWorkspaceSize(x, gamma, beta, numGroups, dataFormatOptional, eps, activateSwish, scale, yOut, meanOut, rstdOut, &workspaceSize, &executor);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnGroupNormSwishGetWorkspaceSize failed. ERROR: %d\n", ret); return ret);
  // Allocate device memory based on workspaceSize computed by the first-phase API.
  void* workspaceAddr = nullptr;
  if (workspaceSize > 0) {
    ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret;);
  }
  // Call the second-phase API of aclnnGroupNormSwish.
  ret = aclnnGroupNormSwish(workspaceAddr, workspaceSize, executor, stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnGroupNormSwish failed. ERROR: %d\n", ret); return ret);
  // 4. (Fixed writing) Wait until the task execution is complete.
  ret = aclrtSynchronizeStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);
  // 5. Obtain the output value and copy the result from the device memory to the host. Modify the code based on the API definition.
  auto size = GetShapeSize(outShape);
  std::vector<float> outResultData(size, 0);
  ret = aclrtMemcpy(outResultData.data(), outResultData.size() * sizeof(outResultData[0]), outDeviceAddr, size * sizeof(float),
                    ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < size; i++) {
    LOG_PRINT("outResultData[%ld] is: %f\n", i, outResultData[i]);
  }

  size = GetShapeSize(meanOutShape);
  std::vector<float> meanResultData(size, 0);
  ret = aclrtMemcpy(meanResultData.data(), meanResultData.size() * sizeof(meanResultData[0]), meanOutDeviceAddr, size * sizeof(float),
                    ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < size; i++) {
    LOG_PRINT("meanResultData[%ld] is: %f\n", i, meanResultData[i]);
  }

  size = GetShapeSize(rstdOutShape);
  std::vector<float> rstdResultData(size, 0);
  ret = aclrtMemcpy(rstdResultData.data(), rstdResultData.size() * sizeof(rstdResultData[0]), rstdOutDeviceAddr, size * sizeof(float),
                    ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < size; i++) {
    LOG_PRINT("rstdResultData[%ld] is: %f\n", i, rstdResultData[i]);
  }

  // 6. Release aclTensor and aclScalar. Modify the code based on the API definition.
  aclDestroyTensor(x);
  aclDestroyTensor(gamma);
  aclDestroyTensor(beta);
  aclDestroyTensor(yOut);
  aclDestroyTensor(meanOut);
  aclDestroyTensor(rstdOut);

  // 7. Release device resources. Modify the code based on the API definition.
  aclrtFree(xDeviceAddr);
  aclrtFree(gammaDeviceAddr);
  aclrtFree(betaDeviceAddr);
  aclrtFree(outDeviceAddr);
  aclrtFree(meanOutDeviceAddr);
  aclrtFree(rstdOutDeviceAddr);

  if (workspaceSize > 0) {
    aclrtFree(workspaceAddr);
  }
  aclrtDestroyStream(stream);
  aclrtResetDevice(deviceId);
  aclFinalize();
  return 0;
}