aclnnDequantSwigluQuantV2

Applicable Products

Product	Supported
Ascend 950PR/Ascend 950DT	√
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	×
Atlas inference products	×
Atlas training products	×

Function

Description: Adds dequant and quant operations before and after the Swish gated linear unit activation function to perform DequantSwigluQuant computation on x. Compared with aclnnDequantSwigluQuant, this API has two categories of new parameters: 1) Three new input parameters used on Ascend 950 chips: dstType, roundModeOptional, and activateDim. 2) On Atlas A2 and Atlas A3 chips, four new parameters are added for the SwiGLU variant used by GPT-OSS: swigluMode, clampLimit, gluAlpha, and gluBias; when using this API on Ascend 950 chips, these four parameters must be set to their default values. Select the appropriate API based on the actual situation.
Formula when swigluMode is 0:
$dequantOut_i = Dequant(x_i)$ $swigluOut_i = Swiglu(dequantOut_i)=Swish(A_i)*B_i$ $out_i = Quant(swigluOut_i)$
Where A_i represents the first half of dequantOut_i, and B_i represents the second half of dequantOut_i.
Formula when swigluMode is 1:
$dequantOut_i = Dequant(x_i)$ $x\_glu = x\_glu.clamp(min=None, max=clampLimit)$ $x\_linear = x\_linear.clamp(min=-clampLimit, max=clampLimit)$ $out\_glu = x\_glu * sigmoid(gluAlpha * x\_glu)$ $swigluOut_i = out\_glu * (x\_linear + gluBias)$ $out_i = Quant(swigluOut_i)$
where x_glu represents the even-indexed part of dequantOut_i, and x_linear represents the odd-indexed part of dequantOut_i.

Function Prototype

Each operator is divided into a two-phase API. You must first call the "aclnnDequantSwigluQuantV2GetWorkspaceSize" API to obtain the workspace size required for computation and the executor that contains the operator computation process, and then call the "aclnnDequantSwigluQuantV2" API to perform the computation.

aclnnStatus aclnnDequantSwigluQuantV2GetWorkspaceSize(
  const aclTensor *x,
  const aclTensor *weightScaleOptional,
  const aclTensor *activationScaleOptional,
  const aclTensor *biasOptional,
  const aclTensor *quantScaleOptional,
  const aclTensor *quantOffsetOptional,
  const aclTensor *groupIndexOptional,
  bool             activateLeft,
  char            *quantModeOptional,
  int64_t          dstType,
  char            *roundModeOptional,
  int64_t          activateDim,
  int64_t          swigluMode,
  double           clampLimit,
  double           gluAlpha,
  double           gluBias,
  const aclTensor *yOut,
  const aclTensor *scaleOut,
  uint64_t        *workspaceSize,
  aclOpExecutor  **executor)

aclnnStatus aclnnDequantSwigluQuantV2(
  void          *workspace,
  uint64_t       workspaceSize,
  aclOpExecutor *executor,
  aclrtStream    stream)

aclnnDequantSwigluQuantV2GetWorkspaceSize

Parameters:

Parameter	Input/Output	Description	Instruction	Data Type	Data Format	Dimension (shape)	Non-contiguous Tensor
x (aclTensor*)	Input	Input data to be processed, i.e., x in the formula.	The shape is [X1, X2, ..., Xn, 2H], with no more than 8 dimensions and no fewer than 2 dimensions. The dimension of input x corresponding to activateDim must be a multiple of 2. Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products: Only 2-dimensional input is supported, with data types of INT32 and BFLOAT16.	FLOAT16, BFLOAT16, INT32	ND	2-8	x
weightScaleOptional (aclTensor*)	Input	Dequantization scale of weight.	Shape supports 1D or 2D, expressed as [2H] or [groupNum, 2H], and the value 2H must be consistent with the last dimension of x. Optional parameter. A null pointer can be passed. When groupIndexOptional is a null pointer, the shape is [2H]; when groupIndexOptional is not a null pointer, the shape is [groupNum, 2H].	FLOAT	ND	1 or 2	x
activationScaleOptional (aclTensor*)	Input	Dequantization scale of the activation function.	Dequantization scale of the activation function. The shape is [X1, X2, ... Xn], with no more than 7 dimensions and no fewer than 1 dimension, one dimension fewer than the dimension of x, and the shape is consistent with that of x in the corresponding dimensions. Optional parameter. A null pointer can be passed.	FLOAT	ND	1-7	x
biasOptional (aclTensor*)	Input	Bias of Matmul, i.e., biasOptional in the formula.	The shape supports 1D or 2D, expressed as [2H] or [groupNum, 2H], where the value 2H must be consistent with the last dimension of x. When groupIndexOptional is a null pointer, the shape is [2H]; when groupIndexOptional is not a null pointer, the shape is [groupNum, 2H]. Optional parameter. A null pointer can be passed.	FLOAT, FLOAT16, BFLOAT16, INT32	ND	1 or 2	x
quantScaleOptional (aclTensor*)	Input	Quantization scale, i.e., quantScaleOptional in the formula.	When quantModeOptional is static, the shape is 1D with a value of 1, expressed as shape[1]. When quantModeOptional is dynamic, the shape is 1D or 2D, expressed as [H], [2H], or [groupNum, H]. When groupIndexOptional is a null pointer and activateDim is the last axis, the shape is [H]. When groupIndexOptional is not a null pointer and activateDim is the last axis, the shape is [groupNum, H]. Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products: When quantModeOptional is static, the shape is expressed as [groupNum] or [groupNum, H]; when quantModeOptional is dynamic, the shape is expressed as [groupNum] or [groupNum, H].	FLOAT, FLOAT16	ND	-	x
quantOffsetOptional (aclTensor*)	Input	Quantization offset.	When quant_mode is dynamic, quantOffset input is not required. In static quantization, quantOffset must be input, and its data type and shape are the same as quantScale.	FLOAT	ND	-	x
groupIndexOptional (aclTensor*)	Input	group_index required for MoE grouping.	shape supports 1D or 2D tensors, with shape [groupNum] or [groupNum, 2], where groupNum is greater than or equal to 1. Optional parameter. A null pointer can be passed. Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products: Only 1D [groupNum] input is supported. Null pointer is not supported.	INT64	ND	1 or 2	x
activateLeft (bool)	Input	Indicates whether to apply swiglu activation to the left half of the input.	When the value is false, activation is applied to the right half of the input. If swigluMode is 1, activateLeft must be true.	-	-	-	-
quantModeOptional (char*)	Input	Indicates whether dynamic quantization or static quantization is used.	Supports "dynamic" and "static". Supports passing a null pointer. When a null pointer is passed, "static" is used by default.	-	-	-	-
dstType (int64_t)	Input	Indicates the data type of the output y.	The value range of dstType is [2, 34, 35, 36, 40, 41], corresponding to INT8, HIFLOAT8, FLOAT8_E5M2, FLOAT8_E4M3FN, FLOAT4_E2M1, and FLOAT4_E1M2, respectively. Atlas A2 training products/Atlas A2 inference products and Atlas A3 training products/Atlas A3 inference products: Only input 2 (INT8) is supported.	-	-	-	-
roundModeOptional (char*)	Input	Indicates the rounding mode for the output y result.	Value range: ["rint", "round", "floor", "ceil", "trunc"]. When the data type of output y is INT8, FLOAT8_E5M2, or FLOAT8_E4M3FN, only the "rint" mode is supported. When the data type of output y is HIFLOAT8, only the "round" mode is supported. A null pointer can be passed. If a null pointer is passed, "rint" is used by default. Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products: Only a null pointer or "rint" is supported as input.	-	-	-	-
activateDim (int64_t)	Input	Specifies the split axis for the swish computation.	The value range of activateDim is [-xDim, xDim - 1] (where xDim refers to the dimension of input x). When activateDim does not correspond to the last axis of x, groupIndexOptional is not allowed. When activateDim does not correspond to the last axis of x, quantModeOptional supports only static. Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products: Only -1 is supported.	-	-	-	-
swigluMode (int64_t)	Input	Indicates the computation mode of swiglu.	Value range: [0, 1]. 0: Indicates the traditional swiglu computation method. 1: Indicates a variant of swiglu that uses odd-even partitioning and supports clamp_limit, activation coefficient, and bias. 0 indicates not used, 1 indicates used, and the default value is 0.	-	-	-	-
clampLimit (double)	Input	Indicates the threshold value used by the variant swiglu.	This parameter is an optional parameter. Used to clip the input. Its value must be greater than 0 and less than infinity to prevent excessively large values from affecting the stability of swiglu computation. The default value is 7.0.	-	-	-	-
gluAlpha (double)	Input	Represents the parameter used by the variant swiglu.	This parameter is an optional parameter. Used to adjust the scaling of the linear part in the glu activation function. The default value is 1.702.	-	-	-	-
gluBias (double)	Input	Represents the bias parameter used by the swiglu variant.	This parameter is an optional parameter. Adds an offset in the linear computation of swiglu. The default value is 1.0.	-	-	-	-
yOut (aclTensor*)	Output	-	When activateDim corresponds to the last axis of x, the shape is [X1,X2,...Xn,H]. When activateDim does not correspond to the last axis of x, the shape is [X1,X2,...,XactivateDim / 2,...,2H]. When the data type of yOut is FLOAT4_E2M1 or FLOAT4_E1M2, the last dimension of yOut must be a multiple of 2. When activateDim does not correspond to the last axis of x, the last axis of yOut must be less than 5120.	INT8, HIFLOAT8, FLOAT8_E5M2, FLOAT8_E4M3FN, FLOAT4_E2M1, FLOAT4_E1M2	ND	2-8	x
scaleOut (aclTensor*)	Output	-	When activateDim corresponds to the last axis of x, the shape is [X1, X2, ..., Xn]. When activateDim does not correspond to the last axis of x, the shape is [X1, X2, ..., XactivateDim / 2, ..., Xn]. When quantModeOptional is static, scaleOut is not computed.	FLOAT	ND	1-7	x
workspaceSize (uint64_t*)	Output	Size of the workspace to be applied for on the Device side.	-	-	-	-	-
executor (aclOpExecutor**)	Output	Returns the op executor, which contains the operator computation flow.	-	-	-	-	-

Return value

aclnnStatus: return code. For details, see aclnn Return Code.

The first-phase API performs input parameter validation and returns an error in the following scenarios:

Return Code	Error Code	Description
ACLNN_ERR_PARAM_NULLPTR	161001	The input x, yOut, or scaleOut is a null pointer. When the data type of x is int32, weightScaleOptional is a null pointer. When quantModeOptional is static, quantScaleOptional is a null pointer.
ACLNN_ERR_PARAM_INVALID	161002	The data type of the input or output is not within the supported range.
		The dimension of the input or output parameter is not within the supported range.
		quantModeOptional is not within the specified value range.
		dstType is not within the specified value range.
		roundModeOptional is not within the specified value range.
		activateDim is not within the specified value range.
		The shapes of weightScaleOptional, activationScaleOptional, biasOptional, quantScaleOptional, and groupIndexOptional do not satisfy the constraints with x.

aclnnDequantSwigluQuantV2

Parameters:

Parameter	Input/Output	Description
workspace	Input	Workspace memory address applied for on the Device side.
workspaceSize	Input	Workspace size applied for on the Device side, obtained from the first-phase API aclnnDequantSwigluQuantV2GetWorkspaceSize.
executor	Input	Operator executor that contains the operator computation flow.
stream	Input	Specifies the stream for task execution.

Return value

aclnnStatus: return code. For details, see aclnn Return Code.

Constraints

Deterministic computation:
- aclnnDequantSwigluQuantV2 defaults to a deterministic implementation.
The dimension of input x corresponding to activateDim must be a multiple of 2, and the number of dimensions of x must be greater than 1.
When the data type of input x is INT32, weightScaleOptional cannot be null. When the data type of input x is not INT32, weightScaleOptional is not allowed and a null pointer must be passed.
When the data type of input x is not INT32, activationScaleOptional is not allowed and a null pointer must be passed.
When the data type of input x is not INT32, biasOptional is not allowed and a null pointer must be passed.
When the data type of output yOut is FLOAT4_E2M1 or FLOAT4_E1M2, the last dimension of yOut must be a multiple of 2.
When the dimension corresponding to activateDim is not the last axis of x, the last axis of output yOut must not exceed 5120.
The sum of all elements in groupIndexOptional must not exceed the product of all axes of input x except the last axis.
The portion of the output yOut and scaleOut that exceeds the sum of all elements in groupIndexOptional is not cleared, and the data in this memory region is garbage.
Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products: When groupIndexOptional is input, the operator imposes an upper limit on the memory size of the input tensor. The last axis of x must not exceed 7232.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_dequant_swiglu_quant_v2.h"

#define CHECK_RET(cond, return_expr) \
  do {                               \
    if (!(cond)) {                   \
      return_expr;                   \
    }                                \
  } while (0)

#define LOG_PRINT(message, ...)     \
  do {                              \
    printf(message, ##__VA_ARGS__); \
  } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape) {
  int64_t shapeSize = 1;
  for (auto i : shape) {
    shapeSize *= i;
  }
  return shapeSize;
}

int Init(int32_t deviceId, aclrtStream* stream) {
  // Boilerplate: initialize resources.
  auto ret = aclInit(nullptr);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
  ret = aclrtSetDevice(deviceId);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
  ret = aclrtCreateStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
  return 0;
}

template <typename T>
int CreateAclTensor(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr,
                    aclDataType dataType, aclTensor** tensor) {
  auto size = GetShapeSize(shape) * sizeof(T);
  // Call aclrtMalloc to allocate device-side memory.
  auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);
  // Call aclrtMemcpy to copy host-side data to the device-side memory.
  ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

  // Compute the strides of the contiguous tensor.
  std::vector<int64_t> strides(shape.size(), 1);
  for (int64_t i = shape.size() - 2; i >= 0; i--) {
    strides[i] = shape[i + 1] * strides[i + 1];
  }

  // Call the aclCreateTensor API to create an aclTensor.
  *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND,
                            shape.data(), shape.size(), *deviceAddr);
  return 0;
}

int main() {
  // 1. (Boilerplate) Initialize the device/stream. See the ACL API manual.
  // Set deviceId based on the actual device.
  int32_t deviceId = 0;
  aclrtStream stream;
  auto ret = Init(deviceId, &stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);

  // 2. Construct the inputs and outputs. Custom construction is required based on the API.
  std::vector<int64_t> xShape = {2, 64};
  std::vector<int64_t> weightScaleShape = {1, 64};
  std::vector<int64_t> activationScaleShape = {2};
  std::vector<int64_t> biasShape = {1, 64};
  std::vector<int64_t> scaleShape = {1, 32}; // quantscale
  std::vector<int64_t> offsetShape = {1};
  std::vector<int64_t> groupIndexShape = {1};
  std::vector<int64_t> outShape = {2, 32};
  std::vector<int64_t> scaleOutShape = {2};

  void* xDeviceAddr = nullptr;
  void* weightScaleDeviceAddr = nullptr;
  void* activationScaleDeviceAddr = nullptr;
  void* biasDeviceAddr = nullptr;

  void* scaleDeviceAddr = nullptr;
  void* offsetDeviceAddr = nullptr;
  void* groupIndexDeviceAddr = nullptr;
  void* outDeviceAddr = nullptr;
  void* scaleOutDeviceAddr = nullptr;

  aclTensor* x = nullptr;
  aclTensor* weightScale = nullptr;
  aclTensor* activationScale= nullptr;
  aclTensor* bias = nullptr;

  aclTensor* scale = nullptr;
  aclTensor* offset = nullptr;
  aclTensor* groupIndex = nullptr;
  aclTensor* out = nullptr;
  aclTensor* scaleOut = nullptr;

  std::vector<int32_t> xHostData = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22,
                                    23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42,
                                    43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,
                                    0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22,
                                    23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42,
                                    43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63};
  std::vector<float> weightScaleData = {1.0};
  std::vector<float> activationScaleData = {1.0};
  std::vector<float> biasData = {1.0};
  std::vector<int64_t> groupIndexData = {1};
  std::vector<float> scaleHostData = {1};
  std::vector<float> offsetHostData = {1};
  std::vector<int8_t> outHostData = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
  std::vector<float> scaleOutHostData = {0, 0};

  bool activateLeft = true;
  int64_t dstType = 2;
  int64_t activateDim = -1;
  int64_t swigluMode = 1;
  float clampLimit = 7.0;
  float gluAlpha = 1.0;
  float gluBias = 1.702;

  // Create the x aclTensor.
  ret = CreateAclTensor(xHostData, xShape, &xDeviceAddr, aclDataType::ACL_INT32, &x);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create weightScale.
  ret = CreateAclTensor(weightScaleData, weightScaleShape, &weightScaleDeviceAddr, aclDataType::ACL_FLOAT, &weightScale);
  // Create activationScale.
  ret = CreateAclTensor(activationScaleData, activationScaleShape, &activationScaleDeviceAddr, aclDataType::ACL_FLOAT, &activationScale);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create bias.
  ret = CreateAclTensor(biasData, biasShape, &biasDeviceAddr, aclDataType::ACL_FLOAT, &bias);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create the scale aclTensor.
  ret = CreateAclTensor(scaleHostData, scaleShape, &scaleDeviceAddr, aclDataType::ACL_FLOAT, &scale);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create offset aclTensor
  ret = CreateAclTensor(offsetHostData, offsetShape, &offsetDeviceAddr, aclDataType::ACL_FLOAT, &offset);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create groupIndex aclTensor
  ret = CreateAclTensor(groupIndexData, groupIndexShape, &groupIndexDeviceAddr, aclDataType::ACL_INT64, &groupIndex);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create out aclTensor
  ret = CreateAclTensor(outHostData, outShape, &outDeviceAddr, aclDataType::ACL_INT8, &out);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create scaleOut aclTensor
  ret = CreateAclTensor(scaleOutHostData, scaleOutShape, &scaleOutDeviceAddr, aclDataType::ACL_FLOAT, &scaleOut);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // 3. Call the CANN operator library API. Replace with the actual API name.
  uint64_t workspaceSize = 0;
  aclOpExecutor* executor;
  // Call the first-phase API of aclnnDequantSwigluQuantV2.
  ret = aclnnDequantSwigluQuantV2GetWorkspaceSize(x, weightScale, activationScale, bias, scale, nullptr, groupIndex, activateLeft, "dynamic", dstType, "rint", activateDim, swigluMode, clampLimit, gluAlpha, gluBias, out, scaleOut, &workspaceSize, &executor);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnDequantSwigluQuantV2GetWorkspaceSize failed. ERROR: %d\n", ret); return ret);
  // Allocate device memory based on the workspaceSize calculated by the first-phase API.
  void* workspaceAddr = nullptr;
  if (workspaceSize > 0) {
    ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
  }
  // Call the second-phase API of aclnnDequantSwigluQuantV2.
  ret = aclnnDequantSwigluQuantV2(workspaceAddr, workspaceSize, executor, stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnDequantSwigluQuantV2 failed. ERROR: %d\n", ret); return ret);

  // 4. (Boilerplate) Synchronously wait for task execution to complete.
  ret = aclrtSynchronizeStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

  // 5. Obtain the output value and copy the result from the device side memory to the host side. Modify this based on the specific API definition.
  auto size = GetShapeSize(outShape);
  std::vector<int8_t> resultData(size, 0);
  ret = aclrtMemcpy(resultData.data(), resultData.size() * sizeof(resultData[0]), outDeviceAddr,size * sizeof(resultData[0]), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < size; i++) {
    LOG_PRINT("result[%ld] is: %d\n", i, resultData[i]);
  }
  // 6. Release aclTensor and aclScalar. Modify based on the specific API definition.
  aclDestroyTensor(x);
  aclDestroyTensor(scale);
  aclDestroyTensor(offset);
  aclDestroyTensor(out);
  aclDestroyTensor(scaleOut);
  // 7. Release device resources. Modify based on the specific API definition.
  aclrtFree(xDeviceAddr);
  aclrtFree(scaleDeviceAddr);
  aclrtFree(offsetDeviceAddr);
  aclrtFree(outDeviceAddr);
  aclrtFree(scaleOutDeviceAddr);
  if (workspaceSize > 0) {
    aclrtFree(workspaceAddr);
  }
  aclrtDestroyStream(stream);
  aclrtResetDevice(deviceId);
  aclFinalize();
  return 0;
}