aclnnAddRmsNormQuant

Applicable Products

Product	Supported
Ascend 950PR/Ascend 950DT	√
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	×
Atlas inference products	√
Atlas training products	×

Function

Description: The RMSNorm operator is a common normalization operation in large models. Compared with the LayerNorm operator, it removes the part of subtracting the mean. The AddRmsNormQuant operator fuses the Add operator before RMSNorm and the Quantize operator after RMSNorm, reducing data movement operations.
Formula:
$x_i={x1}_{i}+{x2}_{i}$ $y_i=\frac{1}{\operatorname{Rms}(\mathbf{x})} * x_i * g_i, \quad \text { where } \operatorname{Rms}(\mathbf{x})=\sqrt{\frac{1}{n} \sum_{i=1}^n x_i^2+eps}$
- When divMode is True:
  $y1Out=round((y/scales1)+zero\_points1)$ $y2Out=round((y/scales2)+zero\_points2)$
- When divMode is False:
  $y1Out=round((y*scales1)+zero\_points1)$ $y2Out=round((y*scales2)+zero\_points2)$

Function Prototype

Each operator uses a two-phase API. You must first call the "aclnnAddRmsNormQuantGetWorkspaceSize" API to obtain the input parameters and calculate the required workspace size based on the computation process, and then call the "aclnnAddRmsNormQuant" API to perform the computation.

aclnnStatus aclnnAddRmsNormQuantGetWorkspaceSize(
  const aclTensor *x1,
  const aclTensor *x2,
  const aclTensor *gamma,
  const aclTensor *scales1,
  const aclTensor *scales2Optional,
  const aclTensor *zeroPoints1Optional,
  const aclTensor *zeroPoints2Optional,
  int64_t          axis,
  double           epsilon,
  bool             divMode,
  aclTensor       *y1Out,
  aclTensor       *y2Out,
  aclTensor       *xOut,
  uint64_t        *workspaceSize,
  aclOpExecutor  **executor)

aclnnStatus aclnnAddRmsNormQuant(
  void          *workspace,
  uint64_t       workspaceSize,
  aclOpExecutor *executor,
  aclrtStream    stream)

aclnnAddRmsNormQuantGetWorkspaceSize

Parameters

Parameter	Input/Output	Description	Instruction	Data Type	Data Format	Dimension (Shape)	Non-contiguous Tensor
x1 (aclTensor*)	Input	Represents the source data tensor during normalization. Corresponds to `x1` in the formula.	Supports null tensor.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
x2 (aclTensor*)	Input	Represents the source data tensor during normalization. Corresponds to `x2` in the formula.	Supports null tensors. The shape and data type remain consistent with `x1`.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
gamma (aclTensor*)	Input	Weight tensor during normalization. Corresponds to `g` in the corresponding formula.	Nullable tensor supported. The shape must remain consistent with the dimensions of `x1` that require norm (layer normalization), and the data type must remain consistent with `x1`.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
scales1 (aclTensor*)	Input	Represents the scales tensor for y1Out obtained during quantization, corresponding to `scales1` in the formula.	Nullable tensor supported. The shape must remain consistent with `gamma`, or the last dimension must remain consistent with `gamma` while other dimensions are 1.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
scales2Optional (aclTensor*)	Input	Scale tensor for y2Out obtained during quantization. Corresponds to `scales2` in the formula.	Supports null tensor. Optional parameter that supports passing a null pointer. The shape and data type remain consistent with `scales1`. When the value of the `divMode` parameter is True, the value of this parameter cannot be 0.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
zeroPoints1Optional (aclTensor*)	Input	Represents the offset tensor used during quantization to obtain y1Out. Corresponds to `zero_points1` in the formula.	Supports null tensor. Optional parameter, supports passing a null pointer. The shape remains consistent with `scales1`.	INT32, FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
zeroPoints2Optional (aclTensor*)	Input	Represents the offset tensor for y2Out obtained during quantization. Corresponds to `zero_points2` in the formula.	Supports null tensor. Optional parameter that supports passing a null pointer. The shape remains consistent with `scales1`, and the data type remains consistent with `zeroPoints1Optional`.	INT32, FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
axis (int64_t)	Input	Specifies the element-wise axis for quantization. Other axes perform broadcast. The specified axis cannot exceed the number of dimensions of the input `x1`. Currently only supports -1. Passing other values has no effect.	-	-	-	-	-
epsilon (double)	Input	The input eps in the formula is used to prevent division-by-zero errors, with a data type of double. It is recommended to pass a small positive number.	-	-	-	-	-
divMode (bool)	Input	Parameter in the formula that determines whether division is used in the quantization formula. The data type is bool.	Supports True and False.	-	-	-	-
y1Out (aclTensor*)	Output	Represents the quantized output tensor, corresponding to `y1Out` in the formula.	Supports nullable tensor. The shape is consistent with the input `x1`/`x2`.	INT8, HIFLOAT8, FLOAT8_E5M2, FLOAT8_E4M3FN	ND	1-8	√
y2Out (aclTensor*)	Output	Represents the quantized output tensor, corresponding to `y2Out` in the formula.	Supports null tensor. The shape and data type remain consistent with `y1Out`.	INT8, HIFLOAT8, FLOAT8_E5M2, FLOAT8_E4M3FN	ND	1-8	√
xOut (aclTensor*)	Output	Represents the sum of x1 and x2, corresponding to `x` in the corresponding formula.	Supports null tensor. The shape remains consistent with `scales1`, and the data type remains consistent with `zeroPoints1Optional`.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
workspaceSize (uint64_t*)	Output	Returns the workspace size to be applied on the Device side.	-	-	-	-	-
executor (aclOpExecutor**)	Output	Returns the op executor, which contains the operator computation flow.	-	-	-	-	-

Atlas inference products:
- Data type:
  - The input parameters x1, x2, and gamma, and the output xOut only support FLOAT16.
  - The input parameters scales1 and scales2Optional only support FLOAT32.
  - The optional parameters zeroPoints1Optional and zeroPoints2Optional only support INT32.
  - The outputs y1Out and y2Out only support INT8.
- The input parameter divMode only supports True.
Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products:
- Data types:
  - The input parameters x1, x2, gamma and the output parameter xOut only support FLOAT16 and BFLOAT16.
  - The input parameters scales1 and scales2Optional only support FLOAT32 and BFLOAT16.
  - The optional parameters zeroPoints1Optional and zeroPoints2Optional only support INT32 and BFLOAT16.
  - The output parameters y1Out and y2Out only support INT8.
- The input parameter divMode only supports True.

Return Value

aclnnStatus: return code. For details, see aclnn Return Code.

The first-phase API performs input parameter validation and returns an error in the following scenarios:

Return Code	Error Code	Description
ACLNN_ERR_PARAM_NULLPTR	161001	The input parameter is a required input, output, or required attribute, and is a null pointer.
ACLNN_ERR_PARAM_INVALID	161002

		The data type of the input or output is not within the supported range.

aclnnAddRmsNormQuant

Parameters

Parameter	Input/Output	Description
workspace	Input	Workspace memory address applied on the Device side.
workspaceSize	Input	Workspace size applied on the Device side, obtained from the first-phase API aclnnAddRmsNormQuantGetWorkspaceSize.
executor	Input	Operator executor, which contains the operator computation flow.
stream	Input	Specifies the stream for task execution.

Return Value

aclnnStatus: return code. For details, see aclnn Return Code.

Constraints

Atlas inference products: The norm axis length of x1, x2, y1Out, y2Out, and xOut, as well as the length of gamma, scales1, scales2Optional, zeroPoints1Optional, and zeroPoints2Optional, must be greater than or equal to 32 bytes.
Data format description:

It is recommended to use the ND format for all input and output tensors. Other data formats are converted to the ND format by the framework by default for processing.

Supported data types by product model:

Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products:

x1 Data Type	x2 Data Type	gamma Data Type	scales1 Data Type	scales2Optional Data Type	zeroPoints1Optional Data Type	zeroPoints2Optional Data Type	y1Out Data Type	y2Out Data Type	xOut Data Type
FLOAT16	FLOAT16	FLOAT16	FLOAT32	FLOAT32	INT32	INT32	INT8	INT8	FLOAT16
BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	INT8	INT8	BFLOAT16

Atlas inference products:

x1 Data Type	x2 Data Type	gamma Data Type	scales1 Data Type	scales2Optional Data Type	zeroPoints1Optional Data Type	zeroPoints2Optional Data Type	y1Out Data Type	y2Out Data Type	xOut Data Type
FLOAT16	FLOAT16	FLOAT16	FLOAT32	FLOAT32	INT32	INT32	INT8	INT8	FLOAT16

Ascend 950PR/Ascend 950DT:

x1 Data Type	x2 Data Type	gamma Data Type	scales1 Data Type	scales2Optional Data Type	zeroPoints1Optional Data Type	zeroPoints2Optional Data Type	y1Out Data Type	y2Out Data Type	xOut Data Type
FLOAT16	FLOAT16	FLOAT16	FLOAT32	FLOAT32	INT32	INT32	INT8	INT8	FLOAT16
BFLOAT16	BFLOAT16	BFLOAT16	FLOAT32	FLOAT32	INT32	INT32	INT8	INT8	BFLOAT16
FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT32	INT8	INT8	FLOAT32
FLOAT16	FLOAT16	FLOAT16	FLOAT16	FLOAT16	FLOAT16	FLOAT16	INT8	INT8	FLOAT16
BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	INT8	INT8	BFLOAT16
FLOAT16	FLOAT16	FLOAT16	FLOAT32	FLOAT32	FLOAT32	FLOAT32	INT8	INT8	FLOAT16
BFLOAT16	BFLOAT16	BFLOAT16	FLOAT32	FLOAT32	FLOAT32	FLOAT32	INT8	INT8	BFLOAT16
FLOAT16	FLOAT16	FLOAT16	FLOAT32	FLOAT32	INT32	INT32	HIFLOAT8	HIFLOAT8	FLOAT16
BFLOAT16	BFLOAT16	BFLOAT16	FLOAT32	FLOAT32	INT32	INT32	HIFLOAT8	HIFLOAT8	BFLOAT16
FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT32	HIFLOAT8	HIFLOAT8	FLOAT32
FLOAT16	FLOAT16	FLOAT16	FLOAT16	FLOAT16	FLOAT16	FLOAT16	HIFLOAT8	HIFLOAT8	FLOAT16
BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	HIFLOAT8	HIFLOAT8	BFLOAT16
FLOAT16	FLOAT16	FLOAT16	FLOAT32	FLOAT32	FLOAT32	FLOAT32	HIFLOAT8	HIFLOAT8	FLOAT16
BFLOAT16	BFLOAT16	BFLOAT16	FLOAT32	FLOAT32	FLOAT32	FLOAT32	HIFLOAT8	HIFLOAT8	BFLOAT16
FLOAT16	FLOAT16	FLOAT16	FLOAT32	FLOAT32	INT32	INT32	FLOAT8_E5M2	FLOAT8_E5M2	FLOAT16
BFLOAT16	BFLOAT16	BFLOAT16	FLOAT32	FLOAT32	INT32	INT32	FLOAT8_E5M2	FLOAT8_E5M2	BFLOAT16
FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT8_E5M2	FLOAT8_E5M2	FLOAT32
FLOAT16	FLOAT16	FLOAT16	FLOAT16	FLOAT16	FLOAT16	FLOAT16	FLOAT8_E5M2	FLOAT8_E5M2	FLOAT16
BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	FLOAT8_E5M2	FLOAT8_E5M2	BFLOAT16
FLOAT16	FLOAT16	FLOAT16	FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT8_E5M2	FLOAT8_E5M2	FLOAT16
BFLOAT16	BFLOAT16	BFLOAT16	FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT8_E5M2	FLOAT8_E5M2	BFLOAT16
FLOAT16	FLOAT16	FLOAT16	FLOAT32	FLOAT32	INT32	INT32	FLOAT8_E4M3FN	FLOAT8_E4M3FN	FLOAT16
BFLOAT16	BFLOAT16	BFLOAT16	FLOAT32	FLOAT32	INT32	INT32	FLOAT8_E4M3FN	FLOAT8_E4M3FN	BFLOAT16
FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT8_E4M3FN	FLOAT8_E4M3FN	FLOAT32
FLOAT16	FLOAT16	FLOAT16	FLOAT16	FLOAT16	FLOAT16	FLOAT16	FLOAT8_E4M3FN	FLOAT8_E4M3FN	FLOAT16
BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	BFLOAT16	FLOAT8_E4M3FN	FLOAT8_E4M3FN	BFLOAT16
FLOAT16	FLOAT16	FLOAT16	FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT8_E4M3FN	FLOAT8_E4M3FN	FLOAT16
BFLOAT16	BFLOAT16	BFLOAT16	FLOAT32	FLOAT32	FLOAT32	FLOAT32	FLOAT8_E4M3FN	FLOAT8_E4M3FN	BFLOAT16

Deterministic computation:
- aclnnAddRmsNormQuant defaults to a deterministic implementation.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_add_rms_norm_quant.h"

#define CHECK_RET(cond, return_expr) \
    do {                             \
        if (!(cond)) {               \
            return_expr;             \
        }                            \
    } while (0)

#define LOG_PRINT(message, ...)         \
    do {                                \
        printf(message, ##__VA_ARGS__); \
    } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape)
{
    int64_t shape_size = 1;
    for (auto i : shape) {
        shape_size *= i;
    }
    return shape_size;
}

int Init(int32_t deviceId, aclrtStream* stream)
{
    // Boilerplate for resource initialization.
    auto ret = aclInit(nullptr);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
    ret = aclrtSetDevice(deviceId);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
    ret = aclrtCreateStream(stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);

    return 0;
}

template <typename T>
int CreateAclTensor(
    const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr, aclDataType dataType,
    aclTensor** tensor)
{
    auto size = GetShapeSize(shape) * sizeof(T);
    // Call aclrtMalloc to apply for device-side memory.
    auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);

    // Call aclrtMemcpy to copy host-side data to the device-side memory.
    ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

    // Compute the strides of the contiguous tensor.
    std::vector<int64_t> strides(shape.size(), 1);
    for (int64_t i = shape.size() - 2; i >= 0; i--) {
        strides[i] = shape[i + 1] * strides[i + 1];
    }

    // Call the aclCreateTensor API to create an aclTensor.
    *tensor = aclCreateTensor(
        shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND, shape.data(), shape.size(),
        *deviceAddr);
    return 0;
}

int main()
{
    // 1. (Boilerplate) Initialize the device/stream. See the acl API manual.
    // Fill in the deviceId based on your actual device.
    int32_t deviceId = 0;
    aclrtStream stream;
    auto ret = Init(deviceId, &stream);
    // Handle the check based on your needs.
    CHECK_RET(ret == 0, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);
    // 2. Construct the input and output. Customize the construction based on the API.
    std::vector<int64_t> xShape = {64, 2};
    std::vector<int64_t> gammaShape = {2};
    std::vector<int64_t> yShape = {64, 2};
    long long xShapeSize = GetShapeSize(xShape);
    long long gammaShapeSize = GetShapeSize(gammaShape);
    void* x1DeviceAddr = nullptr;
    void* x2DeviceAddr = nullptr;
    void* gammaDeviceAddr = nullptr;
    void* scales1DeviceAddr = nullptr;
    void* zeroPoints1DeviceAddr = nullptr;
    void* y1DeviceAddr = nullptr;
    void* y2DeviceAddr = nullptr;
    void* xDeviceAddr = nullptr;
    aclTensor* x1 = nullptr;
    aclTensor* x2 = nullptr;
    aclTensor* gamma = nullptr;
    aclTensor* scales1 = nullptr;
    aclTensor* zeroPoints1 = nullptr;
    aclTensor* y1 = nullptr;
    aclTensor* y2 = nullptr;
    aclTensor* x = nullptr;
    std::vector<int16_t> x1HostData(xShapeSize, 0);
    std::vector<int16_t> x2HostData(xShapeSize, 0);
    std::vector<int16_t> gammaHostData(gammaShapeSize, 0);
    std::vector<float> scales1HostData(gammaShapeSize, 1);
    std::vector<int32_t> zeroPoints1HostData(gammaShapeSize, 100);
    std::vector<int8_t> y1HostData(xShapeSize, 0);
    std::vector<int8_t> y2HostData(xShapeSize, 0);
    std::vector<int16_t> xHostData(xShapeSize, 0);
    float epsilon = 1e-6;
    int64_t axis = -1;
    bool divMode = true;
    // Create the x1 aclTensor.
    ret = CreateAclTensor(x1HostData, xShape, &x1DeviceAddr, aclDataType::ACL_FLOAT16, &x1);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the x2 aclTensor.
    ret = CreateAclTensor(x2HostData, xShape, &x2DeviceAddr, aclDataType::ACL_FLOAT16, &x2);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the gamma aclTensor.
    ret = CreateAclTensor(gammaHostData, gammaShape, &gammaDeviceAddr, aclDataType::ACL_FLOAT16, &gamma);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the scales1 aclTensor.
    ret = CreateAclTensor(scales1HostData, gammaShape, &scales1DeviceAddr, aclDataType::ACL_FLOAT, &scales1);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the zeroPoints1 aclTensor.
    ret =
        CreateAclTensor(zeroPoints1HostData, gammaShape, &zeroPoints1DeviceAddr, aclDataType::ACL_INT32, &zeroPoints1);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the y1 aclTensor.
    ret = CreateAclTensor(y1HostData, yShape, &y1DeviceAddr, aclDataType::ACL_INT8, &y1);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the y2 aclTensor.
    ret = CreateAclTensor(y2HostData, yShape, &y2DeviceAddr, aclDataType::ACL_INT8, &y2);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the x aclTensor.
    ret = CreateAclTensor(xHostData, xShape, &xDeviceAddr, aclDataType::ACL_FLOAT16, &x);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // 3. Call the CANN operator library API. Replace it with the specific API.
    uint64_t workspaceSize = 0;
    aclOpExecutor* executor;
    // Call the aclnnAddRmsNormQuant first-phase API.
    ret = aclnnAddRmsNormQuantGetWorkspaceSize(
        x1, x2, gamma, scales1, nullptr, zeroPoints1, nullptr, axis, epsilon, divMode, y1, y2, x, &workspaceSize,
        &executor);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnAddRmsNormQuantGetWorkspaceSize failed. ERROR: %d\n", ret); return ret);
    // Apply for device memory based on the workspaceSize calculated by the first-phase API.
    void* workspaceAddr = nullptr;
    if (workspaceSize > 0) {
        ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
        CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret;);
    }
    // Call the aclnnAddRmsNormQuant second-phase API.
    ret = aclnnAddRmsNormQuant(workspaceAddr, workspaceSize, executor, stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnAddRmsNormQuant failed. ERROR: %d\n", ret); return ret);
    // 4. (Boilerplate) Synchronously wait for task execution to complete.
    ret = aclrtSynchronizeStream(stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);
    // 5. Obtain the output value and copy the result from the device-side memory to the host side. Modify this based on the specific API definition.
    auto size = GetShapeSize(yShape);
    std::vector<int8_t> resultData(size, 0);
    ret = aclrtMemcpy(
        resultData.data(), resultData.size() * sizeof(resultData[0]), y1DeviceAddr, size * sizeof(int8_t),
        ACL_MEMCPY_DEVICE_TO_HOST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
    for (int64_t i = 0; i < size; i++) {
        LOG_PRINT("result[%ld] is: %d\n", i, resultData[i]);
    }

    // 6. Release aclTensor and aclScalar. Modify based on the specific API definition.
    aclDestroyTensor(x1);
    aclDestroyTensor(x2);
    aclDestroyTensor(gamma);
    aclDestroyTensor(scales1);
    aclDestroyTensor(zeroPoints1);
    aclDestroyTensor(y1);
    aclDestroyTensor(y2);
    aclDestroyTensor(x);

    // 7. Release device resources. Modify based on the specific API definition.
    aclrtFree(x1DeviceAddr);
    aclrtFree(x2DeviceAddr);
    aclrtFree(gammaDeviceAddr);
    aclrtFree(scales1DeviceAddr);
    aclrtFree(zeroPoints1DeviceAddr);
    aclrtFree(y1DeviceAddr);
    aclrtFree(y2DeviceAddr);
    aclrtFree(xDeviceAddr);

    if (workspaceSize > 0) {
        aclrtFree(workspaceAddr);
    }
    aclrtDestroyStream(stream);
    aclrtResetDevice(deviceId);
    aclFinalize();
    return 0;
}